2024-05-11T13:03:28.068 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-11T13:03:28.068 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405 --name rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} -- /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/orig.config.yaml 2024-05-11T13:03:28.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T13:03:28.235 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405 branch: wip-rishabh-testing-20240510.053205 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} email: null first_in_suite: false job_id: '7702405' kernel: flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240510.053205 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240510.053205 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4085 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e sleep_before_teardown: 0 subset: 100/512 suite: fs suite_branch: wip-rishabh-testing-20240510.053205 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDAIQfT060MUdqdBJeQ4DHql6Zr6tNs4r12T40wWYlJTXp8gsVZYbtis+zPMaknLBeq/8ZexvsIqz+eUmvY7QwA= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyuJkuhlAcYw/ORlvJ4AJrOkkqAPOM44/Dvo1MbF+OajJZolDUPkImn4u//Hm5TFKUIGUyGI1s4MJ7XrdAiggY= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAiR+fnfSCNl996TyyumW/Cttfo0ZCs3q6GzxYYhzvOfLP3jKCeBuNeIkRsJ7jZSS+YVy+YCWzNUzP70IBtVNkU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-11_05:57:22 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-11T13:03:28.236 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa; will attempt to use it 2024-05-11T13:03:28.237 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa/tasks 2024-05-11T13:03:28.237 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-11T13:03:28.239 INFO:teuthology.task.internal:Checking packages... 2024-05-11T13:03:28.264 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '308506802b0bdea4df7050cfc1fcbdc04fea6a6e' 2024-05-11T13:03:28.264 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-11T13:03:28.265 INFO:teuthology.packaging:ref: None 2024-05-11T13:03:28.265 INFO:teuthology.packaging:tag: None 2024-05-11T13:03:28.265 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240510.053205 2024-05-11T13:03:28.265 INFO:teuthology.packaging:sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:03:28.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240510.053205 2024-05-11T13:03:28.565 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3601.g30850680 2024-05-11T13:03:28.567 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-11T13:03:28.573 INFO:teuthology.task.internal:no buildpackages task found 2024-05-11T13:03:28.573 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-11T13:03:28.590 INFO:teuthology.task.internal:Saving configuration 2024-05-11T13:03:28.613 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-11T13:03:28.618 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T13:03:28.642 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 12:58:23.651338', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDAIQfT060MUdqdBJeQ4DHql6Zr6tNs4r12T40wWYlJTXp8gsVZYbtis+zPMaknLBeq/8ZexvsIqz+eUmvY7QwA='} 2024-05-11T13:03:28.666 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 12:58:23.647911', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyuJkuhlAcYw/ORlvJ4AJrOkkqAPOM44/Dvo1MbF+OajJZolDUPkImn4u//Hm5TFKUIGUyGI1s4MJ7XrdAiggY='} 2024-05-11T13:03:28.690 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 12:58:23.649839', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAiR+fnfSCNl996TyyumW/Cttfo0ZCs3q6GzxYYhzvOfLP3jKCeBuNeIkRsJ7jZSS+YVy+YCWzNUzP70IBtVNkU='} 2024-05-11T13:03:28.690 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-11T13:03:28.696 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-11T13:03:28.696 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-11T13:03:28.696 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-11T13:03:28.696 INFO:teuthology.run_tasks:Running task console_log... 2024-05-11T13:03:28.799 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdb03781160>, signals=[15]) 2024-05-11T13:03:28.800 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-11T13:03:28.808 INFO:teuthology.task.internal:Opening connections... 2024-05-11T13:03:28.808 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:03:28.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:03:28.928 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-05-11T13:03:28.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:03:29.008 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-05-11T13:03:29.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:03:29.082 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-11T13:03:29.088 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-05-11T13:03:29.106 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-05-11T13:03:29.110 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-05-11T13:03:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-05-11T13:03:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-05-11T13:03:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-05-11T13:03:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-05-11T13:03:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-05-11T13:03:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-05-11T13:03:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T13:03:29.166 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-05-11T13:03:29.166 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-05-11T13:03:29.166 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T13:03:29.166 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-05-11T13:03:29.166 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T13:03:29.166 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T13:03:29.166 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T13:03:29.167 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-05-11T13:03:29.190 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-05-11T13:03:29.207 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-05-11T13:03:29.208 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-05-11T13:03:29.264 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-05-11T13:03:29.264 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-05-11T13:03:29.264 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-05-11T13:03:29.264 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-05-11T13:03:29.264 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T13:03:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T13:03:29.265 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-05-11T13:03:29.281 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-05-11T13:03:29.299 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-05-11T13:03:29.299 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-05-11T13:03:29.356 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-05-11T13:03:29.356 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-05-11T13:03:29.356 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-05-11T13:03:29.356 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-05-11T13:03:29.356 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T13:03:29.357 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T13:03:29.357 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-05-11T13:03:29.381 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-11T13:03:29.392 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-11T13:03:29.397 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-11T13:03:29.397 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-05-11T13:03:29.400 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-05-11T13:03:29.402 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-05-11T13:03:29.418 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-11T13:03:29.425 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-11T13:03:29.426 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-05-11T13:03:29.457 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-05-11T13:03:29.460 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-05-11T13:03:29.575 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-11T13:03:29.653 INFO:teuthology.run_tasks:Running task kernel... 2024-05-11T13:03:29.668 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-11T13:03:29.668 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}}, timeout 300 2024-05-11T13:03:29.668 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T13:03:29.669 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T13:03:29.670 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T13:03:29.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:03:29.688 DEBUG:teuthology.task.kernel:sha1 for mon.b is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.688 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-11T13:03:29.688 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-05-11T13:03:29.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:03:29.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:03:29.691 DEBUG:teuthology.task.kernel:sha1 for mon.a is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.691 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-11T13:03:29.692 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-11T13:03:29.692 DEBUG:teuthology.task.kernel:sha1 for mon.c is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.692 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-11T13:03:29.693 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-05-11T13:03:29.745 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-437.el9.x86_64 2024-05-11T13:03:29.745 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.745 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-11T13:03:29.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.750 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-437.el9.x86_64 2024-05-11T13:03:29.750 INFO:teuthology.orchestra.run.smithi165.stdout:5.14.0-437.el9.x86_64 2024-05-11T13:03:29.750 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.750 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-11T13:03:29.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.751 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.752 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-11T13:03:29.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:29.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum upgrade -y linux-firmware 2024-05-11T13:03:29.905 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum upgrade -y linux-firmware 2024-05-11T13:03:29.907 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum upgrade -y linux-firmware 2024-05-11T13:03:31.927 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-11T13:03:31.967 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-11T13:03:32.183 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-11T13:03:33.720 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-11T13:03:34.042 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-11T13:03:39.734 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-11T13:03:40.462 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-05-11T13:03:41.468 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-05-11T13:03:42.004 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-11T13:03:42.619 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-11T13:03:47.735 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-11T13:03:49.841 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-05-11T13:03:49.842 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-05-11T13:03:49.842 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-05-11T13:03:49.949 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.c... 2024-05-11T13:03:49.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:03:50.092 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-11T13:03:50.092 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-11T13:03:50.094 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi165.front.sepia.ceph.com to complete... 2024-05-11T13:03:50.574 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-05-11T13:03:54.717 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-11T13:03:56.458 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-11T13:03:58.129 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-11T13:04:01.299 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-05-11T13:04:03.284 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-11T13:04:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11 13:04:03 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-11T13:04:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:FINISHED --2024-05-11 13:04:03-- 2024-05-11T13:04:03.388 INFO:teuthology.orchestra.run.smithi165.stderr:Total wall clock time: 13s 2024-05-11T13:04:03.388 INFO:teuthology.orchestra.run.smithi165.stderr:Downloaded: 1 files, 341M in 13s (26.2 MB/s) 2024-05-11T13:04:03.390 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.c... 2024-05-11T13:04:03.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-11T13:04:05.380 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-05-11T13:04:05.381 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-05-11T13:04:05.381 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-05-11T13:04:05.477 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.b... 2024-05-11T13:04:05.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:04:05.488 INFO:teuthology.orchestra.run.smithi165.stdout:Verifying... ######################################## 2024-05-11T13:04:05.626 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-11T13:04:05.627 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-11T13:04:05.629 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi071.front.sepia.ceph.com to complete... 2024-05-11T13:04:05.686 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing... ######################################## 2024-05-11T13:04:05.821 INFO:teuthology.orchestra.run.smithi165.stdout:Updating / installing... 2024-05-11T13:04:06.630 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-11T13:04:08.839 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-11T13:04:08.840 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-11T13:04:08.840 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-11T13:04:08.942 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.a... 2024-05-11T13:04:08.943 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T13:04:09.176 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-11T13:04:09.177 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-11T13:04:09.179 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi063.front.sepia.ceph.com to complete... 2024-05-11T13:04:13.819 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-11T13:04:31.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11 13:04:31 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-11T13:04:31.393 INFO:teuthology.orchestra.run.smithi071.stderr:FINISHED --2024-05-11 13:04:31-- 2024-05-11T13:04:31.393 INFO:teuthology.orchestra.run.smithi071.stderr:Total wall clock time: 26s 2024-05-11T13:04:31.393 INFO:teuthology.orchestra.run.smithi071.stderr:Downloaded: 1 files, 341M in 25s (13.4 MB/s) 2024-05-11T13:04:31.396 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.b... 2024-05-11T13:04:31.397 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-11T13:04:33.490 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying... ######################################## 2024-05-11T13:04:33.706 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing... ######################################## 2024-05-11T13:04:33.818 INFO:teuthology.orchestra.run.smithi071.stdout:Updating / installing... 2024-05-11T13:04:41.776 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-11T13:04:50.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11 13:04:50 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-11T13:04:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:FINISHED --2024-05-11 13:04:50-- 2024-05-11T13:04:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:Total wall clock time: 42s 2024-05-11T13:04:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:Downloaded: 1 files, 341M in 41s (8.32 MB/s) 2024-05-11T13:04:50.730 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.a... 2024-05-11T13:04:50.731 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-11T13:04:52.747 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi165.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-11T13:04:52.747 DEBUG:teuthology.orchestra.run.smithi165:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-11T13:04:52.823 INFO:teuthology.orchestra.run.smithi165.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:52.823 INFO:teuthology.orchestra.run.smithi165.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-11T13:04:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-11T13:04:52.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-11T13:04:52.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-11T13:04:52.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-11T13:04:52.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-11T13:04:52.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-11T13:04:52.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-11T13:04:52.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-11T13:04:52.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-11T13:04:52.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-11T13:04:52.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-11T13:04:52.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-11T13:04:52.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-11T13:04:52.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-11T13:04:52.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-11T13:04:52.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-11T13:04:52.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-11T13:04:52.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-11T13:04:52.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-11T13:04:52.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-11T13:04:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-11T13:04:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-11T13:04:52.846 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-11T13:04:52.847 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-11T13:04:52.848 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-11T13:04:52.849 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-11T13:04:52.850 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-11T13:04:52.851 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-11T13:04:52.852 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-11T13:04:52.853 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-11T13:04:52.854 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-11T13:04:52.855 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-11T13:04:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-11T13:04:52.857 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-11T13:04:52.858 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-11T13:04:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-11T13:04:52.860 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-11T13:04:52.861 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-11T13:04:52.862 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-11T13:04:52.863 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-11T13:04:52.864 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-11T13:04:52.865 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-11T13:04:52.866 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-11T13:04:52.867 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-11T13:04:52.868 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-11T13:04:52.869 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-11T13:04:52.870 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-11T13:04:52.871 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-11T13:04:52.872 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-11T13:04:52.873 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-11T13:04:52.874 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-11T13:04:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-11T13:04:52.876 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-11T13:04:52.877 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-11T13:04:52.878 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-11T13:04:52.879 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-11T13:04:52.880 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-11T13:04:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-11T13:04:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-11T13:04:52.883 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-11T13:04:52.884 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-11T13:04:52.885 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-11T13:04:52.886 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-11T13:04:52.887 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-11T13:04:52.888 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-11T13:04:52.889 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-11T13:04:52.890 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-11T13:04:52.891 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-11T13:04:52.892 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-11T13:04:52.893 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-11T13:04:52.894 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-11T13:04:52.895 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-11T13:04:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-11T13:04:52.897 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-11T13:04:52.898 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-11T13:04:52.899 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-11T13:04:52.900 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-11T13:04:52.901 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-11T13:04:52.902 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-11T13:04:52.903 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-11T13:04:52.904 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-11T13:04:52.905 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-11T13:04:52.906 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-11T13:04:52.907 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-11T13:04:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-11T13:04:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-11T13:04:52.910 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-11T13:04:52.911 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-11T13:04:52.912 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-11T13:04:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-11T13:04:52.914 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-11T13:04:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-11T13:04:52.916 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-11T13:04:52.917 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-11T13:04:52.918 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-11T13:04:52.919 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-11T13:04:52.920 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-11T13:04:52.921 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-11T13:04:52.922 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-11T13:04:52.923 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-11T13:04:52.924 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-11T13:04:52.925 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-11T13:04:52.926 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-11T13:04:52.927 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-11T13:04:52.928 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-11T13:04:52.929 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-11T13:04:52.930 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-11T13:04:52.931 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-11T13:04:52.932 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-11T13:04:52.933 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-11T13:04:52.934 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-11T13:04:52.935 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-11T13:04:52.936 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-11T13:04:52.937 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-11T13:04:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-11T13:04:52.939 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-11T13:04:52.940 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-11T13:04:52.941 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-11T13:04:52.942 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-11T13:04:52.943 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-11T13:04:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-11T13:04:52.945 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-11T13:04:52.946 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-11T13:04:52.947 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-11T13:04:52.948 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-11T13:04:52.949 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-11T13:04:52.950 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-11T13:04:52.951 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-11T13:04:52.952 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-11T13:04:52.953 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-11T13:04:52.954 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-11T13:04:52.955 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-11T13:04:52.956 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-11T13:04:52.957 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-11T13:04:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-11T13:04:52.959 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-11T13:04:52.960 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-11T13:04:52.961 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-11T13:04:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-11T13:04:52.963 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-11T13:04:52.964 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-11T13:04:52.964 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-11T13:04:52.964 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-11T13:04:52.964 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-11T13:04:52.964 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T13:04:52.968 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:52.969 DEBUG:teuthology.orchestra.run.smithi165:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-11T13:04:52.969 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying... ######################################## 2024-05-11T13:04:52.991 INFO:teuthology.orchestra.run.smithi165.stdout:postinstall scriptlet (using /bin/sh): 2024-05-11T13:04:52.991 INFO:teuthology.orchestra.run.smithi165.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout:for file in vmlinuz System.map config; do 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: fi 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout:done 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:52.992 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-05-11T13:04:52.993 INFO:teuthology.orchestra.run.smithi165.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-11T13:04:52.993 INFO:teuthology.orchestra.run.smithi165.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T13:04:52.993 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:52.993 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-05-11T13:04:52.994 DEBUG:teuthology.orchestra.run.smithi165:> sudo rpm -qi grub2-tools 2024-05-11T13:04:53.039 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing... ######################################## 2024-05-11T13:04:53.069 INFO:teuthology.orchestra.run.smithi165.stdout:Name : grub2-tools 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Epoch : 1 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Version : 2.06 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Release : 68.el9 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Architecture: x86_64 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Group : Unspecified 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Size : 8264254 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:License : GPLv3+ 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T13:04:53.070 INFO:teuthology.orchestra.run.smithi165.stdout:Packager : builder@centos.org 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:Vendor : CentOS 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:Summary : Support tools for GRUB. 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:Description : 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:hardware devices. 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:04:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T13:04:53.072 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T13:04:53.072 INFO:teuthology.task.kernel:Updating grub on smithi165 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:53.072 DEBUG:teuthology.orchestra.run.smithi165:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T13:04:53.168 INFO:teuthology.orchestra.run.smithi063.stdout:Updating / installing... 2024-05-11T13:04:53.661 INFO:teuthology.orchestra.run.smithi165.stderr:Generating grub configuration file ... 2024-05-11T13:04:55.129 INFO:teuthology.orchestra.run.smithi165.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T13:04:55.161 INFO:teuthology.orchestra.run.smithi165.stderr:done 2024-05-11T13:04:55.163 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-05-11T13:04:55.180 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.GSBMOvT2rS 2024-05-11T13:04:55.180 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GSBMOvT2rS 2024-05-11T13:04:55.270 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.GSBMOvT2rS 2024-05-11T13:04:55.373 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.GSBMOvT2rS is 6KB 2024-05-11T13:04:55.422 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.GSBMOvT2rS 2024-05-11T13:04:55.436 DEBUG:teuthology.orchestra.run.smithi165:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T13:04:55.498 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T13:04:55.498 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T13:04:55.498 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T13:04:55.498 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-11T13:04:55.500 DEBUG:teuthology.orchestra.run.smithi165:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:04:55.672 DEBUG:teuthology.orchestra.run.smithi165:> sudo shutdown -r now 2024-05-11T13:05:01.445 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-11T13:05:19.254 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-11T13:05:19.254 DEBUG:teuthology.orchestra.run.smithi071:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-11T13:05:19.334 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:19.334 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:19.334 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:19.334 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-11T13:05:19.335 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-11T13:05:19.335 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-11T13:05:19.335 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-11T13:05:19.335 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-11T13:05:19.335 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-11T13:05:19.336 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-11T13:05:19.336 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-11T13:05:19.336 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-11T13:05:19.336 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-11T13:05:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-11T13:05:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-11T13:05:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-11T13:05:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-11T13:05:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-11T13:05:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-11T13:05:19.338 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-11T13:05:19.338 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-11T13:05:19.338 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-11T13:05:19.338 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-11T13:05:19.338 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-11T13:05:19.338 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-11T13:05:19.339 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-11T13:05:19.339 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-11T13:05:19.339 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-11T13:05:19.339 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-11T13:05:19.339 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-11T13:05:19.339 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-11T13:05:19.340 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-11T13:05:19.340 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-11T13:05:19.340 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-11T13:05:19.340 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-11T13:05:19.340 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-11T13:05:19.340 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-11T13:05:19.340 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-11T13:05:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-11T13:05:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-11T13:05:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-11T13:05:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-11T13:05:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-11T13:05:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-11T13:05:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-11T13:05:19.342 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-11T13:05:19.342 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-11T13:05:19.342 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-11T13:05:19.342 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-11T13:05:19.342 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-11T13:05:19.343 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-11T13:05:19.343 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-11T13:05:19.343 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-11T13:05:19.343 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-11T13:05:19.343 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-11T13:05:19.343 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-11T13:05:19.344 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-11T13:05:19.344 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-11T13:05:19.344 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-11T13:05:19.344 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-11T13:05:19.344 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-11T13:05:19.344 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-11T13:05:19.344 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-11T13:05:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-11T13:05:19.346 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-11T13:05:19.346 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-11T13:05:19.346 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-11T13:05:19.346 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-11T13:05:19.346 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-11T13:05:19.346 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-11T13:05:19.347 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-11T13:05:19.348 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-11T13:05:19.349 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-11T13:05:19.350 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-11T13:05:19.351 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-11T13:05:19.352 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-11T13:05:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-11T13:05:19.354 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-11T13:05:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-11T13:05:19.356 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-11T13:05:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-11T13:05:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-11T13:05:19.359 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-11T13:05:19.359 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-11T13:05:19.359 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-11T13:05:19.359 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-11T13:05:19.359 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-11T13:05:19.362 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-11T13:05:19.362 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-11T13:05:19.362 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-11T13:05:19.362 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-11T13:05:19.363 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-11T13:05:19.364 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-11T13:05:19.365 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-11T13:05:19.366 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-11T13:05:19.367 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-11T13:05:19.368 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-11T13:05:19.369 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-11T13:05:19.369 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-11T13:05:19.369 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-11T13:05:19.369 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-11T13:05:19.369 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-11T13:05:19.369 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-11T13:05:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-11T13:05:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-11T13:05:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-11T13:05:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-11T13:05:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-11T13:05:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-11T13:05:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-11T13:05:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-11T13:05:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-11T13:05:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-11T13:05:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-11T13:05:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-11T13:05:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-11T13:05:19.377 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-11T13:05:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-11T13:05:19.379 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-11T13:05:19.380 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-11T13:05:19.381 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-11T13:05:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-11T13:05:19.383 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-11T13:05:19.384 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-11T13:05:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-11T13:05:19.386 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-11T13:05:19.387 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-11T13:05:19.388 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-11T13:05:19.388 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-11T13:05:19.388 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-11T13:05:19.388 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-11T13:05:19.388 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-11T13:05:19.388 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-11T13:05:19.388 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-11T13:05:19.389 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-11T13:05:19.390 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-11T13:05:19.391 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-11T13:05:19.391 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-11T13:05:19.391 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-11T13:05:19.391 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-11T13:05:19.391 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-11T13:05:19.392 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-11T13:05:19.392 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-11T13:05:19.392 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-11T13:05:19.392 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-11T13:05:19.392 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-11T13:05:19.392 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-11T13:05:19.393 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-11T13:05:19.393 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-11T13:05:19.393 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-11T13:05:19.393 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-11T13:05:19.393 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-11T13:05:19.393 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-11T13:05:19.394 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-11T13:05:19.395 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-11T13:05:19.395 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-11T13:05:19.395 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-11T13:05:19.395 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-11T13:05:19.395 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-11T13:05:19.395 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-11T13:05:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-11T13:05:19.397 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-11T13:05:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-11T13:05:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-11T13:05:19.400 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-11T13:05:19.401 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-11T13:05:19.402 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-11T13:05:19.403 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-11T13:05:19.404 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-11T13:05:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-11T13:05:19.406 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-11T13:05:19.407 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-11T13:05:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-11T13:05:19.409 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-11T13:05:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-11T13:05:19.411 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-11T13:05:19.412 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-11T13:05:19.413 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-11T13:05:19.414 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-11T13:05:19.415 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-11T13:05:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-11T13:05:19.417 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-11T13:05:19.418 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-11T13:05:19.418 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-11T13:05:19.418 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-11T13:05:19.418 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-11T13:05:19.418 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-11T13:05:19.418 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-11T13:05:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-11T13:05:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-11T13:05:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-11T13:05:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-11T13:05:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-11T13:05:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-11T13:05:19.420 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-11T13:05:19.421 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-11T13:05:19.422 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-11T13:05:19.423 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-11T13:05:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-11T13:05:19.425 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-11T13:05:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-11T13:05:19.427 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-11T13:05:19.428 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-11T13:05:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-11T13:05:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-11T13:05:19.431 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-11T13:05:19.432 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-11T13:05:19.433 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-11T13:05:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-11T13:05:19.435 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-11T13:05:19.436 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-11T13:05:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-11T13:05:19.438 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-11T13:05:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-11T13:05:19.440 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-11T13:05:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-11T13:05:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-11T13:05:19.443 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-11T13:05:19.444 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-11T13:05:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-11T13:05:19.446 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-11T13:05:19.447 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-11T13:05:19.448 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-11T13:05:19.449 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-11T13:05:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-11T13:05:19.451 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-11T13:05:19.452 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-11T13:05:19.453 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-11T13:05:19.454 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-11T13:05:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-11T13:05:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-11T13:05:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-11T13:05:19.458 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-11T13:05:19.458 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-11T13:05:19.458 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-11T13:05:19.458 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-11T13:05:19.458 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-11T13:05:19.458 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-11T13:05:19.459 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-11T13:05:19.459 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-11T13:05:19.459 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-11T13:05:19.459 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-11T13:05:19.459 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-11T13:05:19.459 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-11T13:05:19.459 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-11T13:05:19.460 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-11T13:05:19.460 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-11T13:05:19.460 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-11T13:05:19.460 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-11T13:05:19.460 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-11T13:05:19.460 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-11T13:05:19.461 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-11T13:05:19.461 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-11T13:05:19.461 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-11T13:05:19.461 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-11T13:05:19.461 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-11T13:05:19.461 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-11T13:05:19.462 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-11T13:05:19.462 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-11T13:05:19.462 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-11T13:05:19.462 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-11T13:05:19.462 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-11T13:05:19.462 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-11T13:05:19.463 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-11T13:05:19.463 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-11T13:05:19.463 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-11T13:05:19.463 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-11T13:05:19.463 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-11T13:05:19.463 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-11T13:05:19.464 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-11T13:05:19.464 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-11T13:05:19.464 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-11T13:05:19.464 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-11T13:05:19.464 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-11T13:05:19.464 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-11T13:05:19.465 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-11T13:05:19.465 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-11T13:05:19.465 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-11T13:05:19.465 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-11T13:05:19.465 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-11T13:05:19.465 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-11T13:05:19.466 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-11T13:05:19.466 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-11T13:05:19.466 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-11T13:05:19.466 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-11T13:05:19.466 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-11T13:05:19.466 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-11T13:05:19.467 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-11T13:05:19.467 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-11T13:05:19.467 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-11T13:05:19.467 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-11T13:05:19.467 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-11T13:05:19.467 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-11T13:05:19.468 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-11T13:05:19.468 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-11T13:05:19.468 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-11T13:05:19.468 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-11T13:05:19.468 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-11T13:05:19.468 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-11T13:05:19.469 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-11T13:05:19.469 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-11T13:05:19.469 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-11T13:05:19.469 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-11T13:05:19.469 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-11T13:05:19.469 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-11T13:05:19.470 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-11T13:05:19.470 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-11T13:05:19.470 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-11T13:05:19.470 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-11T13:05:19.470 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-11T13:05:19.470 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-11T13:05:19.471 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-11T13:05:19.471 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-11T13:05:19.471 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-11T13:05:19.471 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-11T13:05:19.471 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-11T13:05:19.471 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-11T13:05:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-11T13:05:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-11T13:05:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-11T13:05:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-11T13:05:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-11T13:05:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-11T13:05:19.473 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-11T13:05:19.473 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-11T13:05:19.473 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-11T13:05:19.473 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-11T13:05:19.473 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-11T13:05:19.473 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-11T13:05:19.474 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-11T13:05:19.474 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-11T13:05:19.474 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-11T13:05:19.474 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-11T13:05:19.474 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-11T13:05:19.474 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-11T13:05:19.474 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-11T13:05:19.475 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-11T13:05:19.475 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-11T13:05:19.475 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-11T13:05:19.475 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-11T13:05:19.475 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-11T13:05:19.475 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-11T13:05:19.476 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-11T13:05:19.476 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-11T13:05:19.476 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-11T13:05:19.476 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-11T13:05:19.476 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-11T13:05:19.476 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-11T13:05:19.477 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-11T13:05:19.477 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-11T13:05:19.477 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-11T13:05:19.477 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-11T13:05:19.477 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-11T13:05:19.477 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-11T13:05:19.478 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-11T13:05:19.478 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-11T13:05:19.478 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-11T13:05:19.478 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-11T13:05:19.478 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-11T13:05:19.478 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-11T13:05:19.479 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-11T13:05:19.479 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-11T13:05:19.479 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-11T13:05:19.479 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-11T13:05:19.479 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-11T13:05:19.479 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-11T13:05:19.480 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-11T13:05:19.480 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-11T13:05:19.480 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-11T13:05:19.480 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-11T13:05:19.480 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-11T13:05:19.481 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-11T13:05:19.481 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-11T13:05:19.481 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-11T13:05:19.481 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-11T13:05:19.481 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-11T13:05:19.481 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-11T13:05:19.482 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-11T13:05:19.482 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-11T13:05:19.482 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-11T13:05:19.482 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-11T13:05:19.482 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-11T13:05:19.482 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-11T13:05:19.483 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-11T13:05:19.483 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-11T13:05:19.483 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-11T13:05:19.483 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-11T13:05:19.483 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-11T13:05:19.483 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-11T13:05:19.484 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-11T13:05:19.484 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-11T13:05:19.484 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-11T13:05:19.484 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-11T13:05:19.484 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-11T13:05:19.484 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-11T13:05:19.485 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-11T13:05:19.485 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-11T13:05:19.485 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-11T13:05:19.485 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-11T13:05:19.485 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-11T13:05:19.485 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-11T13:05:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-11T13:05:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-11T13:05:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-11T13:05:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-11T13:05:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-11T13:05:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-11T13:05:19.487 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-11T13:05:19.487 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-11T13:05:19.487 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-11T13:05:19.487 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-11T13:05:19.487 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-11T13:05:19.487 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-11T13:05:19.488 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-11T13:05:19.488 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-11T13:05:19.488 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-11T13:05:19.488 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-11T13:05:19.488 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-11T13:05:19.488 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-11T13:05:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-11T13:05:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-11T13:05:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-11T13:05:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-11T13:05:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-11T13:05:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-11T13:05:19.490 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-11T13:05:19.490 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-11T13:05:19.490 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-11T13:05:19.490 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-11T13:05:19.490 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-11T13:05:19.491 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-11T13:05:19.491 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-11T13:05:19.491 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-11T13:05:19.491 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-11T13:05:19.491 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-11T13:05:19.491 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-11T13:05:19.492 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-11T13:05:19.492 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-11T13:05:19.492 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-11T13:05:19.492 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-11T13:05:19.492 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-11T13:05:19.492 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-11T13:05:19.493 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-11T13:05:19.493 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-11T13:05:19.493 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-11T13:05:19.493 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-11T13:05:19.493 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-11T13:05:19.493 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-11T13:05:19.494 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-11T13:05:19.494 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-11T13:05:19.494 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-11T13:05:19.494 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-11T13:05:19.494 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-11T13:05:19.494 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-11T13:05:19.495 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-11T13:05:19.495 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-11T13:05:19.495 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-11T13:05:19.495 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-11T13:05:19.495 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-11T13:05:19.495 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-11T13:05:19.496 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-11T13:05:19.496 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-11T13:05:19.496 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-11T13:05:19.496 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-11T13:05:19.496 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-11T13:05:19.496 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-11T13:05:19.497 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-11T13:05:19.497 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-11T13:05:19.497 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-11T13:05:19.497 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-11T13:05:19.497 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-11T13:05:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-11T13:05:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-11T13:05:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-11T13:05:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-11T13:05:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-11T13:05:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-11T13:05:19.499 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-11T13:05:19.499 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-11T13:05:19.499 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-11T13:05:19.499 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-11T13:05:19.499 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-11T13:05:19.500 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-11T13:05:19.500 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-11T13:05:19.500 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-11T13:05:19.500 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-11T13:05:19.500 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-11T13:05:19.500 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-11T13:05:19.501 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-11T13:05:19.501 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-11T13:05:19.501 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-11T13:05:19.501 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-11T13:05:19.501 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-11T13:05:19.501 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-11T13:05:19.502 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-11T13:05:19.502 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-11T13:05:19.502 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-11T13:05:19.502 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-11T13:05:19.502 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-11T13:05:19.503 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-11T13:05:19.503 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-11T13:05:19.503 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-11T13:05:19.503 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-11T13:05:19.503 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-11T13:05:19.503 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-11T13:05:19.504 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-11T13:05:19.504 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-11T13:05:19.504 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-11T13:05:19.504 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-11T13:05:19.504 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-11T13:05:19.504 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-11T13:05:19.505 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-11T13:05:19.505 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-11T13:05:19.505 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-11T13:05:19.505 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-11T13:05:19.505 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-11T13:05:19.505 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-11T13:05:19.506 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-11T13:05:19.506 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-11T13:05:19.506 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-11T13:05:19.506 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-11T13:05:19.506 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-11T13:05:19.507 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-11T13:05:19.507 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-11T13:05:19.507 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-11T13:05:19.507 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-11T13:05:19.507 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-11T13:05:19.507 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-11T13:05:19.508 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-11T13:05:19.509 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-11T13:05:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-11T13:05:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-11T13:05:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-11T13:05:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-11T13:05:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-11T13:05:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-11T13:05:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-11T13:05:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-11T13:05:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-11T13:05:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-11T13:05:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-11T13:05:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-11T13:05:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-11T13:05:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-11T13:05:19.512 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-11T13:05:19.512 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-11T13:05:19.512 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-11T13:05:19.512 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-11T13:05:19.512 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-11T13:05:19.512 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-11T13:05:19.513 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-11T13:05:19.514 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-11T13:05:19.515 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-11T13:05:19.516 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-11T13:05:19.517 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-11T13:05:19.518 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-11T13:05:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-11T13:05:19.520 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-11T13:05:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-11T13:05:19.522 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-11T13:05:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-11T13:05:19.524 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-11T13:05:19.525 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-11T13:05:19.526 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-11T13:05:19.527 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-11T13:05:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-11T13:05:19.529 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-11T13:05:19.530 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-11T13:05:19.530 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-11T13:05:19.530 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-11T13:05:19.530 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-11T13:05:19.530 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-11T13:05:19.530 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-11T13:05:19.530 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-11T13:05:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T13:05:19.535 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:19.536 DEBUG:teuthology.orchestra.run.smithi071:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-11T13:05:19.559 INFO:teuthology.orchestra.run.smithi071.stdout:postinstall scriptlet (using /bin/sh): 2024-05-11T13:05:19.559 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T13:05:19.559 INFO:teuthology.orchestra.run.smithi071.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T13:05:19.559 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-05-11T13:05:19.559 INFO:teuthology.orchestra.run.smithi071.stdout:for file in vmlinuz System.map config; do 2024-05-11T13:05:19.559 INFO:teuthology.orchestra.run.smithi071.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout: fi 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:done 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-05-11T13:05:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-11T13:05:19.561 INFO:teuthology.orchestra.run.smithi071.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:19.561 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-05-11T13:05:19.561 INFO:teuthology.orchestra.run.smithi071.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-11T13:05:19.561 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T13:05:19.561 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:19.561 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-05-11T13:05:19.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-05-11T13:05:19.635 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-05-11T13:05:19.635 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-05-11T13:05:19.635 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 68.el9 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8264254 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-05-11T13:05:19.636 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-05-11T13:05:19.637 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:05:19.637 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T13:05:19.637 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T13:05:19.637 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T13:05:19.637 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-05-11T13:05:19.637 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:05:19.637 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T13:05:19.638 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T13:05:19.638 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:19.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T13:05:20.176 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-05-11T13:05:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T13:05:21.468 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-05-11T13:05:21.470 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-05-11T13:05:21.500 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.IlpRRWz71x 2024-05-11T13:05:21.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IlpRRWz71x 2024-05-11T13:05:21.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.IlpRRWz71x 2024-05-11T13:05:21.705 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.IlpRRWz71x is 6KB 2024-05-11T13:05:21.753 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.IlpRRWz71x 2024-05-11T13:05:21.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T13:05:21.828 INFO:teuthology.orchestra.run.smithi071.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T13:05:21.828 INFO:teuthology.orchestra.run.smithi071.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T13:05:21.829 INFO:teuthology.orchestra.run.smithi071.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T13:05:21.829 INFO:teuthology.orchestra.run.smithi071.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-11T13:05:21.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:21.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-05-11T13:05:25.675 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-11T13:05:25.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-11T13:05:25.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:05:39.402 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-11T13:05:39.449 DEBUG:teuthology.orchestra.run.smithi063:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-11T13:05:39.528 INFO:teuthology.orchestra.run.smithi063.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:39.528 INFO:teuthology.orchestra.run.smithi063.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:39.528 INFO:teuthology.orchestra.run.smithi063.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:39.528 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-11T13:05:39.529 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-11T13:05:39.530 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-11T13:05:39.531 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-11T13:05:39.532 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-11T13:05:39.533 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-11T13:05:39.534 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-11T13:05:39.535 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-11T13:05:39.536 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-11T13:05:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-11T13:05:39.538 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-11T13:05:39.539 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-11T13:05:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-11T13:05:39.541 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-11T13:05:39.542 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-11T13:05:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-11T13:05:39.544 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-11T13:05:39.545 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-11T13:05:39.546 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-11T13:05:39.547 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-11T13:05:39.548 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-11T13:05:39.549 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-11T13:05:39.550 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-11T13:05:39.551 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-11T13:05:39.552 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-11T13:05:39.553 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-11T13:05:39.554 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-11T13:05:39.555 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-11T13:05:39.556 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-11T13:05:39.557 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-11T13:05:39.558 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-11T13:05:39.559 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-11T13:05:39.560 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-11T13:05:39.561 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-11T13:05:39.562 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-11T13:05:39.563 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-11T13:05:39.564 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-11T13:05:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-11T13:05:39.566 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-11T13:05:39.567 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-11T13:05:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-11T13:05:39.569 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-11T13:05:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-11T13:05:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-11T13:05:39.572 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-11T13:05:39.573 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-11T13:05:39.574 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-11T13:05:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-11T13:05:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-11T13:05:39.577 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-11T13:05:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-11T13:05:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-11T13:05:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-11T13:05:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-11T13:05:39.582 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-11T13:05:39.583 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-11T13:05:39.584 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-11T13:05:39.585 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-11T13:05:39.586 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-11T13:05:39.587 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-11T13:05:39.588 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-11T13:05:39.589 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-11T13:05:39.590 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-11T13:05:39.591 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-11T13:05:39.592 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-11T13:05:39.593 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-11T13:05:39.594 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-11T13:05:39.595 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-11T13:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-11T13:05:39.597 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-11T13:05:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-11T13:05:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-11T13:05:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-11T13:05:39.601 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-11T13:05:39.602 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-11T13:05:39.603 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-11T13:05:39.604 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-11T13:05:39.605 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-11T13:05:39.606 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-11T13:05:39.607 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-11T13:05:39.608 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-11T13:05:39.609 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-11T13:05:39.610 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-11T13:05:39.611 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-11T13:05:39.612 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-11T13:05:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-11T13:05:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-11T13:05:39.615 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-11T13:05:39.616 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-11T13:05:39.617 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-11T13:05:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-11T13:05:39.619 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-11T13:05:39.620 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-11T13:05:39.621 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-11T13:05:39.622 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-11T13:05:39.623 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-11T13:05:39.624 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-11T13:05:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-11T13:05:39.626 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-11T13:05:39.627 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-11T13:05:39.628 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-11T13:05:39.629 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-11T13:05:39.630 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-11T13:05:39.631 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-11T13:05:39.632 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-11T13:05:39.633 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-11T13:05:39.634 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-11T13:05:39.635 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-11T13:05:39.636 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-11T13:05:39.637 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-11T13:05:39.637 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-11T13:05:39.637 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-11T13:05:39.637 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-11T13:05:39.637 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-11T13:05:39.637 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-11T13:05:39.638 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-11T13:05:39.639 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-11T13:05:39.640 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-11T13:05:39.641 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-11T13:05:39.642 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-11T13:05:39.643 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-11T13:05:39.644 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-11T13:05:39.645 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-11T13:05:39.646 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-11T13:05:39.647 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-11T13:05:39.648 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-11T13:05:39.649 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-11T13:05:39.650 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-11T13:05:39.651 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-11T13:05:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-11T13:05:39.653 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-11T13:05:39.654 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-11T13:05:39.655 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-11T13:05:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-11T13:05:39.657 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-11T13:05:39.658 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-11T13:05:39.659 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-11T13:05:39.660 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-11T13:05:39.661 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-11T13:05:39.662 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-11T13:05:39.663 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-11T13:05:39.664 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-11T13:05:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-11T13:05:39.666 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T13:05:39.670 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:39.670 DEBUG:teuthology.orchestra.run.smithi063:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-11T13:05:39.694 INFO:teuthology.orchestra.run.smithi063.stdout:postinstall scriptlet (using /bin/sh): 2024-05-11T13:05:39.694 INFO:teuthology.orchestra.run.smithi063.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T13:05:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T13:05:39.694 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout:for file in vmlinuz System.map config; do 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: fi 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout:done 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-11T13:05:39.695 INFO:teuthology.orchestra.run.smithi063.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-11T13:05:39.696 INFO:teuthology.orchestra.run.smithi063.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T13:05:39.696 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:39.696 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-11T13:05:39.697 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -qi grub2-tools 2024-05-11T13:05:39.774 INFO:teuthology.orchestra.run.smithi063.stdout:Name : grub2-tools 2024-05-11T13:05:39.774 INFO:teuthology.orchestra.run.smithi063.stdout:Epoch : 1 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Version : 2.06 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Release : 68.el9 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: x86_64 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Group : Unspecified 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Size : 8264254 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:License : GPLv3+ 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Packager : builder@centos.org 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:Vendor : CentOS 2024-05-11T13:05:39.775 INFO:teuthology.orchestra.run.smithi063.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout:Summary : Support tools for GRUB. 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout:Description : 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout:hardware devices. 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:05:39.776 INFO:teuthology.orchestra.run.smithi063.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T13:05:39.777 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T13:05:39.777 INFO:teuthology.task.kernel:Updating grub on smithi063 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:39.777 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T13:05:40.318 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-11T13:05:41.776 INFO:teuthology.orchestra.run.smithi063.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T13:05:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-11T13:05:41.809 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-11T13:05:41.830 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.6MNkGoBwaV 2024-05-11T13:05:41.830 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6MNkGoBwaV 2024-05-11T13:05:41.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.6MNkGoBwaV 2024-05-11T13:05:42.041 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.6MNkGoBwaV is 6KB 2024-05-11T13:05:42.089 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.6MNkGoBwaV 2024-05-11T13:05:42.105 DEBUG:teuthology.orchestra.run.smithi063:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T13:05:42.168 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T13:05:42.168 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T13:05:42.168 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T13:05:42.168 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-11T13:05:42.169 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:05:42.328 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-05-11T13:05:44.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-05-11T13:05:51.994 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-11T13:05:51.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-11T13:05:51.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:05:53.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-11T13:05:53.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:05:56.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-05-11T13:06:08.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-11T13:06:08.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:06:10.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-05-11T13:06:12.333 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-11T13:06:12.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-11T13:06:12.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:06:19.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-11T13:06:19.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:06:22.402 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-05-11T13:06:26.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-05-11T13:06:30.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-11T13:06:34.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-11T13:06:34.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:06:39.825 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-11T13:06:39.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:06:41.721 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-11T13:06:41.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:06:42.253 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-05-11T13:06:42.877 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-11T13:06:42.877 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-11T13:06:42.877 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-05-11T13:06:42.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-11T13:06:42.916 INFO:teuthology.orchestra.run.smithi165.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:06:42.916 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:06:42.916 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T13:06:42.916 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T13:06:43.917 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-11T13:06:43.917 DEBUG:teuthology.orchestra.run.smithi165:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-11T13:06:44.069 INFO:teuthology.orchestra.run.smithi165.stdout:ttyS1 2024-05-11T13:06:44.144 DEBUG:teuthology.parallel:result is None 2024-05-11T13:06:54.879 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-11T13:06:54.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:07:13.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-11T13:07:28.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-11T13:07:28.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:07:29.200 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-11T13:07:29.888 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-11T13:07:29.888 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-11T13:07:29.889 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-11T13:07:29.932 INFO:teuthology.orchestra.run.smithi063.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:07:29.933 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:07:29.933 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T13:07:29.933 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T13:07:30.934 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-11T13:07:30.934 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-11T13:07:31.075 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-05-11T13:07:31.116 DEBUG:teuthology.parallel:result is None 2024-05-11T13:07:34.410 DEBUG:teuthology.orchestra.remote:timed out 2024-05-11T13:07:49.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-11T13:07:49.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:07:49.848 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-05-11T13:07:50.508 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-11T13:07:50.508 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-11T13:07:50.508 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-05-11T13:07:50.562 INFO:teuthology.orchestra.run.smithi071.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:07:50.562 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-11T13:07:50.562 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T13:07:50.562 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T13:07:51.563 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-11T13:07:51.563 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-11T13:07:51.701 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-05-11T13:07:51.758 DEBUG:teuthology.parallel:result is None 2024-05-11T13:07:51.759 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-11T13:07:51.766 INFO:teuthology.task.internal:Creating test directory... 2024-05-11T13:07:51.766 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T13:07:51.775 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T13:07:51.781 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T13:07:51.848 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-11T13:07:51.855 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-11T13:07:51.877 INFO:teuthology.task.internal:Creating archive directory... 2024-05-11T13:07:51.877 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T13:07:51.885 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T13:07:51.892 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T13:07:51.971 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-11T13:07:51.978 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-11T13:07:51.979 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T13:07:51.986 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T13:07:51.994 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T13:07:52.080 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T13:07:52.084 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T13:07:52.084 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T13:07:52.112 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T13:07:52.113 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T13:07:52.119 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T13:07:52.124 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-11T13:07:52.131 INFO:teuthology.task.internal:Configuring sudo... 2024-05-11T13:07:52.131 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T13:07:52.136 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T13:07:52.143 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T13:07:52.230 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-11T13:07:52.241 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-11T13:07:52.241 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T13:07:52.249 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T13:07:52.257 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T13:07:52.314 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T13:07:52.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T13:07:52.508 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T13:07:52.591 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T13:07:53.042 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:07:53.043 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T13:07:53.132 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T13:07:53.222 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T13:07:53.321 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T13:07:53.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T13:07:53.488 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:07:53.488 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T13:07:53.568 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T13:07:53.656 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T13:07:53.751 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T13:07:53.837 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T13:07:53.921 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:07:53.921 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T13:07:54.004 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-05-11T13:07:54.011 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-05-11T13:07:54.019 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-05-11T13:07:54.124 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T13:07:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T13:07:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T13:07:54.530 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-11T13:07:54.537 INFO:teuthology.task.internal:Starting timer... 2024-05-11T13:07:54.538 INFO:teuthology.run_tasks:Running task pcp... 2024-05-11T13:07:54.599 INFO:teuthology.run_tasks:Running task selinux... 2024-05-11T13:07:54.607 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-11T13:07:54.607 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-05-11T13:07:54.738 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-05-11T13:07:54.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-05-11T13:07:54.890 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-05-11T13:07:54.896 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-05-11T13:07:55.019 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-05-11T13:07:55.023 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-11T13:07:55.023 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-05-11T13:07:55.076 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-05-11T13:07:55.076 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-05-11T13:07:55.135 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-05-11T13:07:55.135 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-05-11T13:07:55.201 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-05-11T13:07:55.201 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi063.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-05-11T13:07:55.202 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T13:07:55.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:07:55.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T13:07:55.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:07:55.355 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T13:07:55.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:07:55.426 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-11T13:07:55.426 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-05-11T13:07:55.504 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-05-11T13:07:55.578 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-05-11T13:07:55.652 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-11T13:07:55.661 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-11T13:07:55.677 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-11T13:07:55.682 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi063.front.sepia.ceph.com,smithi071.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-11T13:13:42.974 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-05-11T13:13:42.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-11T13:13:42.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:13:43.088 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-11T13:13:43.245 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-11T13:13:43.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-11T13:13:43.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:13:43.358 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-05-11T13:13:43.511 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-11T13:13:43.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-11T13:13:43.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T13:13:43.624 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-05-11T13:13:43.780 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-11T13:13:43.781 INFO:teuthology.run_tasks:Running task clock... 2024-05-11T13:13:43.794 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-11T13:13:43.794 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T13:13:43.795 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T13:13:43.802 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T13:13:43.802 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T13:13:43.810 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T13:13:43.810 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T13:13:43.893 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T13:13:43.898 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T13:13:43.901 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T13:13:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T13:13:43.939 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T13:13:43.944 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T13:13:44.007 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-05-11T13:13:44.016 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-05-11T13:13:44.019 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-05-11T13:13:44.039 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-05-11T13:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-05-11T13:13:44.050 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-05-11T13:13:44.079 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T13:13:44.089 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T13:13:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T13:13:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T13:13:44.131 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T13:13:44.137 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T13:13:44.252 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-05-11T13:13:44.261 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T13:13:44.261 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-05-11T13:13:44.261 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.261 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.261 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.261 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.262 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-05-11T13:13:44.271 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T13:13:44.271 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-05-11T13:13:44.271 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.271 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.271 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.271 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.277 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-11T13:13:44.286 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T13:13:44.286 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-11T13:13:44.287 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.287 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.287 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.287 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:13:44.288 INFO:teuthology.run_tasks:Running task install... 2024-05-11T13:13:44.296 DEBUG:teuthology.task.install:project ceph 2024-05-11T13:13:44.297 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}} 2024-05-11T13:13:44.297 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'} 2024-05-11T13:13:44.297 INFO:teuthology.task.install:Using flavor: default 2024-05-11T13:13:44.303 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-11T13:13:44.303 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-11T13:13:44.303 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T13:13:44.304 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:13:44.306 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T13:13:44.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:13:44.307 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T13:13:44.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:13:44.454 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T13:13:44.454 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T13:13:44.465 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T13:13:44.465 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T13:13:44.529 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T13:13:44.529 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T13:13:44.677 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T13:13:44.678 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:13:44.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T13:13:44.685 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T13:13:44.685 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:13:44.686 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T13:13:44.738 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T13:13:44.739 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:13:44.739 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T13:13:44.768 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T13:13:44.768 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T13:13:44.776 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T13:13:44.776 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T13:13:44.832 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T13:13:44.833 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T13:13:44.895 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T13:13:44.901 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T13:13:44.969 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T13:13:45.000 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T13:13:45.019 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T13:13:45.075 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-05-11T13:13:45.079 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-05-11T13:13:45.085 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T13:13:45.098 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-05-11T13:13:45.102 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-05-11T13:13:45.169 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-05-11T13:13:45.173 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-05-11T13:13:45.416 INFO:teuthology.orchestra.run.smithi071.stdout:54 files removed 2024-05-11T13:13:45.435 INFO:teuthology.orchestra.run.smithi165.stdout:54 files removed 2024-05-11T13:13:45.454 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T13:13:45.474 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T13:13:45.510 INFO:teuthology.orchestra.run.smithi063.stdout:54 files removed 2024-05-11T13:13:45.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T13:13:46.585 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 144 kB/s | 84 kB 00:00 2024-05-11T13:13:46.600 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 122 kB/s | 84 kB 00:00 2024-05-11T13:13:46.787 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 93 kB/s | 84 kB 00:00 2024-05-11T13:13:47.417 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-05-11T13:13:47.469 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-05-11T13:13:47.546 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-05-11T13:13:48.293 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-05-11T13:13:48.513 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-05-11T13:13:48.514 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 2.0 kB/s | 1.9 kB 00:00 2024-05-11T13:13:49.345 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-11T13:13:49.609 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.1 MB 00:01 2024-05-11T13:13:50.816 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.1 MB 00:02 2024-05-11T13:13:51.585 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-11T13:13:52.354 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-11T13:13:54.550 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-05-11T13:13:56.036 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-11T13:13:57.364 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-05-11T13:13:57.762 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-05-11T13:13:59.039 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-11T13:13:59.592 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-05-11T13:13:59.653 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-11T13:14:00.639 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-11T13:14:01.323 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-05-11T13:14:03.250 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-11T13:14:06.089 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-05-11T13:14:06.824 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-11T13:14:08.741 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T13:14:08.742 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T13:14:08.763 INFO:teuthology.orchestra.run.smithi063.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T13:14:08.764 INFO:teuthology.orchestra.run.smithi063.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T13:14:08.766 INFO:teuthology.orchestra.run.smithi063.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T13:14:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T13:14:08.770 INFO:teuthology.orchestra.run.smithi063.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T13:14:08.773 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T13:14:08.774 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T13:14:08.848 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-11T13:14:08.868 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-11T13:14:08.868 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-11T13:14:08.868 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-11T13:14:08.868 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T13:14:08.869 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T13:14:08.870 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T13:14:08.871 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T13:14:08.872 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T13:14:08.873 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T13:14:08.874 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T13:14:08.875 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T13:14:08.876 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T13:14:08.877 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T13:14:08.878 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T13:14:08.879 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout:Install 148 Packages 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 8 Packages 2024-05-11T13:14:08.880 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:14:08.881 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 226 M 2024-05-11T13:14:08.881 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-11T13:14:09.394 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-11T13:14:09.410 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T13:14:09.412 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T13:14:09.433 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T13:14:09.434 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T13:14:09.435 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T13:14:09.437 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T13:14:09.439 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T13:14:09.442 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T13:14:09.444 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T13:14:09.514 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T13:14:09.539 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T13:14:09.540 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T13:14:09.541 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-05-11T13:14:09.542 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T13:14:09.543 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T13:14:09.544 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T13:14:09.545 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T13:14:09.546 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T13:14:09.547 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T13:14:09.548 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T13:14:09.549 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:14:09.550 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-05-11T13:14:09.551 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-05-11T13:14:09.551 INFO:teuthology.orchestra.run.smithi071.stdout:Install 148 Packages 2024-05-11T13:14:09.551 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 8 Packages 2024-05-11T13:14:09.551 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:14:09.551 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 226 M 2024-05-11T13:14:09.551 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-05-11T13:14:09.607 INFO:teuthology.orchestra.run.smithi063.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-05-11T13:14:09.791 INFO:teuthology.orchestra.run.smithi063.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 4.8 MB/s | 890 kB 00:00 2024-05-11T13:14:09.841 INFO:teuthology.orchestra.run.smithi063.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 2.9 MB/s | 145 kB 00:00 2024-05-11T13:14:10.000 INFO:teuthology.orchestra.run.smithi063.stdout:(4/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 15 MB/s | 2.4 MB 00:00 2024-05-11T13:14:10.125 INFO:teuthology.orchestra.run.smithi063.stdout:(5/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 12 MB/s | 1.5 MB 00:00 2024-05-11T13:14:10.425 INFO:teuthology.orchestra.run.smithi063.stdout:(6/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 16 MB/s | 4.7 MB 00:00 2024-05-11T13:14:10.651 INFO:teuthology.orchestra.run.smithi071.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-05-11T13:14:11.509 INFO:teuthology.orchestra.run.smithi063.stdout:(7/156): ceph-osd-19.0.0-3601.g30850680.el9.x86 16 MB/s | 17 MB 00:01 2024-05-11T13:14:11.568 INFO:teuthology.orchestra.run.smithi071.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 972 kB/s | 890 kB 00:00 2024-05-11T13:14:11.692 INFO:teuthology.orchestra.run.smithi063.stdout:(8/156): ceph-base-19.0.0-3601.g30850680.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-05-11T13:14:11.726 INFO:teuthology.orchestra.run.smithi071.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 920 kB/s | 145 kB 00:00 2024-05-11T13:14:11.793 INFO:teuthology.orchestra.run.smithi063.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 255 kB/s | 25 kB 00:00 2024-05-11T13:14:11.946 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T13:14:11.948 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T13:14:11.967 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T13:14:11.969 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T13:14:11.970 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T13:14:11.971 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T13:14:11.974 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T13:14:11.977 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T13:14:11.978 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T13:14:12.050 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-05-11T13:14:12.066 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-05-11T13:14:12.066 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-05-11T13:14:12.066 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-05-11T13:14:12.066 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-05-11T13:14:12.066 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T13:14:12.066 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T13:14:12.067 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T13:14:12.068 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T13:14:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T13:14:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T13:14:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T13:14:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T13:14:12.073 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T13:14:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T13:14:12.075 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T13:14:12.076 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:14:12.077 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-05-11T13:14:12.078 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-05-11T13:14:12.078 INFO:teuthology.orchestra.run.smithi165.stdout:Install 148 Packages 2024-05-11T13:14:12.078 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 8 Packages 2024-05-11T13:14:12.078 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:14:12.078 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 226 M 2024-05-11T13:14:12.078 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-05-11T13:14:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:(10/156): ceph-radosgw-19.0.0-3601.g30850680.el 12 MB/s | 11 MB 00:00 2024-05-11T13:14:12.493 INFO:teuthology.orchestra.run.smithi063.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 2.4 MB/s | 225 kB 00:00 2024-05-11T13:14:12.568 INFO:teuthology.orchestra.run.smithi063.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 366 kB/s | 27 kB 00:00 2024-05-11T13:14:12.627 INFO:teuthology.orchestra.run.smithi063.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 556 kB/s | 32 kB 00:00 2024-05-11T13:14:12.677 INFO:teuthology.orchestra.run.smithi071.stdout:(4/156): ceph-base-19.0.0-3601.g30850680.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-11T13:14:12.719 INFO:teuthology.orchestra.run.smithi063.stdout:(14/156): libcephfs2-19.0.0-3601.g30850680.el9. 8.0 MB/s | 746 kB 00:00 2024-05-11T13:14:12.794 INFO:teuthology.orchestra.run.smithi063.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 2.2 MB/s | 163 kB 00:00 2024-05-11T13:14:12.861 INFO:teuthology.orchestra.run.smithi063.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 1.9 MB/s | 127 kB 00:00 2024-05-11T13:14:12.915 INFO:teuthology.orchestra.run.smithi071.stdout:(5/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-05-11T13:14:12.945 INFO:teuthology.orchestra.run.smithi063.stdout:(17/156): libradosstriper1-19.0.0-3601.g3085068 5.9 MB/s | 501 kB 00:00 2024-05-11T13:14:13.354 INFO:teuthology.orchestra.run.smithi165.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-05-11T13:14:13.369 INFO:teuthology.orchestra.run.smithi071.stdout:(6/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-05-11T13:14:13.553 INFO:teuthology.orchestra.run.smithi063.stdout:(18/156): librgw2-19.0.0-3601.g30850680.el9.x86 8.9 MB/s | 5.4 MB 00:00 2024-05-11T13:14:13.645 INFO:teuthology.orchestra.run.smithi063.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 497 kB/s | 45 kB 00:00 2024-05-11T13:14:13.737 INFO:teuthology.orchestra.run.smithi063.stdout:(20/156): python3-ceph-common-19.0.0-3601.g3085 1.5 MB/s | 135 kB 00:00 2024-05-11T13:14:13.829 INFO:teuthology.orchestra.run.smithi063.stdout:(21/156): python3-cephfs-19.0.0-3601.g30850680. 1.7 MB/s | 161 kB 00:00 2024-05-11T13:14:13.879 INFO:teuthology.orchestra.run.smithi165.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 1.7 MB/s | 890 kB 00:00 2024-05-11T13:14:13.946 INFO:teuthology.orchestra.run.smithi063.stdout:(22/156): python3-rados-19.0.0-3601.g30850680.e 2.7 MB/s | 321 kB 00:00 2024-05-11T13:14:13.970 INFO:teuthology.orchestra.run.smithi165.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 1.6 MB/s | 145 kB 00:00 2024-05-11T13:14:14.046 INFO:teuthology.orchestra.run.smithi063.stdout:(23/156): python3-rbd-19.0.0-3601.g30850680.el9 3.0 MB/s | 300 kB 00:00 2024-05-11T13:14:14.139 INFO:teuthology.orchestra.run.smithi063.stdout:(24/156): python3-rgw-19.0.0-3601.g30850680.el9 1.1 MB/s | 100 kB 00:00 2024-05-11T13:14:14.230 INFO:teuthology.orchestra.run.smithi063.stdout:(25/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 946 kB/s | 86 kB 00:00 2024-05-11T13:14:14.752 INFO:teuthology.orchestra.run.smithi071.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 2.6 MB/s | 4.7 MB 00:01 2024-05-11T13:14:14.831 INFO:teuthology.orchestra.run.smithi063.stdout:(26/156): rbd-mirror-19.0.0-3601.g30850680.el9. 5.2 MB/s | 3.1 MB 00:00 2024-05-11T13:14:14.914 INFO:teuthology.orchestra.run.smithi063.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 2.0 MB/s | 171 kB 00:00 2024-05-11T13:14:14.989 INFO:teuthology.orchestra.run.smithi063.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 460 kB/s | 34 kB 00:00 2024-05-11T13:14:15.054 INFO:teuthology.orchestra.run.smithi165.stdout:(4/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-05-11T13:14:15.056 INFO:teuthology.orchestra.run.smithi063.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 2.1 MB/s | 142 kB 00:00 2024-05-11T13:14:15.715 INFO:teuthology.orchestra.run.smithi063.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3601.g30850 5.4 MB/s | 3.5 MB 00:00 2024-05-11T13:14:15.746 INFO:teuthology.orchestra.run.smithi165.stdout:(5/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-05-11T13:14:15.855 INFO:teuthology.orchestra.run.smithi165.stdout:(6/156): ceph-base-19.0.0-3601.g30850680.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-05-11T13:14:16.940 INFO:teuthology.orchestra.run.smithi063.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-05-11T13:14:17.032 INFO:teuthology.orchestra.run.smithi063.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3601.g30 3.0 MB/s | 274 kB 00:00 2024-05-11T13:14:17.116 INFO:teuthology.orchestra.run.smithi063.stdout:(33/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 598 kB/s | 49 kB 00:00 2024-05-11T13:14:17.191 INFO:teuthology.orchestra.run.smithi063.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3601.g3 223 kB/s | 16 kB 00:00 2024-05-11T13:14:17.275 INFO:teuthology.orchestra.run.smithi063.stdout:(35/156): ceph-volume-19.0.0-3601.g30850680.el9 3.2 MB/s | 273 kB 00:00 2024-05-11T13:14:17.458 INFO:teuthology.orchestra.run.smithi063.stdout:(36/156): cephadm-19.0.0-3601.g30850680.el9.noa 4.1 MB/s | 764 kB 00:00 2024-05-11T13:14:17.534 INFO:teuthology.orchestra.run.smithi063.stdout:(37/156): cephfs-top-19.0.0-3601.g30850680.el9. 243 kB/s | 18 kB 00:00 2024-05-11T13:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 2.6 MB/s | 4.7 MB 00:01 2024-05-11T13:14:17.642 INFO:teuthology.orchestra.run.smithi063.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 750 kB/s | 80 kB 00:00 2024-05-11T13:14:17.768 INFO:teuthology.orchestra.run.smithi063.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-11T13:14:17.818 INFO:teuthology.orchestra.run.smithi063.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-05-11T13:14:17.868 INFO:teuthology.orchestra.run.smithi063.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-05-11T13:14:17.951 INFO:teuthology.orchestra.run.smithi063.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-11T13:14:17.993 INFO:teuthology.orchestra.run.smithi063.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-11T13:14:18.035 INFO:teuthology.orchestra.run.smithi063.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 3.3 MB/s | 135 kB 00:00 2024-05-11T13:14:18.078 INFO:teuthology.orchestra.run.smithi063.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-05-11T13:14:18.128 INFO:teuthology.orchestra.run.smithi063.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-05-11T13:14:18.610 INFO:teuthology.orchestra.run.smithi071.stdout:(8/156): ceph-common-19.0.0-3601.g30850680.el9. 2.5 MB/s | 21 MB 00:08 2024-05-11T13:14:18.655 INFO:teuthology.orchestra.run.smithi165.stdout:(8/156): ceph-common-19.0.0-3601.g30850680.el9. 3.7 MB/s | 21 MB 00:05 2024-05-11T13:14:18.660 INFO:teuthology.orchestra.run.smithi071.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 516 kB/s | 25 kB 00:00 2024-05-11T13:14:18.697 INFO:teuthology.orchestra.run.smithi165.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 621 kB/s | 25 kB 00:00 2024-05-11T13:14:18.878 INFO:teuthology.orchestra.run.smithi071.stdout:(10/156): ceph-osd-19.0.0-3601.g30850680.el9.x8 3.1 MB/s | 17 MB 00:05 2024-05-11T13:14:18.988 INFO:teuthology.orchestra.run.smithi071.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 2.0 MB/s | 225 kB 00:00 2024-05-11T13:14:19.053 INFO:teuthology.orchestra.run.smithi071.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 420 kB/s | 27 kB 00:00 2024-05-11T13:14:19.112 INFO:teuthology.orchestra.run.smithi071.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 551 kB/s | 32 kB 00:00 2024-05-11T13:14:19.187 INFO:teuthology.orchestra.run.smithi063.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 651 kB/s | 689 kB 00:01 2024-05-11T13:14:19.287 INFO:teuthology.orchestra.run.smithi071.stdout:(14/156): libcephfs2-19.0.0-3601.g30850680.el9. 4.2 MB/s | 746 kB 00:00 2024-05-11T13:14:19.346 INFO:teuthology.orchestra.run.smithi071.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 2.8 MB/s | 163 kB 00:00 2024-05-11T13:14:19.663 INFO:teuthology.orchestra.run.smithi071.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 401 kB/s | 127 kB 00:00 2024-05-11T13:14:19.763 INFO:teuthology.orchestra.run.smithi165.stdout:(10/156): ceph-radosgw-19.0.0-3601.g30850680.el 4.8 MB/s | 11 MB 00:02 2024-05-11T13:14:19.787 INFO:teuthology.orchestra.run.smithi063.stdout:(48/156): ceph-common-19.0.0-3601.g30850680.el9 2.0 MB/s | 21 MB 00:10 2024-05-11T13:14:19.847 INFO:teuthology.orchestra.run.smithi165.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 2.7 MB/s | 225 kB 00:00 2024-05-11T13:14:19.889 INFO:teuthology.orchestra.run.smithi165.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 665 kB/s | 27 kB 00:00 2024-05-11T13:14:19.931 INFO:teuthology.orchestra.run.smithi165.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 780 kB/s | 32 kB 00:00 2024-05-11T13:14:20.046 INFO:teuthology.orchestra.run.smithi071.stdout:(17/156): libradosstriper1-19.0.0-3601.g3085068 1.3 MB/s | 501 kB 00:00 2024-05-11T13:14:20.089 INFO:teuthology.orchestra.run.smithi165.stdout:(14/156): libcephfs2-19.0.0-3601.g30850680.el9. 4.6 MB/s | 746 kB 00:00 2024-05-11T13:14:20.156 INFO:teuthology.orchestra.run.smithi165.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 2.4 MB/s | 163 kB 00:00 2024-05-11T13:14:20.215 INFO:teuthology.orchestra.run.smithi165.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 2.2 MB/s | 127 kB 00:00 2024-05-11T13:14:20.287 INFO:teuthology.orchestra.run.smithi063.stdout:(49/156): ceph-test-19.0.0-3601.g30850680.el9.x 5.8 MB/s | 49 MB 00:08 2024-05-11T13:14:20.329 INFO:teuthology.orchestra.run.smithi063.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 826 kB/s | 943 kB 00:01 2024-05-11T13:14:20.357 INFO:teuthology.orchestra.run.smithi165.stdout:(17/156): libradosstriper1-19.0.0-3601.g3085068 3.5 MB/s | 501 kB 00:00 2024-05-11T13:14:20.430 INFO:teuthology.orchestra.run.smithi063.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 153 kB/s | 21 kB 00:00 2024-05-11T13:14:20.455 INFO:teuthology.orchestra.run.smithi063.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-05-11T13:14:20.538 INFO:teuthology.orchestra.run.smithi063.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-05-11T13:14:20.597 INFO:teuthology.orchestra.run.smithi063.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 317 kB/s | 18 kB 00:00 2024-05-11T13:14:20.630 INFO:teuthology.orchestra.run.smithi071.stdout:(18/156): ceph-radosgw-19.0.0-3601.g30850680.el 1.8 MB/s | 11 MB 00:05 2024-05-11T13:14:20.630 INFO:teuthology.orchestra.run.smithi063.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-05-11T13:14:20.656 INFO:teuthology.orchestra.run.smithi063.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-05-11T13:14:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 786 kB/s | 45 kB 00:00 2024-05-11T13:14:20.730 INFO:teuthology.orchestra.run.smithi071.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 456 kB/s | 45 kB 00:00 2024-05-11T13:14:20.823 INFO:teuthology.orchestra.run.smithi063.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 838 kB/s | 160 kB 00:00 2024-05-11T13:14:20.839 INFO:teuthology.orchestra.run.smithi071.stdout:(20/156): python3-ceph-common-19.0.0-3601.g3085 1.2 MB/s | 135 kB 00:00 2024-05-11T13:14:20.907 INFO:teuthology.orchestra.run.smithi063.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.6 MB/s | 3.0 MB 00:00 2024-05-11T13:14:20.957 INFO:teuthology.orchestra.run.smithi063.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 519 kB/s | 25 kB 00:00 2024-05-11T13:14:20.981 INFO:teuthology.orchestra.run.smithi071.stdout:(21/156): python3-cephfs-19.0.0-3601.g30850680. 1.1 MB/s | 161 kB 00:00 2024-05-11T13:14:20.999 INFO:teuthology.orchestra.run.smithi063.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-05-11T13:14:21.024 INFO:teuthology.orchestra.run.smithi063.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 718 kB/s | 47 kB 00:00 2024-05-11T13:14:21.082 INFO:teuthology.orchestra.run.smithi063.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2024-05-11T13:14:21.117 INFO:teuthology.orchestra.run.smithi063.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-11T13:14:21.133 INFO:teuthology.orchestra.run.smithi063.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 710 kB/s | 35 kB 00:00 2024-05-11T13:14:21.156 INFO:teuthology.orchestra.run.smithi071.stdout:(22/156): librgw2-19.0.0-3601.g30850680.el9.x86 4.8 MB/s | 5.4 MB 00:01 2024-05-11T13:14:21.224 INFO:teuthology.orchestra.run.smithi165.stdout:(18/156): librgw2-19.0.0-3601.g30850680.el9.x86 6.2 MB/s | 5.4 MB 00:00 2024-05-11T13:14:21.225 INFO:teuthology.orchestra.run.smithi063.stdout:(66/156): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-05-11T13:14:21.231 INFO:teuthology.orchestra.run.smithi071.stdout:(23/156): python3-rados-19.0.0-3601.g30850680.e 1.3 MB/s | 321 kB 00:00 2024-05-11T13:14:21.265 INFO:teuthology.orchestra.run.smithi071.stdout:(24/156): python3-rbd-19.0.0-3601.g30850680.el9 2.7 MB/s | 300 kB 00:00 2024-05-11T13:14:21.274 INFO:teuthology.orchestra.run.smithi165.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 925 kB/s | 45 kB 00:00 2024-05-11T13:14:21.307 INFO:teuthology.orchestra.run.smithi071.stdout:(25/156): python3-rgw-19.0.0-3601.g30850680.el9 1.3 MB/s | 100 kB 00:00 2024-05-11T13:14:21.332 INFO:teuthology.orchestra.run.smithi071.stdout:(26/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-11T13:14:21.333 INFO:teuthology.orchestra.run.smithi165.stdout:(20/156): python3-ceph-common-19.0.0-3601.g3085 2.3 MB/s | 135 kB 00:00 2024-05-11T13:14:21.400 INFO:teuthology.orchestra.run.smithi165.stdout:(21/156): python3-cephfs-19.0.0-3601.g30850680. 2.4 MB/s | 161 kB 00:00 2024-05-11T13:14:21.428 INFO:teuthology.orchestra.run.smithi071.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 1.8 MB/s | 171 kB 00:00 2024-05-11T13:14:21.483 INFO:teuthology.orchestra.run.smithi165.stdout:(22/156): python3-rados-19.0.0-3601.g30850680.e 3.8 MB/s | 321 kB 00:00 2024-05-11T13:14:21.491 INFO:teuthology.orchestra.run.smithi071.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 552 kB/s | 34 kB 00:00 2024-05-11T13:14:21.492 INFO:teuthology.orchestra.run.smithi063.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-05-11T13:14:21.558 INFO:teuthology.orchestra.run.smithi071.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 2.1 MB/s | 142 kB 00:00 2024-05-11T13:14:21.575 INFO:teuthology.orchestra.run.smithi165.stdout:(23/156): python3-rbd-19.0.0-3601.g30850680.el9 3.2 MB/s | 300 kB 00:00 2024-05-11T13:14:21.592 INFO:teuthology.orchestra.run.smithi063.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-05-11T13:14:21.642 INFO:teuthology.orchestra.run.smithi165.stdout:(24/156): python3-rgw-19.0.0-3601.g30850680.el9 1.5 MB/s | 100 kB 00:00 2024-05-11T13:14:21.643 INFO:teuthology.orchestra.run.smithi063.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 911 kB 00:00 2024-05-11T13:14:21.676 INFO:teuthology.orchestra.run.smithi063.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-11T13:14:21.717 INFO:teuthology.orchestra.run.smithi165.stdout:(25/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-11T13:14:21.793 INFO:teuthology.orchestra.run.smithi063.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-05-11T13:14:21.860 INFO:teuthology.orchestra.run.smithi063.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-11T13:14:22.044 INFO:teuthology.orchestra.run.smithi063.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-05-11T13:14:22.119 INFO:teuthology.orchestra.run.smithi063.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-11T13:14:22.203 INFO:teuthology.orchestra.run.smithi063.stdout:(75/156): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-05-11T13:14:22.219 INFO:teuthology.orchestra.run.smithi071.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3601.g30850 5.4 MB/s | 3.5 MB 00:00 2024-05-11T13:14:22.236 INFO:teuthology.orchestra.run.smithi063.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-11T13:14:22.262 INFO:teuthology.orchestra.run.smithi063.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 444 kB/s | 26 kB 00:00 2024-05-11T13:14:22.295 INFO:teuthology.orchestra.run.smithi063.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 331 kB/s | 19 kB 00:00 2024-05-11T13:14:22.301 INFO:teuthology.orchestra.run.smithi165.stdout:(26/156): rbd-mirror-19.0.0-3601.g30850680.el9. 5.3 MB/s | 3.1 MB 00:00 2024-05-11T13:14:22.368 INFO:teuthology.orchestra.run.smithi165.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 2.5 MB/s | 171 kB 00:00 2024-05-11T13:14:22.421 INFO:teuthology.orchestra.run.smithi063.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-11T13:14:22.435 INFO:teuthology.orchestra.run.smithi165.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 522 kB/s | 34 kB 00:00 2024-05-11T13:14:22.446 INFO:teuthology.orchestra.run.smithi063.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 232 kB/s | 35 kB 00:00 2024-05-11T13:14:22.471 INFO:teuthology.orchestra.run.smithi063.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 827 kB/s | 172 kB 00:00 2024-05-11T13:14:22.502 INFO:teuthology.orchestra.run.smithi165.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 2.1 MB/s | 142 kB 00:00 2024-05-11T13:14:22.513 INFO:teuthology.orchestra.run.smithi063.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-05-11T13:14:22.580 INFO:teuthology.orchestra.run.smithi063.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-05-11T13:14:22.647 INFO:teuthology.orchestra.run.smithi063.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-05-11T13:14:22.672 INFO:teuthology.orchestra.run.smithi063.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 464 kB/s | 42 kB 00:00 2024-05-11T13:14:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-11T13:14:22.823 INFO:teuthology.orchestra.run.smithi063.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-11T13:14:22.856 INFO:teuthology.orchestra.run.smithi063.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-05-11T13:14:22.882 INFO:teuthology.orchestra.run.smithi063.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 885 kB/s | 51 kB 00:00 2024-05-11T13:14:22.915 INFO:teuthology.orchestra.run.smithi063.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-11T13:14:22.940 INFO:teuthology.orchestra.run.smithi063.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 642 kB/s | 54 kB 00:00 2024-05-11T13:14:22.974 INFO:teuthology.orchestra.run.smithi063.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 719 kB/s | 42 kB 00:00 2024-05-11T13:14:22.976 INFO:teuthology.orchestra.run.smithi071.stdout:(31/156): rbd-mirror-19.0.0-3601.g30850680.el9. 1.9 MB/s | 3.1 MB 00:01 2024-05-11T13:14:23.008 INFO:teuthology.orchestra.run.smithi063.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 668 kB/s | 44 kB 00:00 2024-05-11T13:14:23.041 INFO:teuthology.orchestra.run.smithi063.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-05-11T13:14:23.075 INFO:teuthology.orchestra.run.smithi063.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 544 kB/s | 36 kB 00:00 2024-05-11T13:14:23.100 INFO:teuthology.orchestra.run.smithi063.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 817 kB/s | 47 kB 00:00 2024-05-11T13:14:23.159 INFO:teuthology.orchestra.run.smithi063.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-05-11T13:14:23.193 INFO:teuthology.orchestra.run.smithi071.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3601.g30 1.2 MB/s | 274 kB 00:00 2024-05-11T13:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3601.g30850 5.1 MB/s | 3.5 MB 00:00 2024-05-11T13:14:23.217 INFO:teuthology.orchestra.run.smithi063.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-11T13:14:23.268 INFO:teuthology.orchestra.run.smithi071.stdout:(33/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 667 kB/s | 49 kB 00:00 2024-05-11T13:14:23.276 INFO:teuthology.orchestra.run.smithi063.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-05-11T13:14:23.301 INFO:teuthology.orchestra.run.smithi063.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 771 kB/s | 64 kB 00:00 2024-05-11T13:14:23.343 INFO:teuthology.orchestra.run.smithi063.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 679 kB/s | 45 kB 00:00 2024-05-11T13:14:23.360 INFO:teuthology.orchestra.run.smithi071.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3601.g3 181 kB/s | 16 kB 00:00 2024-05-11T13:14:23.368 INFO:teuthology.orchestra.run.smithi063.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 335 kB/s | 22 kB 00:00 2024-05-11T13:14:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:(35/156): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-05-11T13:14:23.602 INFO:teuthology.orchestra.run.smithi071.stdout:(36/156): ceph-volume-19.0.0-3601.g30850680.el9 1.1 MB/s | 273 kB 00:00 2024-05-11T13:14:23.686 INFO:teuthology.orchestra.run.smithi071.stdout:(37/156): cephadm-19.0.0-3601.g30850680.el9.noa 4.7 MB/s | 764 kB 00:00 2024-05-11T13:14:23.694 INFO:teuthology.orchestra.run.smithi063.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-05-11T13:14:23.703 INFO:teuthology.orchestra.run.smithi071.stdout:(38/156): cephfs-top-19.0.0-3601.g30850680.el9. 183 kB/s | 18 kB 00:00 2024-05-11T13:14:23.736 INFO:teuthology.orchestra.run.smithi063.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-05-11T13:14:23.854 INFO:teuthology.orchestra.run.smithi071.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 484 kB/s | 80 kB 00:00 2024-05-11T13:14:23.861 INFO:teuthology.orchestra.run.smithi063.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-05-11T13:14:23.945 INFO:teuthology.orchestra.run.smithi071.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-11T13:14:23.953 INFO:teuthology.orchestra.run.smithi063.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-11T13:14:24.012 INFO:teuthology.orchestra.run.smithi063.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 854 kB/s | 548 kB 00:00 2024-05-11T13:14:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-05-11T13:14:24.037 INFO:teuthology.orchestra.run.smithi063.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 314 kB/s | 26 kB 00:00 2024-05-11T13:14:24.054 INFO:teuthology.orchestra.run.smithi071.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-05-11T13:14:24.087 INFO:teuthology.orchestra.run.smithi063.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 654 kB/s | 49 kB 00:00 2024-05-11T13:14:24.105 INFO:teuthology.orchestra.run.smithi071.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-11T13:14:24.112 INFO:teuthology.orchestra.run.smithi063.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 905 kB/s | 67 kB 00:00 2024-05-11T13:14:24.163 INFO:teuthology.orchestra.run.smithi071.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-11T13:14:24.247 INFO:teuthology.orchestra.run.smithi071.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-11T13:14:24.271 INFO:teuthology.orchestra.run.smithi063.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-11T13:14:24.296 INFO:teuthology.orchestra.run.smithi063.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 727 kB/s | 151 kB 00:00 2024-05-11T13:14:24.297 INFO:teuthology.orchestra.run.smithi071.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 4.5 MB/s | 1.2 MB 00:00 2024-05-11T13:14:24.331 INFO:teuthology.orchestra.run.smithi071.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-11T13:14:24.346 INFO:teuthology.orchestra.run.smithi063.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-05-11T13:14:24.372 INFO:teuthology.orchestra.run.smithi063.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 582 kB/s | 43 kB 00:00 2024-05-11T13:14:24.405 INFO:teuthology.orchestra.run.smithi063.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 559 kB/s | 32 kB 00:00 2024-05-11T13:14:24.431 INFO:teuthology.orchestra.run.smithi063.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-05-11T13:14:24.464 INFO:teuthology.orchestra.run.smithi063.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-11T13:14:24.494 INFO:teuthology.orchestra.run.smithi165.stdout:(31/156): ceph-test-19.0.0-3601.g30850680.el9.x 8.5 MB/s | 49 MB 00:05 2024-05-11T13:14:24.498 INFO:teuthology.orchestra.run.smithi063.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-05-11T13:14:24.531 INFO:teuthology.orchestra.run.smithi063.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-11T13:14:24.557 INFO:teuthology.orchestra.run.smithi063.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 613 kB/s | 35 kB 00:00 2024-05-11T13:14:24.615 INFO:teuthology.orchestra.run.smithi063.stdout:(121/156): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-05-11T13:14:24.649 INFO:teuthology.orchestra.run.smithi063.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-11T13:14:24.661 INFO:teuthology.orchestra.run.smithi165.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-05-11T13:14:24.694 INFO:teuthology.orchestra.run.smithi165.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3601.g30 1.3 MB/s | 274 kB 00:00 2024-05-11T13:14:24.720 INFO:teuthology.orchestra.run.smithi165.stdout:(34/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 857 kB/s | 49 kB 00:00 2024-05-11T13:14:24.741 INFO:teuthology.orchestra.run.smithi063.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-11T13:14:24.745 INFO:teuthology.orchestra.run.smithi165.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3601.g3 333 kB/s | 16 kB 00:00 2024-05-11T13:14:24.758 INFO:teuthology.orchestra.run.smithi063.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-11T13:14:24.783 INFO:teuthology.orchestra.run.smithi063.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 866 kB/s | 144 kB 00:00 2024-05-11T13:14:24.795 INFO:teuthology.orchestra.run.smithi165.stdout:(36/156): ceph-volume-19.0.0-3601.g30850680.el9 3.6 MB/s | 273 kB 00:00 2024-05-11T13:14:24.800 INFO:teuthology.orchestra.run.smithi063.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 305 kB/s | 18 kB 00:00 2024-05-11T13:14:24.817 INFO:teuthology.orchestra.run.smithi063.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 398 kB/s | 23 kB 00:00 2024-05-11T13:14:24.834 INFO:teuthology.orchestra.run.smithi063.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-05-11T13:14:24.845 INFO:teuthology.orchestra.run.smithi165.stdout:(37/156): cephfs-top-19.0.0-3601.g30850680.el9. 368 kB/s | 18 kB 00:00 2024-05-11T13:14:24.859 INFO:teuthology.orchestra.run.smithi063.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 339 kB/s | 20 kB 00:00 2024-05-11T13:14:24.909 INFO:teuthology.orchestra.run.smithi063.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-11T13:14:24.935 INFO:teuthology.orchestra.run.smithi063.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 464 kB/s | 46 kB 00:00 2024-05-11T13:14:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 790 kB/s | 79 kB 00:00 2024-05-11T13:14:24.985 INFO:teuthology.orchestra.run.smithi063.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 775 kB/s | 58 kB 00:00 2024-05-11T13:14:25.019 INFO:teuthology.orchestra.run.smithi063.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-05-11T13:14:25.044 INFO:teuthology.orchestra.run.smithi063.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-11T13:14:25.078 INFO:teuthology.orchestra.run.smithi063.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 487 kB/s | 28 kB 00:00 2024-05-11T13:14:25.082 INFO:teuthology.orchestra.run.smithi071.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 880 kB/s | 689 kB 00:00 2024-05-11T13:14:25.103 INFO:teuthology.orchestra.run.smithi063.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 529 kB/s | 31 kB 00:00 2024-05-11T13:14:25.132 INFO:teuthology.orchestra.run.smithi071.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-05-11T13:14:25.145 INFO:teuthology.orchestra.run.smithi063.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-05-11T13:14:25.146 INFO:teuthology.orchestra.run.smithi165.stdout:(38/156): ceph-osd-19.0.0-3601.g30850680.el9.x8 1.8 MB/s | 17 MB 00:09 2024-05-11T13:14:25.170 INFO:teuthology.orchestra.run.smithi063.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 893 kB/s | 59 kB 00:00 2024-05-11T13:14:25.179 INFO:teuthology.orchestra.run.smithi165.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 241 kB/s | 80 kB 00:00 2024-05-11T13:14:25.190 INFO:teuthology.orchestra.run.smithi071.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 993 kB/s | 107 kB 00:00 2024-05-11T13:14:25.204 INFO:teuthology.orchestra.run.smithi063.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-11T13:14:25.229 INFO:teuthology.orchestra.run.smithi063.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-05-11T13:14:25.241 INFO:teuthology.orchestra.run.smithi071.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-05-11T13:14:25.246 INFO:teuthology.orchestra.run.smithi165.stdout:(40/156): cephadm-19.0.0-3601.g30850680.el9.noa 1.5 MB/s | 764 kB 00:00 2024-05-11T13:14:25.254 INFO:teuthology.orchestra.run.smithi063.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 614 kB/s | 51 kB 00:00 2024-05-11T13:14:25.289 INFO:teuthology.orchestra.run.smithi063.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-11T13:14:25.296 INFO:teuthology.orchestra.run.smithi165.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-05-11T13:14:25.308 INFO:teuthology.orchestra.run.smithi071.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-11T13:14:25.330 INFO:teuthology.orchestra.run.smithi063.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-05-11T13:14:25.350 INFO:teuthology.orchestra.run.smithi071.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-05-11T13:14:25.355 INFO:teuthology.orchestra.run.smithi063.stdout:(145/156): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-05-11T13:14:25.380 INFO:teuthology.orchestra.run.smithi165.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-05-11T13:14:25.439 INFO:teuthology.orchestra.run.smithi165.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-05-11T13:14:25.458 INFO:teuthology.orchestra.run.smithi071.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 169 kB/s | 18 kB 00:00 2024-05-11T13:14:25.463 INFO:teuthology.orchestra.run.smithi165.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-05-11T13:14:25.489 INFO:teuthology.orchestra.run.smithi063.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 100 kB/s | 20 kB 00:00 2024-05-11T13:14:25.489 INFO:teuthology.orchestra.run.smithi165.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-05-11T13:14:25.523 INFO:teuthology.orchestra.run.smithi063.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-05-11T13:14:25.523 INFO:teuthology.orchestra.run.smithi165.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-11T13:14:25.556 INFO:teuthology.orchestra.run.smithi165.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-11T13:14:25.575 INFO:teuthology.orchestra.run.smithi071.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-05-11T13:14:25.589 INFO:teuthology.orchestra.run.smithi063.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-05-11T13:14:25.590 INFO:teuthology.orchestra.run.smithi165.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-05-11T13:14:25.674 INFO:teuthology.orchestra.run.smithi165.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-11T13:14:25.732 INFO:teuthology.orchestra.run.smithi165.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-05-11T13:14:25.757 INFO:teuthology.orchestra.run.smithi165.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 260 kB/s | 21 kB 00:00 2024-05-11T13:14:25.781 INFO:teuthology.orchestra.run.smithi063.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-05-11T13:14:25.799 INFO:teuthology.orchestra.run.smithi165.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-05-11T13:14:25.858 INFO:teuthology.orchestra.run.smithi165.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 329 kB/s | 33 kB 00:00 2024-05-11T13:14:25.908 INFO:teuthology.orchestra.run.smithi165.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-05-11T13:14:26.034 INFO:teuthology.orchestra.run.smithi165.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-05-11T13:14:26.059 INFO:teuthology.orchestra.run.smithi165.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 91 kB/s | 18 kB 00:00 2024-05-11T13:14:26.082 INFO:teuthology.orchestra.run.smithi063.stdout:(150/156): librados2-19.0.0-3601.g30850680.el9. 5.7 MB/s | 3.4 MB 00:00 2024-05-11T13:14:26.142 INFO:teuthology.orchestra.run.smithi071.stdout:(56/156): ceph-test-19.0.0-3601.g30850680.el9.x 6.6 MB/s | 49 MB 00:07 2024-05-11T13:14:26.149 INFO:teuthology.orchestra.run.smithi063.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 5.0 MB/s | 1.8 MB 00:00 2024-05-11T13:14:26.174 INFO:teuthology.orchestra.run.smithi063.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 458 kB/s | 42 kB 00:00 2024-05-11T13:14:26.176 INFO:teuthology.orchestra.run.smithi071.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 267 kB/s | 160 kB 00:00 2024-05-11T13:14:26.184 INFO:teuthology.orchestra.run.smithi165.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.8 MB/s | 3.0 MB 00:00 2024-05-11T13:14:26.210 INFO:teuthology.orchestra.run.smithi165.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 303 kB/s | 45 kB 00:00 2024-05-11T13:14:26.234 INFO:teuthology.orchestra.run.smithi165.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 801 kB/s | 160 kB 00:00 2024-05-11T13:14:26.249 INFO:teuthology.orchestra.run.smithi063.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.4 MB/s | 248 kB 00:00 2024-05-11T13:14:26.268 INFO:teuthology.orchestra.run.smithi071.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-05-11T13:14:26.285 INFO:teuthology.orchestra.run.smithi165.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 519 kB/s | 25 kB 00:00 2024-05-11T13:14:26.349 INFO:teuthology.orchestra.run.smithi165.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.0 MB/s | 662 kB 00:00 2024-05-11T13:14:26.358 INFO:teuthology.orchestra.run.smithi063.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-05-11T13:14:26.394 INFO:teuthology.orchestra.run.smithi165.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-05-11T13:14:26.415 INFO:teuthology.orchestra.run.smithi071.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 3.9 MB/s | 582 kB 00:00 2024-05-11T13:14:26.417 INFO:teuthology.orchestra.run.smithi063.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-11T13:14:26.419 INFO:teuthology.orchestra.run.smithi165.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 357 kB/s | 47 kB 00:00 2024-05-11T13:14:26.478 INFO:teuthology.orchestra.run.smithi165.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 292 kB 00:00 2024-05-11T13:14:26.503 INFO:teuthology.orchestra.run.smithi165.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 426 kB/s | 35 kB 00:00 2024-05-11T13:14:26.527 INFO:teuthology.orchestra.run.smithi071.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 230 kB/s | 25 kB 00:00 2024-05-11T13:14:26.536 INFO:teuthology.orchestra.run.smithi165.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-11T13:14:26.552 INFO:teuthology.orchestra.run.smithi071.stdout:(61/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 111 kB/s | 45 kB 00:00 2024-05-11T13:14:26.616 INFO:teuthology.orchestra.run.smithi063.stdout:(156/156): librbd1-19.0.0-3601.g30850680.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-05-11T13:14:26.619 INFO:teuthology.orchestra.run.smithi071.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-05-11T13:14:26.623 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-11T13:14:26.623 INFO:teuthology.orchestra.run.smithi063.stdout:Total 13 MB/s | 226 MB 00:17 2024-05-11T13:14:26.644 INFO:teuthology.orchestra.run.smithi071.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-05-11T13:14:26.678 INFO:teuthology.orchestra.run.smithi165.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-11T13:14:26.728 INFO:teuthology.orchestra.run.smithi071.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-11T13:14:26.753 INFO:teuthology.orchestra.run.smithi071.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 326 kB/s | 35 kB 00:00 2024-05-11T13:14:26.804 INFO:teuthology.orchestra.run.smithi165.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-11T13:14:26.896 INFO:teuthology.orchestra.run.smithi165.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-11T13:14:26.904 INFO:teuthology.orchestra.run.smithi071.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 834 kB/s | 292 kB 00:00 2024-05-11T13:14:26.937 INFO:teuthology.orchestra.run.smithi165.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-05-11T13:14:26.996 INFO:teuthology.orchestra.run.smithi165.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 697 kB/s | 133 kB 00:00 2024-05-11T13:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 961 kB/s | 87 kB 00:00 2024-05-11T13:14:27.155 INFO:teuthology.orchestra.run.smithi165.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-05-11T13:14:27.255 INFO:teuthology.orchestra.run.smithi165.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-11T13:14:27.279 INFO:teuthology.orchestra.run.smithi071.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 8.6 MB/s | 4.7 MB 00:00 2024-05-11T13:14:27.338 INFO:teuthology.orchestra.run.smithi071.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-05-11T13:14:27.389 INFO:teuthology.orchestra.run.smithi165.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-05-11T13:14:27.413 INFO:teuthology.orchestra.run.smithi071.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-05-11T13:14:27.446 INFO:teuthology.orchestra.run.smithi071.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-11T13:14:27.447 INFO:teuthology.orchestra.run.smithi165.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-05-11T13:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 181 kB/s | 26 kB 00:00 2024-05-11T13:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-11T13:14:27.580 INFO:teuthology.orchestra.run.smithi071.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-05-11T13:14:27.665 INFO:teuthology.orchestra.run.smithi165.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-05-11T13:14:27.689 INFO:teuthology.orchestra.run.smithi071.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 811 kB/s | 87 kB 00:00 2024-05-11T13:14:27.715 INFO:teuthology.orchestra.run.smithi165.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 220 kB/s | 35 kB 00:00 2024-05-11T13:14:27.740 INFO:teuthology.orchestra.run.smithi165.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 826 kB/s | 172 kB 00:00 2024-05-11T13:14:27.816 INFO:teuthology.orchestra.run.smithi165.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-05-11T13:14:27.849 INFO:teuthology.orchestra.run.smithi165.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-11T13:14:27.908 INFO:teuthology.orchestra.run.smithi165.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 849 kB/s | 77 kB 00:00 2024-05-11T13:14:27.933 INFO:teuthology.orchestra.run.smithi165.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 511 kB/s | 42 kB 00:00 2024-05-11T13:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-11T13:14:28.084 INFO:teuthology.orchestra.run.smithi165.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-11T13:14:28.117 INFO:teuthology.orchestra.run.smithi165.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-05-11T13:14:28.167 INFO:teuthology.orchestra.run.smithi165.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 617 kB/s | 51 kB 00:00 2024-05-11T13:14:28.193 INFO:teuthology.orchestra.run.smithi165.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 718 kB/s | 54 kB 00:00 2024-05-11T13:14:28.218 INFO:teuthology.orchestra.run.smithi165.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-05-11T13:14:28.268 INFO:teuthology.orchestra.run.smithi165.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 560 kB/s | 42 kB 00:00 2024-05-11T13:14:28.293 INFO:teuthology.orchestra.run.smithi165.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 596 kB/s | 44 kB 00:00 2024-05-11T13:14:28.352 INFO:teuthology.orchestra.run.smithi165.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-05-11T13:14:28.394 INFO:teuthology.orchestra.run.smithi165.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-05-11T13:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 577 kB/s | 47 kB 00:00 2024-05-11T13:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-11T13:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 582 kB/s | 58 kB 00:00 2024-05-11T13:14:28.603 INFO:teuthology.orchestra.run.smithi165.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-05-11T13:14:28.620 INFO:teuthology.orchestra.run.smithi165.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 770 kB/s | 64 kB 00:00 2024-05-11T13:14:28.679 INFO:teuthology.orchestra.run.smithi165.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-05-11T13:14:28.738 INFO:teuthology.orchestra.run.smithi165.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 192 kB/s | 22 kB 00:00 2024-05-11T13:14:28.821 INFO:teuthology.orchestra.run.smithi071.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 4.1 MB/s | 5.8 MB 00:01 2024-05-11T13:14:28.948 INFO:teuthology.orchestra.run.smithi071.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-11T13:14:28.955 INFO:teuthology.orchestra.run.smithi165.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-05-11T13:14:29.238 INFO:teuthology.orchestra.run.smithi165.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-05-11T13:14:29.298 INFO:teuthology.orchestra.run.smithi071.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 710 kB/s | 248 kB 00:00 2024-05-11T13:14:29.407 INFO:teuthology.orchestra.run.smithi071.stdout:(76/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-05-11T13:14:29.432 INFO:teuthology.orchestra.run.smithi071.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-05-11T13:14:29.508 INFO:teuthology.orchestra.run.smithi071.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 193 kB/s | 19 kB 00:00 2024-05-11T13:14:29.641 INFO:teuthology.orchestra.run.smithi071.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-05-11T13:14:29.647 INFO:teuthology.orchestra.run.smithi165.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 48 kB/s | 19 kB 00:00 2024-05-11T13:14:29.724 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-11T13:14:29.739 INFO:teuthology.orchestra.run.smithi165.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 432 kB/s | 309 kB 00:00 2024-05-11T13:14:29.742 INFO:teuthology.orchestra.run.smithi071.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 558 kB/s | 172 kB 00:00 2024-05-11T13:14:29.780 INFO:teuthology.orchestra.run.smithi165.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 526 kB/s | 548 kB 00:01 2024-05-11T13:14:29.847 INFO:teuthology.orchestra.run.smithi165.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-05-11T13:14:29.906 INFO:teuthology.orchestra.run.smithi165.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-05-11T13:14:29.965 INFO:teuthology.orchestra.run.smithi165.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 578 kB/s | 67 kB 00:00 2024-05-11T13:14:29.985 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-11T13:14:29.985 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-11T13:14:30.032 INFO:teuthology.orchestra.run.smithi165.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-11T13:14:30.108 INFO:teuthology.orchestra.run.smithi071.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 685 kB/s | 250 kB 00:00 2024-05-11T13:14:30.224 INFO:teuthology.orchestra.run.smithi165.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-05-11T13:14:30.274 INFO:teuthology.orchestra.run.smithi165.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-05-11T13:14:30.333 INFO:teuthology.orchestra.run.smithi165.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-05-11T13:14:30.432 INFO:teuthology.orchestra.run.smithi165.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-05-11T13:14:30.458 INFO:teuthology.orchestra.run.smithi165.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-05-11T13:14:30.467 INFO:teuthology.orchestra.run.smithi071.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 621 kB/s | 222 kB 00:00 2024-05-11T13:14:30.483 INFO:teuthology.orchestra.run.smithi165.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 154 kB/s | 32 kB 00:00 2024-05-11T13:14:30.559 INFO:teuthology.orchestra.run.smithi165.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-11T13:14:30.592 INFO:teuthology.orchestra.run.smithi165.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-11T13:14:30.626 INFO:teuthology.orchestra.run.smithi165.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-05-11T13:14:30.650 INFO:teuthology.orchestra.run.smithi071.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 423 kB/s | 77 kB 00:00 2024-05-11T13:14:30.667 INFO:teuthology.orchestra.run.smithi165.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-05-11T13:14:30.726 INFO:teuthology.orchestra.run.smithi165.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-11T13:14:30.760 INFO:teuthology.orchestra.run.smithi071.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 391 kB/s | 42 kB 00:00 2024-05-11T13:14:30.793 INFO:teuthology.orchestra.run.smithi165.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-11T13:14:30.827 INFO:teuthology.orchestra.run.smithi165.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 108 kB/s | 11 kB 00:00 2024-05-11T13:14:30.911 INFO:teuthology.orchestra.run.smithi165.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-05-11T13:14:30.936 INFO:teuthology.orchestra.run.smithi165.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2024-05-11T13:14:30.961 INFO:teuthology.orchestra.run.smithi165.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-05-11T13:14:31.011 INFO:teuthology.orchestra.run.smithi165.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-05-11T13:14:31.193 INFO:teuthology.orchestra.run.smithi071.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 623 kB/s | 269 kB 00:00 2024-05-11T13:14:31.239 INFO:teuthology.orchestra.run.smithi165.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 71 kB/s | 20 kB 00:00 2024-05-11T13:14:31.262 INFO:teuthology.orchestra.run.smithi165.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 185 kB/s | 46 kB 00:00 2024-05-11T13:14:31.312 INFO:teuthology.orchestra.run.smithi165.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-05-11T13:14:31.354 INFO:teuthology.orchestra.run.smithi165.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 696 kB/s | 79 kB 00:00 2024-05-11T13:14:31.371 INFO:teuthology.orchestra.run.smithi165.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2024-05-11T13:14:31.421 INFO:teuthology.orchestra.run.smithi165.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-11T13:14:31.455 INFO:teuthology.orchestra.run.smithi165.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-05-11T13:14:31.468 INFO:teuthology.orchestra.run.smithi071.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 579 kB/s | 159 kB 00:00 2024-05-11T13:14:31.480 INFO:teuthology.orchestra.run.smithi165.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2024-05-11T13:14:31.513 INFO:teuthology.orchestra.run.smithi165.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-05-11T13:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-05-11T13:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-11T13:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-05-11T13:14:31.665 INFO:teuthology.orchestra.run.smithi165.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-05-11T13:14:31.698 INFO:teuthology.orchestra.run.smithi165.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-05-11T13:14:31.732 INFO:teuthology.orchestra.run.smithi165.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-11T13:14:31.807 INFO:teuthology.orchestra.run.smithi165.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 633 kB/s | 90 kB 00:00 2024-05-11T13:14:31.832 INFO:teuthology.orchestra.run.smithi165.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-05-11T13:14:31.866 INFO:teuthology.orchestra.run.smithi165.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-05-11T13:14:31.894 INFO:teuthology.orchestra.run.smithi071.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 657 kB/s | 279 kB 00:00 2024-05-11T13:14:31.950 INFO:teuthology.orchestra.run.smithi165.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-11T13:14:31.994 INFO:teuthology.orchestra.run.smithi071.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 514 kB/s | 51 kB 00:00 2024-05-11T13:14:32.016 INFO:teuthology.orchestra.run.smithi165.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-05-11T13:14:32.219 INFO:teuthology.orchestra.run.smithi071.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 239 kB/s | 54 kB 00:00 2024-05-11T13:14:32.242 INFO:teuthology.orchestra.run.smithi165.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-05-11T13:14:32.342 INFO:teuthology.orchestra.run.smithi165.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-05-11T13:14:32.384 INFO:teuthology.orchestra.run.smithi165.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-11T13:14:32.434 INFO:teuthology.orchestra.run.smithi165.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-05-11T13:14:32.518 INFO:teuthology.orchestra.run.smithi165.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-05-11T13:14:32.635 INFO:teuthology.orchestra.run.smithi165.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-05-11T13:14:32.782 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-11T13:14:32.787 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-11T13:14:33.276 INFO:teuthology.orchestra.run.smithi165.stdout:(155/156): librados2-19.0.0-3601.g30850680.el9. 2.4 MB/s | 3.4 MB 00:01 2024-05-11T13:14:33.995 INFO:teuthology.orchestra.run.smithi071.stdout:(90/156): python3-babel-2.9.1-2.el9.noarch.rpm 968 kB/s | 6.0 MB 00:06 2024-05-11T13:14:34.103 INFO:teuthology.orchestra.run.smithi071.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 388 kB/s | 42 kB 00:00 2024-05-11T13:14:34.193 INFO:teuthology.orchestra.run.smithi165.stdout:(156/156): librbd1-19.0.0-3601.g30850680.el9.x8 1.4 MB/s | 3.1 MB 00:02 2024-05-11T13:14:34.206 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-05-11T13:14:34.206 INFO:teuthology.orchestra.run.smithi165.stdout:Total 10 MB/s | 226 MB 00:22 2024-05-11T13:14:34.228 INFO:teuthology.orchestra.run.smithi071.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 355 kB/s | 44 kB 00:00 2024-05-11T13:14:34.595 INFO:teuthology.orchestra.run.smithi071.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 603 kB/s | 221 kB 00:00 2024-05-11T13:14:34.704 INFO:teuthology.orchestra.run.smithi071.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 333 kB/s | 36 kB 00:00 2024-05-11T13:14:34.836 INFO:teuthology.orchestra.run.smithi071.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 360 kB/s | 47 kB 00:00 2024-05-11T13:14:35.321 INFO:teuthology.orchestra.run.smithi071.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 630 kB/s | 305 kB 00:00 2024-05-11T13:14:35.447 INFO:teuthology.orchestra.run.smithi071.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 463 kB/s | 58 kB 00:00 2024-05-11T13:14:35.664 INFO:teuthology.orchestra.run.smithi071.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 433 kB/s | 94 kB 00:00 2024-05-11T13:14:35.897 INFO:teuthology.orchestra.run.smithi071.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 274 kB/s | 64 kB 00:00 2024-05-11T13:14:36.036 INFO:teuthology.orchestra.run.smithi071.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 325 kB/s | 45 kB 00:00 2024-05-11T13:14:36.156 INFO:teuthology.orchestra.run.smithi071.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 187 kB/s | 22 kB 00:00 2024-05-11T13:14:36.615 INFO:teuthology.orchestra.run.smithi071.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-05-11T13:14:37.057 INFO:teuthology.orchestra.run.smithi071.stdout:(103/156): python3-numpy-1.20.1-5.el9.x86_64.rp 704 kB/s | 5.1 MB 00:07 2024-05-11T13:14:37.099 INFO:teuthology.orchestra.run.smithi071.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-05-11T13:14:37.161 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-05-11T13:14:37.262 INFO:teuthology.orchestra.run.smithi071.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 120 kB/s | 19 kB 00:00 2024-05-11T13:14:37.404 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-05-11T13:14:37.405 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-05-11T13:14:37.450 INFO:teuthology.orchestra.run.smithi071.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 789 kB/s | 309 kB 00:00 2024-05-11T13:14:37.550 INFO:teuthology.orchestra.run.smithi071.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-05-11T13:14:37.659 INFO:teuthology.orchestra.run.smithi071.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-11T13:14:37.684 INFO:teuthology.orchestra.run.smithi071.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 366 kB/s | 49 kB 00:00 2024-05-11T13:14:37.759 INFO:teuthology.orchestra.run.smithi071.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 677 kB/s | 67 kB 00:00 2024-05-11T13:14:37.793 INFO:teuthology.orchestra.run.smithi071.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-11T13:14:37.902 INFO:teuthology.orchestra.run.smithi071.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-05-11T13:14:37.969 INFO:teuthology.orchestra.run.smithi071.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-05-11T13:14:37.994 INFO:teuthology.orchestra.run.smithi071.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-05-11T13:14:38.061 INFO:teuthology.orchestra.run.smithi071.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 352 kB/s | 32 kB 00:00 2024-05-11T13:14:38.086 INFO:teuthology.orchestra.run.smithi071.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-05-11T13:14:38.170 INFO:teuthology.orchestra.run.smithi071.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-05-11T13:14:38.212 INFO:teuthology.orchestra.run.smithi071.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-11T13:14:38.287 INFO:teuthology.orchestra.run.smithi071.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-11T13:14:38.313 INFO:teuthology.orchestra.run.smithi071.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-05-11T13:14:38.388 INFO:teuthology.orchestra.run.smithi071.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-11T13:14:38.480 INFO:teuthology.orchestra.run.smithi071.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-05-11T13:14:38.572 INFO:teuthology.orchestra.run.smithi071.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-05-11T13:14:38.681 INFO:teuthology.orchestra.run.smithi071.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-05-11T13:14:38.756 INFO:teuthology.orchestra.run.smithi071.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-05-11T13:14:38.773 INFO:teuthology.orchestra.run.smithi071.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 254 kB/s | 23 kB 00:00 2024-05-11T13:14:38.840 INFO:teuthology.orchestra.run.smithi071.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 234 kB/s | 19 kB 00:00 2024-05-11T13:14:38.857 INFO:teuthology.orchestra.run.smithi071.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-05-11T13:14:38.949 INFO:teuthology.orchestra.run.smithi071.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 508 kB/s | 46 kB 00:00 2024-05-11T13:14:39.041 INFO:teuthology.orchestra.run.smithi071.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 866 kB/s | 79 kB 00:00 2024-05-11T13:14:39.083 INFO:teuthology.orchestra.run.smithi071.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-05-11T13:14:39.183 INFO:teuthology.orchestra.run.smithi071.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-11T13:14:39.208 INFO:teuthology.orchestra.run.smithi071.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 347 kB/s | 58 kB 00:00 2024-05-11T13:14:39.267 INFO:teuthology.orchestra.run.smithi071.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-05-11T13:14:39.301 INFO:teuthology.orchestra.run.smithi071.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 984 kB/s | 90 kB 00:00 2024-05-11T13:14:39.393 INFO:teuthology.orchestra.run.smithi071.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-05-11T13:14:39.418 INFO:teuthology.orchestra.run.smithi071.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 264 kB/s | 31 kB 00:00 2024-05-11T13:14:39.493 INFO:teuthology.orchestra.run.smithi071.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-05-11T13:14:39.519 INFO:teuthology.orchestra.run.smithi071.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 593 kB/s | 59 kB 00:00 2024-05-11T13:14:39.569 INFO:teuthology.orchestra.run.smithi071.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-05-11T13:14:39.603 INFO:teuthology.orchestra.run.smithi071.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 614 kB/s | 51 kB 00:00 2024-05-11T13:14:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-11T13:14:39.703 INFO:teuthology.orchestra.run.smithi071.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 903 kB/s | 90 kB 00:00 2024-05-11T13:14:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T13:14:39.762 INFO:teuthology.orchestra.run.smithi071.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-05-11T13:14:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T13:14:39.815 INFO:teuthology.orchestra.run.smithi071.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-05-11T13:14:39.846 INFO:teuthology.orchestra.run.smithi071.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-05-11T13:14:39.913 INFO:teuthology.orchestra.run.smithi071.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-11T13:14:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T13:14:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T13:14:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T13:14:40.029 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-05-11T13:14:40.029 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-05-11T13:14:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T13:14:40.213 INFO:teuthology.orchestra.run.smithi071.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-05-11T13:14:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T13:14:40.639 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:14:40.664 INFO:teuthology.orchestra.run.smithi071.stdout:(148/156): librbd1-19.0.0-3601.g30850680.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-05-11T13:14:40.823 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:14:40.857 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:14:40.994 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:14:41.014 INFO:teuthology.orchestra.run.smithi071.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 6.6 MB/s | 2.3 MB 00:00 2024-05-11T13:14:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T13:14:41.231 INFO:teuthology.orchestra.run.smithi071.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 8.5 MB/s | 1.8 MB 00:00 2024-05-11T13:14:41.290 INFO:teuthology.orchestra.run.smithi071.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 725 kB/s | 42 kB 00:00 2024-05-11T13:14:41.352 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T13:14:41.365 INFO:teuthology.orchestra.run.smithi071.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.3 MB/s | 248 kB 00:00 2024-05-11T13:14:41.411 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T13:14:41.436 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:14:41.448 INFO:teuthology.orchestra.run.smithi071.stdout:(153/156): librados2-19.0.0-3601.g30850680.el9. 2.2 MB/s | 3.4 MB 00:01 2024-05-11T13:14:41.540 INFO:teuthology.orchestra.run.smithi071.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-05-11T13:14:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:14:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T13:14:41.698 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T13:14:41.730 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T13:14:41.800 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T13:14:41.816 INFO:teuthology.orchestra.run.smithi071.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 2.4 MB 00:00 2024-05-11T13:14:42.119 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T13:14:42.441 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T13:14:42.586 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T13:14:42.673 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T13:14:42.825 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:14:42.939 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:14:43.251 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:14:43.404 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T13:14:43.454 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T13:14:43.498 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T13:14:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T13:14:43.657 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T13:14:43.721 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T13:14:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T13:14:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T13:14:43.937 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T13:14:44.010 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T13:14:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T13:14:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T13:14:44.241 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T13:14:44.281 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T13:14:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T13:14:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T13:14:44.498 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T13:14:44.570 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T13:14:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T13:14:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T13:14:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T13:14:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T13:14:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T13:14:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T13:14:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T13:14:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T13:14:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T13:14:46.143 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T13:14:46.204 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T13:14:46.332 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T13:14:46.408 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T13:14:46.446 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-05-11T13:14:46.493 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T13:14:46.529 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T13:14:46.549 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T13:14:46.567 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T13:14:46.622 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T13:14:46.631 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T13:14:46.677 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T13:14:46.731 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T13:14:46.738 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T13:14:46.797 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T13:14:46.805 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T13:14:46.843 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T13:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T13:14:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T13:14:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T13:14:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T13:14:47.179 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T13:14:47.321 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T13:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T13:14:47.369 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T13:14:47.428 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T13:14:47.469 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T13:14:47.474 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:14:47.519 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T13:14:47.641 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:14:47.683 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:14:47.741 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T13:14:47.812 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:14:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T13:14:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T13:14:47.973 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T13:14:48.010 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T13:14:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T13:14:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T13:14:48.157 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T13:14:48.185 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T13:14:48.211 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T13:14:48.236 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T13:14:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:14:48.283 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T13:14:48.333 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T13:14:48.390 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:14:48.411 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T13:14:48.430 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T13:14:48.456 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T13:14:48.532 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T13:14:48.564 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T13:14:48.602 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T13:14:48.642 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T13:14:48.662 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T13:14:48.704 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T13:14:48.775 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T13:14:48.936 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T13:14:48.940 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T13:14:49.001 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T13:14:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T13:14:49.461 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T13:14:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T13:14:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:14:49.873 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:14:50.011 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T13:14:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T13:14:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T13:14:50.235 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T13:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:14:50.302 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T13:14:50.413 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T13:14:50.458 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T13:14:50.473 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T13:14:50.512 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T13:14:50.571 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T13:14:50.696 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T13:14:50.715 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T13:14:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T13:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T13:14:50.773 INFO:teuthology.orchestra.run.smithi071.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 897 kB/s | 16 MB 00:18 2024-05-11T13:14:50.779 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-05-11T13:14:50.780 INFO:teuthology.orchestra.run.smithi071.stdout:Total 5.5 MB/s | 226 MB 00:41 2024-05-11T13:14:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T13:14:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T13:14:50.889 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T13:14:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T13:14:51.045 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T13:14:51.125 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T13:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T13:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T13:14:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T13:14:51.391 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T13:14:51.449 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T13:14:51.481 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T13:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T13:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T13:14:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T13:14:51.769 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T13:14:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T13:14:51.942 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T13:14:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T13:14:52.713 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T13:14:52.809 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T13:14:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T13:14:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T13:14:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T13:14:52.987 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T13:14:53.180 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T13:14:53.254 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T13:14:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T13:14:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T13:14:53.541 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T13:14:53.579 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T13:14:53.598 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T13:14:53.617 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T13:14:53.684 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T13:14:53.693 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T13:14:53.730 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T13:14:53.735 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T13:14:53.761 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T13:14:53.798 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T13:14:53.824 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-05-11T13:14:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T13:14:54.051 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T13:14:54.064 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-05-11T13:14:54.064 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-05-11T13:14:54.128 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T13:14:54.148 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T13:14:54.172 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T13:14:54.189 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T13:14:54.231 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T13:14:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T13:14:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T13:14:54.368 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T13:14:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T13:14:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T13:14:54.466 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T13:14:54.503 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T13:14:54.535 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T13:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T13:14:54.913 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T13:14:54.957 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T13:14:54.973 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T13:14:55.000 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T13:14:55.000 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T13:14:55.046 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T13:14:55.075 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T13:14:55.092 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T13:14:55.102 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T13:14:55.139 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T13:14:55.184 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T13:14:55.230 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T13:14:55.272 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T13:14:55.337 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T13:14:55.378 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T13:14:55.517 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T13:14:55.583 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T13:14:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T13:14:55.684 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T13:14:55.847 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T13:14:55.913 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T13:14:56.722 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-05-11T13:14:56.723 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-05-11T13:14:56.803 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T13:14:56.866 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T13:14:56.898 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T13:14:57.022 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T13:14:57.093 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T13:14:57.193 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T13:14:57.241 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T13:14:57.478 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T13:14:57.514 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T13:14:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T13:14:57.923 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:14:57.973 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:14:58.454 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T13:14:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T13:14:58.657 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:14:58.701 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T13:14:58.853 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T13:14:58.853 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T13:14:58.853 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T13:14:58.854 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:14:58.894 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T13:15:00.221 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T13:15:00.327 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T13:15:00.362 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T13:15:00.399 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T13:15:00.773 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T13:15:00.822 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T13:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T13:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T13:15:01.037 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T13:15:01.584 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T13:15:01.617 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T13:15:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T13:15:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T13:15:03.352 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-05-11T13:15:03.480 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T13:15:03.557 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T13:15:03.700 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T13:15:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T13:15:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T13:15:03.926 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T13:15:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T13:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:15:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:15:04.489 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:15:04.602 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:15:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:15:04.792 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:15:04.829 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T13:15:05.127 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T13:15:05.153 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:15:05.193 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T13:15:05.218 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T13:15:05.219 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:15:05.366 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:15:05.380 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T13:15:05.380 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T13:15:05.381 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T13:15:05.381 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:05.394 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T13:15:05.419 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T13:15:05.497 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T13:15:05.529 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T13:15:05.599 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T13:15:06.012 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T13:15:06.257 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T13:15:06.461 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T13:15:06.549 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T13:15:06.727 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:15:06.831 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:15:07.135 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T13:15:07.339 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T13:15:07.370 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T13:15:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T13:15:07.551 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T13:15:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T13:15:07.685 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T13:15:07.730 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T13:15:07.770 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T13:15:07.886 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T13:15:07.968 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T13:15:08.005 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T13:15:08.060 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T13:15:08.169 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T13:15:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T13:15:08.291 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T13:15:08.325 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T13:15:08.447 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T13:15:08.504 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T13:15:08.613 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T13:15:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T13:15:08.881 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T13:15:09.579 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T13:15:09.688 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T13:15:09.723 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T13:15:09.767 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T13:15:09.808 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T13:15:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T13:15:10.094 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T13:15:10.175 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T13:15:10.293 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T13:15:10.369 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T13:15:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T13:15:10.485 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T13:15:10.523 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T13:15:10.586 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T13:15:10.631 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T13:15:10.714 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T13:15:10.780 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T13:15:10.988 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T13:15:11.039 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T13:15:11.068 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T13:15:11.131 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T13:15:11.260 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T13:15:11.302 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T13:15:11.349 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T13:15:11.384 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T13:15:11.416 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T13:15:11.606 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T13:15:11.788 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T13:15:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T13:15:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T13:15:11.931 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T13:15:11.990 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T13:15:12.045 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T13:15:12.086 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T13:15:12.132 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T13:15:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T13:15:12.258 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T13:15:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-05-11T13:15:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:12.330 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T13:15:12.478 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T13:15:12.537 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T13:15:12.555 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T13:15:12.585 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T13:15:12.585 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:12.585 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T13:15:12.585 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:12.597 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T13:15:12.669 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T13:15:12.836 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T13:15:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T13:15:13.867 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T13:15:13.926 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T13:15:13.960 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T13:15:14.074 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T13:15:14.147 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T13:15:14.251 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T13:15:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T13:15:14.319 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T13:15:14.365 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T13:15:14.366 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:14.366 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T13:15:14.366 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T13:15:14.366 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T13:15:14.366 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:14.558 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T13:15:14.593 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T13:15:14.622 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T13:15:14.705 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T13:15:14.707 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T13:15:14.707 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:14.707 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T13:15:14.707 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T13:15:14.708 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T13:15:14.708 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:15.215 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T13:15:15.310 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T13:15:15.311 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:15.311 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T13:15:15.311 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T13:15:15.311 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T13:15:15.311 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:15.450 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T13:15:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T13:15:15.557 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T13:15:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T13:15:15.707 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T13:15:15.836 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T13:15:15.896 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T13:15:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T13:15:16.104 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T13:15:16.274 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T13:15:16.320 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T13:15:16.368 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T13:15:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T13:15:17.104 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T13:15:17.148 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T13:15:17.205 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T13:15:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T13:15:17.316 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T13:15:17.348 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T13:15:17.400 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T13:15:17.436 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T13:15:17.470 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T13:15:17.893 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T13:15:18.046 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T13:15:18.201 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T13:15:18.237 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T13:15:18.300 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T13:15:18.720 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T13:15:18.767 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-05-11T13:15:18.848 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T13:15:18.924 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T13:15:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T13:15:19.052 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T13:15:19.067 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T13:15:19.111 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T13:15:19.112 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:19.112 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T13:15:19.112 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:19.932 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T13:15:19.967 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T13:15:20.133 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T13:15:20.320 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T13:15:20.344 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T13:15:20.417 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T13:15:20.417 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:20.417 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T13:15:20.417 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T13:15:20.417 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T13:15:20.417 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:20.470 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T13:15:20.510 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T13:15:20.617 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T13:15:20.685 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T13:15:20.777 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T13:15:20.856 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T13:15:20.856 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:20.856 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T13:15:20.856 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T13:15:20.856 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T13:15:20.857 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:21.102 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T13:15:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T13:15:21.170 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:21.170 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T13:15:21.170 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T13:15:21.170 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T13:15:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:21.678 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T13:15:21.745 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T13:15:21.749 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T13:15:21.749 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:21.750 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T13:15:21.750 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T13:15:21.750 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T13:15:21.750 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:21.774 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:15:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T13:15:21.817 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:15:21.873 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T13:15:21.874 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:21.874 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T13:15:21.874 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T13:15:21.874 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T13:15:21.874 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:21.878 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T13:15:21.923 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T13:15:21.941 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T13:15:22.013 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T13:15:22.018 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T13:15:22.018 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:22.018 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T13:15:22.019 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:22.090 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T13:15:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T13:15:22.200 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:22.201 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T13:15:22.201 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T13:15:22.201 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T13:15:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:22.246 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T13:15:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T13:15:22.513 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T13:15:22.559 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T13:15:22.600 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T13:15:22.664 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T13:15:22.664 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T13:15:22.664 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T13:15:22.664 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T13:15:22.681 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T13:15:22.681 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:22.681 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T13:15:22.681 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T13:15:22.681 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T13:15:22.681 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:22.735 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T13:15:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T13:15:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T13:15:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T13:15:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T13:15:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T13:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T13:15:23.749 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T13:15:23.795 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T13:15:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T13:15:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T13:15:25.259 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T13:15:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T13:15:26.411 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T13:15:26.444 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T13:15:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T13:15:26.783 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T13:15:26.813 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T13:15:26.884 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T13:15:26.884 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:26.884 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T13:15:26.884 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T13:15:26.884 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T13:15:26.884 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:26.939 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T13:15:26.976 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T13:15:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T13:15:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T13:15:28.103 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T13:15:28.172 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T13:15:28.225 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T13:15:28.245 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T13:15:28.257 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T13:15:28.283 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T13:15:28.323 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T13:15:28.351 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T13:15:28.351 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:28.351 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T13:15:28.351 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T13:15:28.351 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T13:15:28.351 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:28.416 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T13:15:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T13:15:28.512 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:28.512 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T13:15:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:28.592 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T13:15:28.636 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T13:15:28.694 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T13:15:28.696 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T13:15:28.696 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:28.696 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T13:15:28.697 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T13:15:28.697 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T13:15:28.697 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T13:15:28.767 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T13:15:28.816 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T13:15:29.093 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T13:15:29.172 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T13:15:29.172 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:29.172 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T13:15:29.172 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T13:15:29.172 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T13:15:29.172 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:29.805 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:29.864 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:31.411 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T13:15:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T13:15:31.576 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T13:15:31.844 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T13:15:31.889 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T13:15:31.924 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T13:15:31.925 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T13:15:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T13:15:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T13:15:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T13:15:32.247 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T13:15:32.285 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T13:15:32.319 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:32.319 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:32.665 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:32.665 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T13:15:32.818 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T13:15:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:33.276 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:33.277 INFO:teuthology.orchestra.run.smithi063.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-11T13:15:33.277 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T13:15:34.308 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T13:15:34.373 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T13:15:34.408 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T13:15:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T13:15:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T13:15:34.812 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T13:15:34.840 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T13:15:34.883 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T13:15:34.925 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-05-11T13:15:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:35.868 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:35.913 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:35.976 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:36.130 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T13:15:36.177 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T13:15:36.177 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:36.177 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T13:15:36.177 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:37.319 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T13:15:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T13:15:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T13:15:37.635 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T13:15:37.679 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T13:15:37.716 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T13:15:37.716 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T13:15:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T13:15:37.849 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T13:15:37.891 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T13:15:37.891 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T13:15:37.924 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T13:15:37.924 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:37.924 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T13:15:37.924 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T13:15:37.924 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T13:15:37.924 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:38.053 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T13:15:38.092 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T13:15:38.124 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:38.124 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T13:15:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T13:15:38.238 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:38.238 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T13:15:38.238 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T13:15:38.238 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T13:15:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:38.464 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:38.464 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T13:15:38.608 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T13:15:38.716 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T13:15:38.736 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:38.794 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T13:15:38.795 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:38.795 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T13:15:38.795 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T13:15:38.795 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T13:15:38.795 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:38.851 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T13:15:38.851 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T13:15:38.851 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T13:15:38.851 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T13:15:38.851 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T13:15:38.851 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T13:15:38.852 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T13:15:38.853 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T13:15:38.866 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T13:15:38.866 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T13:15:38.866 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T13:15:38.866 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T13:15:38.866 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T13:15:38.866 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T13:15:38.867 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T13:15:38.868 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T13:15:38.869 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T13:15:38.870 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T13:15:38.871 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T13:15:38.872 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T13:15:38.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T13:15:38.874 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T13:15:38.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T13:15:38.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T13:15:38.934 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T13:15:38.996 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T13:15:39.009 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:39.009 INFO:teuthology.orchestra.run.smithi165.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-11T13:15:39.009 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:39.070 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T13:15:39.303 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T13:15:39.362 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T13:15:39.449 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T13:15:39.557 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T13:15:39.751 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T13:15:39.795 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T13:15:39.843 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T13:15:39.869 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T13:15:40.513 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T13:15:40.546 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T13:15:40.601 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T13:15:40.711 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T13:15:40.740 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T13:15:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T13:15:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T13:15:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T13:15:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T13:15:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T13:15:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T13:15:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T13:15:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T13:15:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T13:15:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T13:15:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T13:15:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T13:15:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T13:15:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T13:15:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T13:15:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T13:15:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T13:15:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T13:15:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T13:15:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:15:41.079 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-11T13:15:41.670 DEBUG:teuthology.parallel:result is None 2024-05-11T13:15:42.057 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T13:15:42.109 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T13:15:43.209 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T13:15:43.242 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T13:15:43.398 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T13:15:43.591 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T13:15:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T13:15:43.694 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T13:15:43.694 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:43.694 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T13:15:43.694 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T13:15:43.695 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T13:15:43.695 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:43.753 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T13:15:43.791 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T13:15:43.943 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T13:15:44.013 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T13:15:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:15:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T13:15:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T13:15:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T13:15:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T13:15:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T13:15:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T13:15:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T13:15:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T13:15:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T13:15:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T13:15:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T13:15:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T13:15:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T13:15:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T13:15:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T13:15:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T13:15:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T13:15:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T13:15:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T13:15:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T13:15:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T13:15:45.056 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T13:15:45.097 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T13:15:45.163 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T13:15:45.163 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:45.163 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T13:15:45.163 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T13:15:45.163 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T13:15:45.163 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:45.231 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T13:15:45.300 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T13:15:45.300 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:45.300 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T13:15:45.300 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T13:15:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T13:15:45.463 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:45.463 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T13:15:45.463 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T13:15:45.464 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T13:15:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T13:15:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T13:15:45.915 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T13:15:45.915 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T13:15:45.915 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T13:15:45.915 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T13:15:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.647 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.649 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T13:15:46.650 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T13:15:46.651 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T13:15:46.652 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T13:15:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T13:15:46.654 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T13:15:46.655 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T13:15:46.656 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T13:15:46.657 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T13:15:46.658 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T13:15:46.659 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T13:15:46.659 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:15:46.659 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-05-11T13:15:47.224 DEBUG:teuthology.parallel:result is None 2024-05-11T13:15:51.018 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T13:15:51.076 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T13:15:51.141 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T13:15:51.181 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T13:15:51.220 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T13:15:51.547 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T13:15:51.611 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T13:15:51.639 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T13:15:51.713 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T13:15:51.766 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T13:15:52.788 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:52.838 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:52.889 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T13:15:54.241 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T13:15:54.353 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T13:15:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T13:15:54.589 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T13:15:54.629 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T13:15:54.675 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T13:15:54.675 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T13:15:54.812 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T13:15:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T13:15:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T13:15:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T13:15:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T13:15:55.075 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:55.075 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:55.414 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T13:15:55.414 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T13:15:55.558 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T13:15:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:55.959 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:15:55.959 INFO:teuthology.orchestra.run.smithi071.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-11T13:15:55.959 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:16:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T13:16:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T13:16:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T13:16:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T13:16:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T13:16:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T13:16:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T13:16:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T13:16:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T13:16:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T13:16:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T13:16:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T13:16:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T13:16:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T13:16:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T13:16:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T13:16:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T13:16:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T13:16:04.258 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T13:16:04.258 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:16:04.258 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-05-11T13:16:04.258 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T13:16:04.258 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.260 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T13:16:04.261 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.262 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T13:16:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T13:16:04.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T13:16:04.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T13:16:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T13:16:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T13:16:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T13:16:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T13:16:04.270 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-11T13:16:04.270 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-05-11T13:16:04.890 DEBUG:teuthology.parallel:result is None 2024-05-11T13:16:04.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:16:05.226 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T13:16:05.303 INFO:teuthology.orchestra.run.smithi063.stdout:19.0.0-3601.g30850680.el9 2024-05-11T13:16:05.304 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T13:16:05.304 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T13:16:05.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:16:05.531 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T13:16:05.582 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-3601.g30850680.el9 2024-05-11T13:16:05.583 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T13:16:05.583 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T13:16:05.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:16:05.826 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T13:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-3601.g30850680.el9 2024-05-11T13:16:05.894 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T13:16:05.894 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T13:16:05.896 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-11T13:16:05.896 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:16:05.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T13:16:05.991 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:16:05.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T13:16:06.056 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:16:06.056 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T13:16:06.150 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-11T13:16:06.150 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:16:06.150 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T13:16:06.245 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T13:16:06.333 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:16:06.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T13:16:06.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T13:16:06.496 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:16:06.496 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T13:16:06.585 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T13:16:06.715 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-11T13:16:06.715 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:16:06.715 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T13:16:06.810 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T13:16:06.942 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:16:06.942 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T13:16:07.004 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T13:16:07.118 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:16:07.118 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T13:16:07.206 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T13:16:07.335 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-11T13:16:07.335 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:16:07.335 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T13:16:07.424 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T13:16:07.505 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:16:07.506 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T13:16:07.583 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T13:16:07.690 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:16:07.690 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T13:16:07.780 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T13:16:07.865 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-11T13:16:07.971 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-11T13:16:07.971 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:16:07.971 INFO:tasks.cephadm:Cluster fsid is a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:16:07.971 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-11T13:16:07.972 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.63', 'mon.b': '172.21.15.71', 'mon.c': '172.21.15.165'} 2024-05-11T13:16:07.972 INFO:tasks.cephadm:First mon is mon.a on smithi063 2024-05-11T13:16:07.972 INFO:tasks.cephadm:First mgr is x 2024-05-11T13:16:07.972 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-11T13:16:07.972 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname $(hostname -s) 2024-05-11T13:16:08.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-05-11T13:16:08.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-05-11T13:16:08.250 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-11T13:16:08.250 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:16:08.464 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79287/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3601-g30850680', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3601.g30850680'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-10 12:10:01.412462', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240510.053205', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-11T13:16:08.605 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-testing-20240510.053205, sha1 308506802b0bdea4df7050cfc1fcbdc04fea6a6e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:16:08.606 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm 2024-05-11T13:16:08.606 DEBUG:teuthology.orchestra.run.smithi063:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:09.101 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r--. 1 ubuntu ubuntu 782934 May 11 13:16 /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:09.101 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:09.727 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 782934 May 11 13:16 /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:09.728 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:10.210 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 782934 May 11 13:16 /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:10.210 DEBUG:teuthology.orchestra.run.smithi063:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:10.273 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:10.334 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-11T13:16:10.411 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e on all hosts... 2024-05-11T13:16:10.412 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e pull 2024-05-11T13:16:10.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e pull 2024-05-11T13:16:10.425 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e pull 2024-05-11T13:16:12.435 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T13:16:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T13:16:12.571 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T13:16:48.364 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-11T13:16:48.364 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T13:16:48.365 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305", 2024-05-11T13:16:48.365 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-05-11T13:16:48.365 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8" 2024-05-11T13:16:48.365 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-05-11T13:16:48.365 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-11T13:16:53.933 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-11T13:16:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T13:16:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305", 2024-05-11T13:16:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-05-11T13:16:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8" 2024-05-11T13:16:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-05-11T13:16:53.933 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-11T13:17:09.646 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:17:09.646 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T13:17:09.646 INFO:teuthology.orchestra.run.smithi063.stdout: "image_id": "3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305", 2024-05-11T13:17:09.646 INFO:teuthology.orchestra.run.smithi063.stdout: "repo_digests": [ 2024-05-11T13:17:09.646 INFO:teuthology.orchestra.run.smithi063.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8" 2024-05-11T13:17:09.646 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:17:09.647 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:17:09.714 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph 2024-05-11T13:17:09.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-05-11T13:17:09.905 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-05-11T13:17:09.999 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /etc/ceph 2024-05-11T13:17:10.093 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-05-11T13:17:10.184 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-05-11T13:17:10.272 INFO:tasks.cephadm:Writing seed config... 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-11T13:17:10.274 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-11T13:17:10.275 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-11T13:17:10.276 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-11T13:17:10.277 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-11T13:17:10.277 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:17:10.277 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-11T13:17:10.338 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a0d7c9ac-0f98-11ef-bc98-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-11T13:17:10.338 DEBUG:teuthology.orchestra.run.smithi063:mon.a> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a.service 2024-05-11T13:17:10.385 DEBUG:teuthology.orchestra.run.smithi063:mgr.x> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.x.service 2024-05-11T13:17:10.431 INFO:tasks.cephadm:Bootstrapping... 2024-05-11T13:17:10.431 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e -v bootstrap --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.63 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-11T13:17:10.707 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-11T13:17:10.707 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e', '-v', 'bootstrap', '--fsid', 'a0d7c9ac-0f98-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.63', '--skip-admin-label'] 2024-05-11T13:17:10.708 INFO:teuthology.orchestra.run.smithi063.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-11T13:17:10.708 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying podman|docker is present... 2024-05-11T13:17:10.765 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-11T13:17:10.765 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying lvm2 is present... 2024-05-11T13:17:10.765 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying time synchronization is in place... 2024-05-11T13:17:10.780 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-11T13:17:10.780 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-11T13:17:10.794 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-11T13:17:10.794 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-11T13:17:10.809 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-05-11T13:17:10.825 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-05-11T13:17:10.825 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-05-11T13:17:10.825 INFO:teuthology.orchestra.run.smithi063.stdout:Repeating the final host check... 2024-05-11T13:17:10.884 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-11T13:17:10.884 INFO:teuthology.orchestra.run.smithi063.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-11T13:17:10.884 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl is present 2024-05-11T13:17:10.884 INFO:teuthology.orchestra.run.smithi063.stdout:lvcreate is present 2024-05-11T13:17:10.899 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-11T13:17:10.899 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-11T13:17:10.913 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-11T13:17:10.913 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-11T13:17:10.929 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-05-11T13:17:10.951 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-05-11T13:17:10.951 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-05-11T13:17:10.951 INFO:teuthology.orchestra.run.smithi063.stdout:Host looks OK 2024-05-11T13:17:10.952 INFO:teuthology.orchestra.run.smithi063.stdout:Cluster fsid: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:10.952 INFO:teuthology.orchestra.run.smithi063.stdout:Acquiring lock 139996865441216 on /run/cephadm/a0d7c9ac-0f98-11ef-bc98-c7b262605968.lock 2024-05-11T13:17:10.952 INFO:teuthology.orchestra.run.smithi063.stdout:Lock 139996865441216 acquired on /run/cephadm/a0d7c9ac-0f98-11ef-bc98-c7b262605968.lock 2024-05-11T13:17:10.954 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 3300 ... 2024-05-11T13:17:10.955 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 6789 ... 2024-05-11T13:17:10.955 INFO:teuthology.orchestra.run.smithi063.stdout:Base mon IP(s) is [172.21.15.63:3300, 172.21.15.63:6789], mon addrv is [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-05-11T13:17:10.970 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.63 metric 100 2024-05-11T13:17:10.970 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.63 metric 100 2024-05-11T13:17:10.984 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-11T13:17:10.984 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-11T13:17:10.984 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1785sec hoplimit 64 pref medium 2024-05-11T13:17:10.996 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-11T13:17:10.996 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-11T13:17:10.996 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-11T13:17:10.996 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-11T13:17:10.996 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-11T13:17:10.996 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-11T13:17:10.999 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-05-11T13:17:10.999 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-05-11T13:17:10.999 INFO:teuthology.orchestra.run.smithi063.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-11T13:17:11.002 INFO:teuthology.orchestra.run.smithi063.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-11T13:17:11.004 INFO:teuthology.orchestra.run.smithi063.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T13:17:11.737 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 2024-05-11T13:17:11.738 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T13:17:11.738 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Getting image source signatures 2024-05-11T13:17:11.738 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:f023f0e3722bad1958426d3c75a01dbae989f69dabcafcf09827f24f8b5e0d6b 2024-05-11T13:17:11.738 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-11T13:17:11.738 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying config sha256:3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 2024-05-11T13:17:11.738 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-11T13:17:12.932 INFO:teuthology.orchestra.run.smithi063.stdout:ceph: stdout ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev) 2024-05-11T13:17:12.933 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph version: ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev) 2024-05-11T13:17:12.933 INFO:teuthology.orchestra.run.smithi063.stdout:Extracting ceph user uid/gid from container image... 2024-05-11T13:17:14.169 INFO:teuthology.orchestra.run.smithi063.stdout:stat: stdout 167 167 2024-05-11T13:17:14.169 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial keys... 2024-05-11T13:17:15.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQDabz9mFlZiJxAAKrwB8cKNHj1CAoKesSI4pQ== 2024-05-11T13:17:16.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQDbbz9mKnAPMBAAOgdI3i1fLU5jLoyDnF7MzQ== 2024-05-11T13:17:17.546 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQDcbz9mpcTSOBAAeMu+ywawJMxh50FEXOKAgQ== 2024-05-11T13:17:17.549 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial monmap... 2024-05-11T13:17:18.865 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool for a [v2:172.21.15.63:3300,v1:172.21.15.63:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout:setting min_mon_release = quincy 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: set fsid to a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-11T13:17:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:17:18.867 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mon... 2024-05-11T13:17:20.098 INFO:teuthology.orchestra.run.smithi063.stdout:create mon.a on 2024-05-11T13:17:20.519 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-11T13:17:20.786 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-11T13:17:21.116 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968.target. 2024-05-11T13:17:21.116 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968.target. 2024-05-11T13:17:21.554 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a 2024-05-11T13:17:21.555 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-11T13:17:21.868 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968.target.wants/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@.service. 2024-05-11T13:17:22.543 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:22 smithi063 podman[29425]: 2024-05-11 13:17:22.530669847 +0000 UTC m=+0.372886882 container init e0adb54f55e530899fd94a220a3d0f711b8d40fa63b19189ae4e925fa6e0efc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-11T13:17:22.543 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:22 smithi063 podman[29425]: 2024-05-11 13:17:22.541926836 +0000 UTC m=+0.384143870 container start e0adb54f55e530899fd94a220a3d0f711b8d40fa63b19189ae4e925fa6e0efc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T13:17:22.634 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T13:17:22.634 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-11T13:17:22.650 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T13:17:22.650 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-11T13:17:22.650 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-11T13:17:22.650 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-05-11T13:17:22.651 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon to start... 2024-05-11T13:17:22.651 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon... 2024-05-11T13:17:22.795 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:22 smithi063 bash[29425]: e0adb54f55e530899fd94a220a3d0f711b8d40fa63b19189ae4e925fa6e0efc9 2024-05-11T13:17:22.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:22 smithi063 systemd[1]: Started Ceph mon.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:17:22.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:22 smithi063 ceph-mon[29443]: mkfs a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:22.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:22 smithi063 ceph-mon[29443]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-11T13:17:23.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-11T13:17:23.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: monmap epoch 1 2024-05-11T13:17:23.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:23.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: last_changed 2024-05-11T13:17:18.239963+0000 2024-05-11T13:17:23.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:17:23.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: min_mon_release 19 (squid) 2024-05-11T13:17:23.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: election_strategy: 1 2024-05-11T13:17:23.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:17:23.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: fsmap 2024-05-11T13:17:23.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: osdmap e1: 0 total, 0 up, 0 in 2024-05-11T13:17:23.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: mgrmap e1: no daemons active 2024-05-11T13:17:23.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:23 smithi063 ceph-mon[29443]: from='client.? 172.21.15.63:0/4141609241' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-11T13:17:24.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout cluster: 2024-05-11T13:17:24.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout id: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:24.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-11T13:17:24.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:24.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout services: 2024-05-11T13:17:24.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.757054s) 2024-05-11T13:17:24.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-11T13:17:24.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-11T13:17:24.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:24.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout data: 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pgs: 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.285+0000 7fb8d3a00700 1 Processor -- start 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.285+0000 7fb8d3a00700 1 -- start start 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.285+0000 7fb8d3a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc1091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.285+0000 7fb8d3a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc109700 con 0x7fb8cc108da0 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.287+0000 7fb8d1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc1091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.287+0000 7fb8d1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc1091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37948/0 (socket says 172.21.15.63:37948) 2024-05-11T13:17:24.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.287+0000 7fb8d1000700 1 -- 172.21.15.63:0/541988600 learned_addr learned my addr 172.21.15.63:0/541988600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:24.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.287+0000 7fb8d1000700 1 -- 172.21.15.63:0/541988600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc109880 con 0x7fb8cc108da0 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.287+0000 7fb8d1000700 1 --2- 172.21.15.63:0/541988600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc1091c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00098a0 tx=0x7fb8c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f05033908557cb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.288+0000 7fb8cb400700 1 -- 172.21.15.63:0/541988600 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c0004160 con 0x7fb8cc108da0 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.288+0000 7fb8cb400700 1 -- 172.21.15.63:0/541988600 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c0004300 con 0x7fb8cc108da0 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.288+0000 7fb8cb400700 1 -- 172.21.15.63:0/541988600 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c00036a0 con 0x7fb8cc108da0 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.288+0000 7fb8d3a00700 1 -- 172.21.15.63:0/541988600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 msgr2=0x7fb8cc1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.288+0000 7fb8d3a00700 1 --2- 172.21.15.63:0/541988600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc1091c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00098a0 tx=0x7fb8c0009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.289+0000 7fb8d3a00700 1 -- 172.21.15.63:0/541988600 shutdown_connections 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.289+0000 7fb8d3a00700 1 --2- 172.21.15.63:0/541988600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc1091c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.289+0000 7fb8d3a00700 1 -- 172.21.15.63:0/541988600 >> 172.21.15.63:0/541988600 conn(0x7fb8cc104400 msgr2=0x7fb8cc106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.289+0000 7fb8d3a00700 1 -- 172.21.15.63:0/541988600 shutdown_connections 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.289+0000 7fb8d3a00700 1 -- 172.21.15.63:0/541988600 wait complete. 2024-05-11T13:17:24.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.290+0000 7fb8d3a00700 1 Processor -- start 2024-05-11T13:17:24.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.291+0000 7fb8d3a00700 1 -- start start 2024-05-11T13:17:24.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.291+0000 7fb8d3a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc193750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:24.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.291+0000 7fb8d3a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10a200 con 0x7fb8cc108da0 2024-05-11T13:17:24.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.291+0000 7fb8d1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc193750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:24.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.291+0000 7fb8d1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc193750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37954/0 (socket says 172.21.15.63:37954) 2024-05-11T13:17:24.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.291+0000 7fb8d1000700 1 -- 172.21.15.63:0/4141609241 learned_addr learned my addr 172.21.15.63:0/4141609241 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:24.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.292+0000 7fb8d1000700 1 -- 172.21.15.63:0/4141609241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc193c90 con 0x7fb8cc108da0 2024-05-11T13:17:24.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.292+0000 7fb8d1000700 1 --2- 172.21.15.63:0/4141609241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc193750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0000c00 tx=0x7fb8c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:24.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.292+0000 7fb8c9600700 1 -- 172.21.15.63:0/4141609241 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c0003db0 con 0x7fb8cc108da0 2024-05-11T13:17:24.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.293+0000 7fb8c9600700 1 -- 172.21.15.63:0/4141609241 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c0011630 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.293+0000 7fb8c9600700 1 -- 172.21.15.63:0/4141609241 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c0011950 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.293+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc193f20 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.293+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1904e0 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.293+0000 7fb8c9600700 1 -- 172.21.15.63:0/4141609241 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb8c001a460 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.293+0000 7fb8c9600700 1 -- 172.21.15.63:0/4141609241 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb8c001b7d0 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.294+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc1940f0 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.297+0000 7fb8c9600700 1 -- 172.21.15.63:0/4141609241 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb8cc1940f0 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.378+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb8cc190ae0 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.378+0000 7fb8c9600700 1 -- 172.21.15.63:0/4141609241 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb8c001b9d0 con 0x7fb8cc108da0 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.379+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 msgr2=0x7fb8cc193750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.379+0000 7fb8d3a00700 1 --2- 172.21.15.63:0/4141609241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc193750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0000c00 tx=0x7fb8c0004750 comp rx=0 tx=0).stop 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.380+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 shutdown_connections 2024-05-11T13:17:24.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.380+0000 7fb8d3a00700 1 --2- 172.21.15.63:0/4141609241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8cc108da0 0x7fb8cc193750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:24.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.380+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 >> 172.21.15.63:0/4141609241 conn(0x7fb8cc104400 msgr2=0x7fb8cc07e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:24.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.380+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 shutdown_connections 2024-05-11T13:17:24.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:23.380+0000 7fb8d3a00700 1 -- 172.21.15.63:0/4141609241 wait complete. 2024-05-11T13:17:24.103 INFO:teuthology.orchestra.run.smithi063.stdout:mon is available 2024-05-11T13:17:24.103 INFO:teuthology.orchestra.run.smithi063.stdout:Assimilating anything we can from ceph.conf... 2024-05-11T13:17:25.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:24 smithi063 ceph-mon[29443]: from='client.? 172.21.15.63:0/4181831153' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-11T13:17:25.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:24 smithi063 ceph-mon[29443]: from='client.? 172.21.15.63:0/4181831153' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-11T13:17:25.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:25.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-05-11T13:17:25.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:25.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-11T13:17:25.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-05-11T13:17:25.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-11T13:17:25.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-11T13:17:25.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-11T13:17:25.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-11T13:17:25.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:25.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-11T13:17:25.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-11T13:17:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.705+0000 7f3d4de00700 1 Processor -- start 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.705+0000 7f3d4de00700 1 -- start start 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.705+0000 7f3d4de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4810b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.705+0000 7f3d4de00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4810b7c0 con 0x7f3d4810add0 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.706+0000 7f3d4ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4810b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.706+0000 7f3d4ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4810b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37966/0 (socket says 172.21.15.63:37966) 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.706+0000 7f3d4ca00700 1 -- 172.21.15.63:0/14162839 learned_addr learned my addr 172.21.15.63:0/14162839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.706+0000 7f3d4ca00700 1 -- 172.21.15.63:0/14162839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4810bf30 con 0x7f3d4810add0 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.707+0000 7f3d4ca00700 1 --2- 172.21.15.63:0/14162839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4810b1f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3d380098a0 tx=0x7f3d38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a3d66c7017e1cc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.707+0000 7f3d47e00700 1 -- 172.21.15.63:0/14162839 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d38004160 con 0x7f3d4810add0 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.707+0000 7f3d47e00700 1 -- 172.21.15.63:0/14162839 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d38004300 con 0x7f3d4810add0 2024-05-11T13:17:25.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.707+0000 7f3d47e00700 1 -- 172.21.15.63:0/14162839 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d380036a0 con 0x7f3d4810add0 2024-05-11T13:17:25.458 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.708+0000 7f3d4de00700 1 -- 172.21.15.63:0/14162839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 msgr2=0x7f3d4810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:25.458 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.708+0000 7f3d4de00700 1 --2- 172.21.15.63:0/14162839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4810b1f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3d380098a0 tx=0x7f3d38009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:25.458 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.708+0000 7f3d4de00700 1 -- 172.21.15.63:0/14162839 shutdown_connections 2024-05-11T13:17:25.458 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.708+0000 7f3d4de00700 1 --2- 172.21.15.63:0/14162839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4810b1f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:25.458 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.708+0000 7f3d4de00700 1 -- 172.21.15.63:0/14162839 >> 172.21.15.63:0/14162839 conn(0x7f3d4807e2e0 msgr2=0x7f3d4807e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:25.458 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.708+0000 7f3d4de00700 1 -- 172.21.15.63:0/14162839 shutdown_connections 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.708+0000 7f3d4de00700 1 -- 172.21.15.63:0/14162839 wait complete. 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.709+0000 7f3d4de00700 1 Processor -- start 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.709+0000 7f3d4de00700 1 -- start start 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.710+0000 7f3d4de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.710+0000 7f3d4de00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4810c460 con 0x7f3d4810add0 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.710+0000 7f3d4ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.710+0000 7f3d4ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4819e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37982/0 (socket says 172.21.15.63:37982) 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.710+0000 7f3d4ca00700 1 -- 172.21.15.63:0/4181831153 learned_addr learned my addr 172.21.15.63:0/4181831153 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.711+0000 7f3d4ca00700 1 -- 172.21.15.63:0/4181831153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4819e970 con 0x7f3d4810add0 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.711+0000 7f3d4ca00700 1 --2- 172.21.15.63:0/4181831153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4819e430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3d38000c00 tx=0x7f3d38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.711+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d38003db0 con 0x7f3d4810add0 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.711+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d38011630 con 0x7f3d4810add0 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.711+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d38011950 con 0x7f3d4810add0 2024-05-11T13:17:25.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.711+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4819ec00 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.711+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a18f0 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.712+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d38011af0 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.712+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d3801c450 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.713+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4806a630 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.715+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3d38032020 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.794+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3d4802e770 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.802+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f3d38004260 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.803+0000 7f3d46000700 1 -- 172.21.15.63:0/4181831153 <== mon.0 v2:172.21.15.63:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3d3801c6a0 con 0x7f3d4810add0 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.804+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 msgr2=0x7f3d4819e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.804+0000 7f3d4de00700 1 --2- 172.21.15.63:0/4181831153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4819e430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3d38000c00 tx=0x7f3d38004750 comp rx=0 tx=0).stop 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.804+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 shutdown_connections 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.804+0000 7f3d4de00700 1 --2- 172.21.15.63:0/4181831153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d4810add0 0x7f3d4819e430 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.804+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 >> 172.21.15.63:0/4181831153 conn(0x7f3d4807e2e0 msgr2=0x7f3d48195130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.805+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 shutdown_connections 2024-05-11T13:17:25.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:24.805+0000 7f3d4de00700 1 -- 172.21.15.63:0/4181831153 wait complete. 2024-05-11T13:17:25.461 INFO:teuthology.orchestra.run.smithi063.stdout:Generating new minimal ceph.conf... 2024-05-11T13:17:26.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:26 smithi063 ceph-mon[29443]: from='client.? 172.21.15.63:0/3120125915' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:17:26.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.053+0000 7ff74b400700 1 Processor -- start 2024-05-11T13:17:26.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.054+0000 7ff74b400700 1 -- start start 2024-05-11T13:17:27.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.054+0000 7ff74b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74410c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:27.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.054+0000 7ff74b400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff74410cdb0 con 0x7ff74410c450 2024-05-11T13:17:27.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.054+0000 7ff748a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74410c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:27.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.055+0000 7ff748a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74410c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37988/0 (socket says 172.21.15.63:37988) 2024-05-11T13:17:27.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.055+0000 7ff748a00700 1 -- 172.21.15.63:0/3820498809 learned_addr learned my addr 172.21.15.63:0/3820498809 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:27.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.055+0000 7ff748a00700 1 -- 172.21.15.63:0/3820498809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74410ac70 con 0x7ff74410c450 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.056+0000 7ff748a00700 1 --2- 172.21.15.63:0/3820498809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74410c870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff7300098a0 tx=0x7ff730009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7d2bec3fa9782f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.056+0000 7ff743e00700 1 -- 172.21.15.63:0/3820498809 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff730004160 con 0x7ff74410c450 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.056+0000 7ff743e00700 1 -- 172.21.15.63:0/3820498809 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff730004300 con 0x7ff74410c450 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.056+0000 7ff743e00700 1 -- 172.21.15.63:0/3820498809 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7300119e0 con 0x7ff74410c450 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.057+0000 7ff74b400700 1 -- 172.21.15.63:0/3820498809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 msgr2=0x7ff74410c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.057+0000 7ff74b400700 1 --2- 172.21.15.63:0/3820498809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74410c870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff7300098a0 tx=0x7ff730009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.057+0000 7ff74b400700 1 -- 172.21.15.63:0/3820498809 shutdown_connections 2024-05-11T13:17:27.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.057+0000 7ff74b400700 1 --2- 172.21.15.63:0/3820498809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74410c870 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:27.005 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.057+0000 7ff74b400700 1 -- 172.21.15.63:0/3820498809 >> 172.21.15.63:0/3820498809 conn(0x7ff74407e6a0 msgr2=0x7ff74407ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:27.005 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.057+0000 7ff74b400700 1 -- 172.21.15.63:0/3820498809 shutdown_connections 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.057+0000 7ff74b400700 1 -- 172.21.15.63:0/3820498809 wait complete. 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.058+0000 7ff74b400700 1 Processor -- start 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.058+0000 7ff74b400700 1 -- start start 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.059+0000 7ff74b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.059+0000 7ff74b400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff74410cf30 con 0x7ff74410c450 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.059+0000 7ff748a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.059+0000 7ff748a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38002/0 (socket says 172.21.15.63:38002) 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.059+0000 7ff748a00700 1 -- 172.21.15.63:0/3120125915 learned_addr learned my addr 172.21.15.63:0/3120125915 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.059+0000 7ff748a00700 1 -- 172.21.15.63:0/3120125915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74419f280 con 0x7ff74410c450 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff748a00700 1 --2- 172.21.15.63:0/3120125915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74419ed40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7300119b0 tx=0x7ff730011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff742000700 1 -- 172.21.15.63:0/3120125915 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7300039f0 con 0x7ff74410c450 2024-05-11T13:17:27.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff742000700 1 -- 172.21.15.63:0/3120125915 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff730003b90 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff742000700 1 -- 172.21.15.63:0/3120125915 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73001a620 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74419f510 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441a2200 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff742000700 1 -- 172.21.15.63:0/3120125915 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff73001ac60 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.060+0000 7ff742000700 1 -- 172.21.15.63:0/3120125915 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff730019700 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.062+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff744064bb0 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.064+0000 7ff742000700 1 -- 172.21.15.63:0/3120125915 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff730024a50 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.143+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff7441a2830 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.144+0000 7ff742000700 1 -- 172.21.15.63:0/3120125915 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff730024430 con 0x7ff74410c450 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.145+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 msgr2=0x7ff74419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.145+0000 7ff74b400700 1 --2- 172.21.15.63:0/3120125915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74419ed40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7300119b0 tx=0x7ff730011a90 comp rx=0 tx=0).stop 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.146+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 shutdown_connections 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.146+0000 7ff74b400700 1 --2- 172.21.15.63:0/3120125915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff74410c450 0x7ff74419ed40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.146+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 >> 172.21.15.63:0/3120125915 conn(0x7ff74407e6a0 msgr2=0x7ff7441083d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:27.008 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.146+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 shutdown_connections 2024-05-11T13:17:27.008 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:26.146+0000 7ff74b400700 1 -- 172.21.15.63:0/3120125915 wait complete. 2024-05-11T13:17:27.008 INFO:teuthology.orchestra.run.smithi063.stdout:Restarting the monitor... 2024-05-11T13:17:27.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:27 smithi063 systemd[1]: Stopping Ceph mon.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:17:27.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:27 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a[29439]: 2024-05-11T13:17:27.522+0000 7fd8e4200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:17:27.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:27 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a[29439]: 2024-05-11T13:17:27.522+0000 7fd8e4200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-11T13:17:27.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:27 smithi063 podman[29686]: 2024-05-11 13:17:27.604953702 +0000 UTC m=+0.224969978 container died e0adb54f55e530899fd94a220a3d0f711b8d40fa63b19189ae4e925fa6e0efc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T13:17:28.223 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:28 smithi063 podman[29686]: 2024-05-11 13:17:28.047169807 +0000 UTC m=+0.667186108 container cleanup e0adb54f55e530899fd94a220a3d0f711b8d40fa63b19189ae4e925fa6e0efc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-11T13:17:28.223 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:28 smithi063 bash[29686]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a 2024-05-11T13:17:28.661 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:28 smithi063 podman[29701]: 2024-05-11 13:17:28.222811664 +0000 UTC m=+0.597171276 container remove e0adb54f55e530899fd94a220a3d0f711b8d40fa63b19189ae4e925fa6e0efc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T13:17:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:28 smithi063 systemd[1]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-11T13:17:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:28 smithi063 systemd[1]: Stopped Ceph mon.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:17:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:28 smithi063 systemd[1]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a.service: Consumed 1.953s CPU time. 2024-05-11T13:17:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:28 smithi063 systemd[1]: Starting Ceph mon.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:17:29.389 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 podman[29790]: 2024-05-11 13:17:28.997002614 +0000 UTC m=+0.035109311 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:17:29.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 podman[29790]: 2024-05-11 13:17:29.098678401 +0000 UTC m=+0.136785129 container create d455117ba008f0b2924b5cb0a0290d1fdd607316082ec64de728a46d41012be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-11T13:17:29.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 podman[29790]: 2024-05-11 13:17:29.326978491 +0000 UTC m=+0.365085226 container init d455117ba008f0b2924b5cb0a0290d1fdd607316082ec64de728a46d41012be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-11T13:17:29.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 podman[29790]: 2024-05-11 13:17:29.339436679 +0000 UTC m=+0.377543414 container start d455117ba008f0b2924b5cb0a0290d1fdd607316082ec64de728a46d41012be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2) 2024-05-11T13:17:29.412 INFO:teuthology.orchestra.run.smithi063.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-11T13:17:29.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: set uid:gid to 167:167 (ceph:ceph) 2024-05-11T13:17:29.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev), process ceph-mon, pid 2 2024-05-11T13:17:29.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: pidfile_write: ignore empty --pid-file 2024-05-11T13:17:29.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: load: jerasure load: lrc 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: RocksDB version: 7.9.2 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Git sha 0 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Compile date 2024-05-10 10:41:35 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: DB SUMMARY 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: DB Session ID: 9FZUR2K1JTM48GT27GJ6 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: CURRENT file: CURRENT 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: IDENTITY file: IDENTITY 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103990 ; 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.error_if_exists: 0 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.create_if_missing: 0 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.paranoid_checks: 1 2024-05-11T13:17:29.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.env: 0x55cd089e0cc0 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.fs: PosixFileSystem 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.info_log: 0x55cd09fafe00 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_file_opening_threads: 16 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.statistics: (nil) 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.use_fsync: 0 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_log_file_size: 0 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.keep_log_file_num: 1000 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.recycle_log_file_num: 0 2024-05-11T13:17:29.688 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.allow_fallocate: 1 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.allow_mmap_reads: 0 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.allow_mmap_writes: 0 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.use_direct_reads: 0 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.create_missing_column_families: 0 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.db_log_dir: 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.wal_dir: 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-11T13:17:29.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.advise_random_on_open: 1 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.db_write_buffer_size: 0 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.write_buffer_manager: 0x55cd09f810e0 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.rate_limiter: (nil) 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.wal_recovery_mode: 2 2024-05-11T13:17:29.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.enable_thread_tracking: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.enable_pipelined_write: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.unordered_write: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.row_cache: None 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.wal_filter: None 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.allow_ingest_behind: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.two_write_queues: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.manual_wal_flush: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.wal_compression: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.atomic_flush: 0 2024-05-11T13:17:29.691 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.log_readahead_size: 0 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.best_efforts_recovery: 0 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 bash[29790]: d455117ba008f0b2924b5cb0a0290d1fdd607316082ec64de728a46d41012be0 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.allow_data_in_errors: 0 2024-05-11T13:17:29.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.db_host_id: __hostname__ 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_background_jobs: 2 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_background_compactions: -1 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_subcompactions: 1 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_total_wal_size: 0 2024-05-11T13:17:29.693 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_open_files: -1 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bytes_per_sync: 0 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_readahead_size: 0 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_background_flushes: -1 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Compression algorithms supported: 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kZSTD supported: 0 2024-05-11T13:17:29.694 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kXpressCompression supported: 0 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kLZ4HCCompression supported: 1 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kZlibCompression supported: 1 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kSnappyCompression supported: 1 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kLZ4Compression supported: 1 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: kBZip2Compression supported: 0 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.merge_operator: 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_filter: None 2024-05-11T13:17:29.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_filter_factory: None 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.sst_partitioner_factory: None 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cd09faffc0) 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: index_type: 0 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: data_block_index_type: 0 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: index_shortening: 1 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: checksum: 4 2024-05-11T13:17:29.696 INFO:journalctl@ceph.mon.a.smithi063.stdout: no_block_cache: 0 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache: 0x55cd09fb31f0 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache_options: 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: capacity : 536870912 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: num_shard_bits : 4 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: strict_capacity_limit : 0 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache_compressed: (nil) 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: persistent_cache: (nil) 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_size: 4096 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_size_deviation: 10 2024-05-11T13:17:29.697 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_restart_interval: 16 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: index_block_restart_interval: 1 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: metadata_block_size: 4096 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: partition_filters: 0 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: use_delta_encoding: 1 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: filter_policy: bloomfilter 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: whole_key_filtering: 1 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: verify_compression: 0 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: format_version: 5 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: enable_index_compression: 1 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_align: 0 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: max_auto_readahead_size: 262144 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: prepopulate_block_cache: 0 2024-05-11T13:17:29.698 INFO:journalctl@ceph.mon.a.smithi063.stdout: initial_auto_readahead_size: 8192 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout: num_file_reads_for_auto_readahead: 2 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.write_buffer_size: 33554432 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_write_buffer_number: 2 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression: NoCompression 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression: Disabled 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.prefix_extractor: nullptr 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.num_levels: 7 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-11T13:17:29.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-11T13:17:29.700 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.level: 32767 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.strategy: 0 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.enabled: false 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-11T13:17:29.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.target_file_size_base: 67108864 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-11T13:17:29.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.arena_block_size: 1048576 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.disable_auto_compactions: 0 2024-05-11T13:17:29.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.inplace_update_support: 0 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-11T13:17:29.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.bloom_locality: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.max_successive_merges: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.paranoid_file_checks: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.force_consistency_checks: 1 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.report_bg_io_stats: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.ttl: 2592000 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.enable_blob_files: false 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.min_blob_size: 0 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.blob_file_size: 268435456 2024-05-11T13:17:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.blob_file_starting_level: 0 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 430e56c4-7368-4556-a317-5aae5a35073d 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433449400423, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-11T13:17:29.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433449402926, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99225, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97171, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13895, "raw_average_key_size": 49, "raw_value_size": 89580, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715433449, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "430e56c4-7368-4556-a317-5aae5a35073d", "db_session_id": "9FZUR2K1JTM48GT27GJ6", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433449403114, "job": 1, "event": "recovery_finished"} 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cd0a0d0000 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: DB pointer 0x55cd0a0bc000 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** DB Stats ** 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T13:17:29.707 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: L0 2/0 98.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: Sum 2/0 98.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 64.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:17:29.708 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative compaction: 0.00 GB write, 12.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval compaction: 0.00 GB write, 12.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Block cache BinnedLRUCache@0x55cd09fb31f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-11T13:17:29.709 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 systemd[1]: Started Ceph mon.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: starting mon.a rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???) e1 preinit fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???).mds e1 new map 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???).mds e1 print_map 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout: e1 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout: legacy client fscid: -1 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout: No filesystems configured 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:17:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: monmap epoch 1 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: last_changed 2024-05-11T13:17:18.239963+0000 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: min_mon_release 19 (squid) 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: election_strategy: 1 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: fsmap 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: osdmap e1: 0 total, 0 up, 0 in 2024-05-11T13:17:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:29 smithi063 ceph-mon[29809]: mgrmap e1: no daemons active 2024-05-11T13:17:30.702 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.005+0000 7f833da00700 1 Processor -- start 2024-05-11T13:17:30.702 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.005+0000 7f833da00700 1 -- start start 2024-05-11T13:17:30.702 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.006+0000 7f833da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f8338109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:30.702 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.006+0000 7f833da00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83381096c0 con 0x7f8338108d60 2024-05-11T13:17:30.702 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.007+0000 7f8336a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f8338109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:30.702 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.007+0000 7f8336a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f8338109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54786/0 (socket says 172.21.15.63:54786) 2024-05-11T13:17:30.702 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.007+0000 7f8336a00700 1 -- 172.21.15.63:0/2614881334 learned_addr learned my addr 172.21.15.63:0/2614881334 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:30.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.007+0000 7f8336a00700 1 -- 172.21.15.63:0/2614881334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8338109840 con 0x7f8338108d60 2024-05-11T13:17:30.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.008+0000 7f8336a00700 1 --2- 172.21.15.63:0/2614881334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f8338109180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83280098a0 tx=0x7f8328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf55c14170e3d88 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:30.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.008+0000 7f8336000700 1 -- 172.21.15.63:0/2614881334 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8328004160 con 0x7f8338108d60 2024-05-11T13:17:30.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.008+0000 7f8336000700 1 -- 172.21.15.63:0/2614881334 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8328004300 con 0x7f8338108d60 2024-05-11T13:17:30.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.008+0000 7f8336000700 1 -- 172.21.15.63:0/2614881334 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83280119e0 con 0x7f8338108d60 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.009+0000 7f833da00700 1 -- 172.21.15.63:0/2614881334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 msgr2=0x7f8338109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.009+0000 7f833da00700 1 --2- 172.21.15.63:0/2614881334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f8338109180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83280098a0 tx=0x7f8328009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.009+0000 7f833da00700 1 -- 172.21.15.63:0/2614881334 shutdown_connections 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.009+0000 7f833da00700 1 --2- 172.21.15.63:0/2614881334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f8338109180 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.009+0000 7f833da00700 1 -- 172.21.15.63:0/2614881334 >> 172.21.15.63:0/2614881334 conn(0x7f8338104400 msgr2=0x7f8338106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.009+0000 7f833da00700 1 -- 172.21.15.63:0/2614881334 shutdown_connections 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.009+0000 7f833da00700 1 -- 172.21.15.63:0/2614881334 wait complete. 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.011+0000 7f833da00700 1 Processor -- start 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.011+0000 7f833da00700 1 -- start start 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.011+0000 7f833da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f833819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.011+0000 7f833da00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f833810ad00 con 0x7f8338108d60 2024-05-11T13:17:30.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.012+0000 7f8336a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f833819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.012+0000 7f8336a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f833819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54802/0 (socket says 172.21.15.63:54802) 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.012+0000 7f8336a00700 1 -- 172.21.15.63:0/1884472839 learned_addr learned my addr 172.21.15.63:0/1884472839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.012+0000 7f8336a00700 1 -- 172.21.15.63:0/1884472839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833819d090 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.012+0000 7f8336a00700 1 --2- 172.21.15.63:0/1884472839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f833819cb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8328009130 tx=0x7f8328011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.013+0000 7f832f400700 1 -- 172.21.15.63:0/1884472839 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8328003b20 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.013+0000 7f832f400700 1 -- 172.21.15.63:0/1884472839 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f832801a8a0 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.013+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833819d320 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.013+0000 7f832f400700 1 -- 172.21.15.63:0/1884472839 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8328024810 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.013+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a0010 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.013+0000 7f832f400700 1 -- 172.21.15.63:0/1884472839 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8328022020 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.013+0000 7f832f400700 1 -- 172.21.15.63:0/1884472839 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f83280193c0 con 0x7f8338108d60 2024-05-11T13:17:30.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.014+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8338064bb0 con 0x7f8338108d60 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.017+0000 7f832f400700 1 -- 172.21.15.63:0/1884472839 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f832802dde0 con 0x7f8338108d60 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.095+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f83381a01e0 con 0x7f8338108d60 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.099+0000 7f832f400700 1 -- 172.21.15.63:0/1884472839 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f832801ad70 con 0x7f8338108d60 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.100+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 msgr2=0x7f833819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.100+0000 7f833da00700 1 --2- 172.21.15.63:0/1884472839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f833819cb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8328009130 tx=0x7f8328011a60 comp rx=0 tx=0).stop 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.100+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 shutdown_connections 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.100+0000 7f833da00700 1 --2- 172.21.15.63:0/1884472839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338108d60 0x7f833819cb50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.100+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 >> 172.21.15.63:0/1884472839 conn(0x7f8338104400 msgr2=0x7f8338104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.101+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 shutdown_connections 2024-05-11T13:17:30.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:30.101+0000 7f833da00700 1 -- 172.21.15.63:0/1884472839 wait complete. 2024-05-11T13:17:30.710 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-11T13:17:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-11T13:17:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mgr... 2024-05-11T13:17:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-11T13:17:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-11T13:17:31.198 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.x 2024-05-11T13:17:31.198 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-11T13:17:31.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:31 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1884472839' entity='client.admin' 2024-05-11T13:17:31.768 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968.target.wants/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@.service. 2024-05-11T13:17:32.181 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:31 smithi063 systemd[1]: Starting Ceph mgr.x for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:17:32.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:32 smithi063 podman[30023]: 2024-05-11 13:17:32.085073765 +0000 UTC m=+0.036234363 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T13:17:36.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:36 smithi063 podman[30023]: 2024-05-11 13:17:36.178219278 +0000 UTC m=+4.129379917 container create a5aa873b8ad4258bf72651b5ea8ae131be23601e25ed6a5092f547c52fa480b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-11T13:17:40.148 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T13:17:40.148 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-11T13:17:40.165 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T13:17:40.165 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-11T13:17:40.165 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-11T13:17:40.165 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-05-11T13:17:40.185 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T13:17:40.186 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-11T13:17:40.202 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T13:17:40.202 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-11T13:17:40.202 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-11T13:17:40.202 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-11T13:17:40.202 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr to start... 2024-05-11T13:17:40.202 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr... 2024-05-11T13:17:40.288 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:40 smithi063 podman[30023]: 2024-05-11 13:17:40.023873489 +0000 UTC m=+7.975034128 container init a5aa873b8ad4258bf72651b5ea8ae131be23601e25ed6a5092f547c52fa480b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-11T13:17:40.288 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:40 smithi063 podman[30023]: 2024-05-11 13:17:40.034414009 +0000 UTC m=+7.985574615 container start a5aa873b8ad4258bf72651b5ea8ae131be23601e25ed6a5092f547c52fa480b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:17:40.288 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:40 smithi063 bash[30023]: a5aa873b8ad4258bf72651b5ea8ae131be23601e25ed6a5092f547c52fa480b2 2024-05-11T13:17:40.288 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:40 smithi063 systemd[1]: Started Ceph mgr.x for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:17:40.552 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:40.286+0000 7f8676669200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:17:40.811 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:40.562+0000 7f8676669200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:17:40.811 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:40.701+0000 7f8676669200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:17:41.142 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:40 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3172718722' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T13:17:41.401 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:41.142+0000 7f8676669200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:17:41.402 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:41.266+0000 7f8676669200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:17:41.697 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:41.698 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:17:41.698 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:17:41.698 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T13:17:41.698 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T13:17:41.698 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T13:17:41.698 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T13:17:41.698 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:41.699 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T13:17:41.700 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T13:17:41.700 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T13:17:41.700 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T13:17:41.700 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T13:17:41.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T13:17:41.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:41.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T13:17:41.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T13:17:41.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:41.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:41.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T13:17:22.624339+0000", 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.834+0000 7f456b600700 1 Processor -- start 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.834+0000 7f456b600700 1 -- start start 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.834+0000 7f456b600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f45641079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.834+0000 7f456b600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f456407ce90 con 0x7f45641075a0 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.835+0000 7f4568c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f45641079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:41.710 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.835+0000 7f4568c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f45641079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47358/0 (socket says 172.21.15.63:47358) 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.835+0000 7f4568c00700 1 -- 172.21.15.63:0/4049390250 learned_addr learned my addr 172.21.15.63:0/4049390250 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.835+0000 7f4568c00700 1 -- 172.21.15.63:0/4049390250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4564107f00 con 0x7f45641075a0 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.835+0000 7f4568c00700 1 --2- 172.21.15.63:0/4049390250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f45641079c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f45580098a0 tx=0x7f4558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3885d5ead351e29 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.836+0000 7f4563400700 1 -- 172.21.15.63:0/4049390250 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558004160 con 0x7f45641075a0 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.836+0000 7f4563400700 1 -- 172.21.15.63:0/4049390250 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4558004300 con 0x7f45641075a0 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.836+0000 7f4563400700 1 -- 172.21.15.63:0/4049390250 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45580119e0 con 0x7f45641075a0 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.836+0000 7f456b600700 1 -- 172.21.15.63:0/4049390250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 msgr2=0x7f45641079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.836+0000 7f456b600700 1 --2- 172.21.15.63:0/4049390250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f45641079c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f45580098a0 tx=0x7f4558009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.837+0000 7f456b600700 1 -- 172.21.15.63:0/4049390250 shutdown_connections 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.837+0000 7f456b600700 1 --2- 172.21.15.63:0/4049390250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f45641079c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.837+0000 7f456b600700 1 -- 172.21.15.63:0/4049390250 >> 172.21.15.63:0/4049390250 conn(0x7f4564103200 msgr2=0x7f4564105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.837+0000 7f456b600700 1 -- 172.21.15.63:0/4049390250 shutdown_connections 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.837+0000 7f456b600700 1 -- 172.21.15.63:0/4049390250 wait complete. 2024-05-11T13:17:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.838+0000 7f456b600700 1 Processor -- start 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.838+0000 7f456b600700 1 -- start start 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.838+0000 7f456b600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f456419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.838+0000 7f456b600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4564108ed0 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.839+0000 7f4568c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f456419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.839+0000 7f4568c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f456419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47364/0 (socket says 172.21.15.63:47364) 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.839+0000 7f4568c00700 1 -- 172.21.15.63:0/3172718722 learned_addr learned my addr 172.21.15.63:0/3172718722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.839+0000 7f4568c00700 1 -- 172.21.15.63:0/3172718722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456419f2e0 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.839+0000 7f4568c00700 1 --2- 172.21.15.63:0/3172718722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f456419eda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4558009130 tx=0x7f4558011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.840+0000 7f4561600700 1 -- 172.21.15.63:0/3172718722 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558003b20 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.840+0000 7f4561600700 1 -- 172.21.15.63:0/3172718722 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f455801a8a0 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.840+0000 7f4561600700 1 -- 172.21.15.63:0/3172718722 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558024810 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.840+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456419f570 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.840+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45641a2220 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.841+0000 7f4561600700 1 -- 172.21.15.63:0/3172718722 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4558022020 con 0x7f45641075a0 2024-05-11T13:17:41.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.841+0000 7f4561600700 1 -- 172.21.15.63:0/3172718722 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f455802ddf0 con 0x7f45641075a0 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.841+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4564064bb0 con 0x7f45641075a0 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.846+0000 7f4561600700 1 -- 172.21.15.63:0/3172718722 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4558043ec0 con 0x7f45641075a0 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.927+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f45640008d0 con 0x7f45641075a0 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.928+0000 7f4561600700 1 -- 172.21.15.63:0/3172718722 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f4558032020 con 0x7f45641075a0 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.929+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 msgr2=0x7f456419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.929+0000 7f456b600700 1 --2- 172.21.15.63:0/3172718722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f456419eda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4558009130 tx=0x7f4558011a60 comp rx=0 tx=0).stop 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.929+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 shutdown_connections 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.929+0000 7f456b600700 1 --2- 172.21.15.63:0/3172718722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45641075a0 0x7f456419eda0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.929+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 >> 172.21.15.63:0/3172718722 conn(0x7f4564103200 msgr2=0x7f45641959c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.929+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 shutdown_connections 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:40.929+0000 7f456b600700 1 -- 172.21.15.63:0/3172718722 wait complete. 2024-05-11T13:17:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (1/15)... 2024-05-11T13:17:43.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:42.928+0000 7f8676669200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:17:43.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:43.163+0000 7f8676669200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:17:43.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:43.383+0000 7f8676669200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:17:44.076 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:43.786+0000 7f8676669200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:17:44.077 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:43.911+0000 7f8676669200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:17:44.350 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:44 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:44.172+0000 7f8676669200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:17:44.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:44 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:44.437+0000 7f8676669200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:17:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:44 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/979245840' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T13:17:45.128 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:45.129 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T13:17:45.130 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T13:17:45.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T13:17:45.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:45.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T13:17:22.624339+0000", 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.348+0000 7fc22e200700 1 Processor -- start 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.348+0000 7fc22e200700 1 -- start start 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.349+0000 7fc22e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22807d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.349+0000 7fc22e200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc22807f100 con 0x7fc22807ece0 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.349+0000 7fc21f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22807d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.349+0000 7fc21f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22807d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47380/0 (socket says 172.21.15.63:47380) 2024-05-11T13:17:45.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.349+0000 7fc21f400700 1 -- 172.21.15.63:0/2411002618 learned_addr learned my addr 172.21.15.63:0/2411002618 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.350+0000 7fc21f400700 1 -- 172.21.15.63:0/2411002618 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22807d7e0 con 0x7fc22807ece0 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.350+0000 7fc21f400700 1 --2- 172.21.15.63:0/2411002618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22807d2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098a0 tx=0x7fc218009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b309328775a09d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.351+0000 7fc226a00700 1 -- 172.21.15.63:0/2411002618 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc218004160 con 0x7fc22807ece0 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.351+0000 7fc226a00700 1 -- 172.21.15.63:0/2411002618 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc218004300 con 0x7fc22807ece0 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.351+0000 7fc226a00700 1 -- 172.21.15.63:0/2411002618 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2180119e0 con 0x7fc22807ece0 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.351+0000 7fc22e200700 1 -- 172.21.15.63:0/2411002618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 msgr2=0x7fc22807d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.351+0000 7fc22e200700 1 --2- 172.21.15.63:0/2411002618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22807d2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098a0 tx=0x7fc218009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.352+0000 7fc22e200700 1 -- 172.21.15.63:0/2411002618 shutdown_connections 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.352+0000 7fc22e200700 1 --2- 172.21.15.63:0/2411002618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22807d2a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.352+0000 7fc22e200700 1 -- 172.21.15.63:0/2411002618 >> 172.21.15.63:0/2411002618 conn(0x7fc228104400 msgr2=0x7fc228106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.352+0000 7fc22e200700 1 -- 172.21.15.63:0/2411002618 shutdown_connections 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.352+0000 7fc22e200700 1 -- 172.21.15.63:0/2411002618 wait complete. 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.353+0000 7fc22e200700 1 Processor -- start 2024-05-11T13:17:45.142 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.353+0000 7fc22e200700 1 -- start start 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.353+0000 7fc22e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.353+0000 7fc22e200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc22810f190 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.354+0000 7fc21f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.354+0000 7fc21f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47382/0 (socket says 172.21.15.63:47382) 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.354+0000 7fc21f400700 1 -- 172.21.15.63:0/979245840 learned_addr learned my addr 172.21.15.63:0/979245840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.354+0000 7fc21f400700 1 -- 172.21.15.63:0/979245840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22819f310 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.355+0000 7fc21f400700 1 --2- 172.21.15.63:0/979245840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22819edd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2180116a0 tx=0x7fc2180046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.355+0000 7fc224c00700 1 -- 172.21.15.63:0/979245840 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc218003b70 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.355+0000 7fc224c00700 1 -- 172.21.15.63:0/979245840 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc21801a8a0 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.355+0000 7fc224c00700 1 -- 172.21.15.63:0/979245840 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc218024890 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.355+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22819f5a0 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.355+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281a2290 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.356+0000 7fc224c00700 1 -- 172.21.15.63:0/979245840 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc218022020 con 0x7fc22807ece0 2024-05-11T13:17:45.143 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.356+0000 7fc224c00700 1 -- 172.21.15.63:0/979245840 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2180193c0 con 0x7fc22807ece0 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.356+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228064bb0 con 0x7fc22807ece0 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.361+0000 7fc224c00700 1 -- 172.21.15.63:0/979245840 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc21801aa50 con 0x7fc22807ece0 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.442+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc2281a27d0 con 0x7fc22807ece0 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.442+0000 7fc224c00700 1 -- 172.21.15.63:0/979245840 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fc21801a430 con 0x7fc22807ece0 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.443+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 msgr2=0x7fc22819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.443+0000 7fc22e200700 1 --2- 172.21.15.63:0/979245840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22819edd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2180116a0 tx=0x7fc2180046a0 comp rx=0 tx=0).stop 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.444+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 shutdown_connections 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.444+0000 7fc22e200700 1 --2- 172.21.15.63:0/979245840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22807ece0 0x7fc22819edd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.444+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 >> 172.21.15.63:0/979245840 conn(0x7fc228104400 msgr2=0x7fc228104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.444+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 shutdown_connections 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:44.444+0000 7fc22e200700 1 -- 172.21.15.63:0/979245840 wait complete. 2024-05-11T13:17:45.144 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (2/15)... 2024-05-11T13:17:45.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:45.270+0000 7f8676669200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:17:45.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:45.566+0000 7f8676669200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:17:46.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:46.134+0000 7f8676669200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:17:46.435 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:46.264+0000 7f8676669200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:17:47.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:46.738+0000 7f8676669200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:17:47.480 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:47.211+0000 7f8676669200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:17:47.480 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:47.339+0000 7f8676669200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:17:47.772 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:47.676+0000 7f8676669200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:17:48.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:47 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2742187065' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T13:17:48.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:47.865+0000 7f8676669200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:17:48.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:47.996+0000 7f8676669200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:17:48.514 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:48.410+0000 7f8676669200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:17:48.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:48.534 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:17:48.534 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:17:48.534 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T13:17:48.534 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T13:17:48.534 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:48.535 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T13:17:48.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:48.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:48.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T13:17:22.624339+0000", 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.770+0000 7f7a32e00700 1 Processor -- start 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.770+0000 7f7a32e00700 1 -- start start 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.771+0000 7f7a32e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.771+0000 7f7a32e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c07d840 con 0x7f7a2c07ee00 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.771+0000 7f7a2be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.771+0000 7f7a2be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47386/0 (socket says 172.21.15.63:47386) 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.771+0000 7f7a2be00700 1 -- 172.21.15.63:0/2939791054 learned_addr learned my addr 172.21.15.63:0/2939791054 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.772+0000 7f7a2be00700 1 -- 172.21.15.63:0/2939791054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c07d9c0 con 0x7f7a2c07ee00 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.772+0000 7f7a2be00700 1 --2- 172.21.15.63:0/2939791054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c07d300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0098a0 tx=0x7f7a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9a0bc6789056a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:48.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.772+0000 7f7a2aa00700 1 -- 172.21.15.63:0/2939791054 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c004160 con 0x7f7a2c07ee00 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.772+0000 7f7a2aa00700 1 -- 172.21.15.63:0/2939791054 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7a1c004300 con 0x7f7a2c07ee00 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.772+0000 7f7a2aa00700 1 -- 172.21.15.63:0/2939791054 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c0119e0 con 0x7f7a2c07ee00 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.773+0000 7f7a32e00700 1 -- 172.21.15.63:0/2939791054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 msgr2=0x7f7a2c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.773+0000 7f7a32e00700 1 --2- 172.21.15.63:0/2939791054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c07d300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0098a0 tx=0x7f7a1c009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.773+0000 7f7a32e00700 1 -- 172.21.15.63:0/2939791054 shutdown_connections 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.773+0000 7f7a32e00700 1 --2- 172.21.15.63:0/2939791054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c07d300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.773+0000 7f7a32e00700 1 -- 172.21.15.63:0/2939791054 >> 172.21.15.63:0/2939791054 conn(0x7f7a2c1043c0 msgr2=0x7f7a2c1067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.773+0000 7f7a32e00700 1 -- 172.21.15.63:0/2939791054 shutdown_connections 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.773+0000 7f7a32e00700 1 -- 172.21.15.63:0/2939791054 wait complete. 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.775+0000 7f7a32e00700 1 Processor -- start 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.775+0000 7f7a32e00700 1 -- start start 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.775+0000 7f7a32e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c1a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:48.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.775+0000 7f7a32e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c10ae60 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.775+0000 7f7a2be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c1a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.776+0000 7f7a2be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c1a3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47392/0 (socket says 172.21.15.63:47392) 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.776+0000 7f7a2be00700 1 -- 172.21.15.63:0/2742187065 learned_addr learned my addr 172.21.15.63:0/2742187065 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.776+0000 7f7a2be00700 1 -- 172.21.15.63:0/2742187065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1a36b0 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.776+0000 7f7a2be00700 1 --2- 172.21.15.63:0/2742187065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c1a3170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009130 tx=0x7f7a1c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.777+0000 7f7a28c00700 1 -- 172.21.15.63:0/2742187065 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c003b20 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.777+0000 7f7a28c00700 1 -- 172.21.15.63:0/2742187065 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7a1c01a8a0 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.777+0000 7f7a28c00700 1 -- 172.21.15.63:0/2742187065 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c024810 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.777+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1a3940 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.777+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1a6630 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.777+0000 7f7a28c00700 1 -- 172.21.15.63:0/2742187065 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a1c022020 con 0x7f7a2c07ee00 2024-05-11T13:17:48.541 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.777+0000 7f7a28c00700 1 -- 172.21.15.63:0/2742187065 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a1c02dd90 con 0x7f7a2c07ee00 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.778+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c064bb0 con 0x7f7a2c07ee00 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.781+0000 7f7a28c00700 1 -- 172.21.15.63:0/2742187065 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7a1c043ec0 con 0x7f7a2c07ee00 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.862+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7a2c0008d0 con 0x7f7a2c07ee00 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.863+0000 7f7a28c00700 1 -- 172.21.15.63:0/2742187065 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f7a1c032020 con 0x7f7a2c07ee00 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.864+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 msgr2=0x7f7a2c1a3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.864+0000 7f7a32e00700 1 --2- 172.21.15.63:0/2742187065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c1a3170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009130 tx=0x7f7a1c011a60 comp rx=0 tx=0).stop 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.864+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 shutdown_connections 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.864+0000 7f7a32e00700 1 --2- 172.21.15.63:0/2742187065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c07ee00 0x7f7a2c1a3170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.864+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 >> 172.21.15.63:0/2742187065 conn(0x7f7a2c1043c0 msgr2=0x7f7a2c19a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.865+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 shutdown_connections 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:47.865+0000 7f7a32e00700 1 -- 172.21.15.63:0/2742187065 wait complete. 2024-05-11T13:17:48.542 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (3/15)... 2024-05-11T13:17:48.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:48.543+0000 7f8676669200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:17:49.627 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:49.331+0000 7f8676669200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:17:49.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:49.624+0000 7f8676669200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: Activating manager daemon x 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: mgrmap e2: x(active, starting, since 0.0124972s) 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: Manager daemon x is now available 2024-05-11T13:17:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T13:17:49.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:49 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T13:17:50.993 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:50 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' 2024-05-11T13:17:50.993 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:50 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' 2024-05-11T13:17:50.993 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:50 smithi063 ceph-mon[29809]: from='mgr.14100 172.21.15.63:0/15379383' entity='mgr.x' 2024-05-11T13:17:50.993 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:50 smithi063 ceph-mon[29809]: mgrmap e3: x(active, since 1.01952s) 2024-05-11T13:17:51.857 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:51 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1856464240' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T13:17:52.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:52.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:17:52.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:17:52.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T13:17:52.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T13:17:52.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 22, 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T13:17:52.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T13:17:52.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:52.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T13:17:52.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:52.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T13:17:52.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T13:17:52.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T13:17:52.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T13:17:52.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T13:17:22.624339+0000", 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.153+0000 7f8ac7e00700 1 Processor -- start 2024-05-11T13:17:52.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.153+0000 7f8ac7e00700 1 -- start start 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.154+0000 7f8ac7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac810b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.154+0000 7f8ac7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810b7e0 con 0x7f8ac810adf0 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.154+0000 7f8ac6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac810b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.154+0000 7f8ac6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac810b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54574/0 (socket says 172.21.15.63:54574) 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.154+0000 7f8ac6a00700 1 -- 172.21.15.63:0/3838527938 learned_addr learned my addr 172.21.15.63:0/3838527938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.155+0000 7f8ac6a00700 1 -- 172.21.15.63:0/3838527938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac810bf50 con 0x7f8ac810adf0 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.155+0000 7f8ac6a00700 1 --2- 172.21.15.63:0/3838527938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac810b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80098a0 tx=0x7f8ab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8d606d4537ae1b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.155+0000 7f8ac6000700 1 -- 172.21.15.63:0/3838527938 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ab8004160 con 0x7f8ac810adf0 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.155+0000 7f8ac6000700 1 -- 172.21.15.63:0/3838527938 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8ab8004300 con 0x7f8ac810adf0 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.155+0000 7f8ac6000700 1 -- 172.21.15.63:0/3838527938 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ab80119e0 con 0x7f8ac810adf0 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.156+0000 7f8ac7e00700 1 -- 172.21.15.63:0/3838527938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 msgr2=0x7f8ac810b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.156+0000 7f8ac7e00700 1 --2- 172.21.15.63:0/3838527938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac810b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80098a0 tx=0x7f8ab8009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:52.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.156+0000 7f8ac7e00700 1 -- 172.21.15.63:0/3838527938 shutdown_connections 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.156+0000 7f8ac7e00700 1 --2- 172.21.15.63:0/3838527938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac810b210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.156+0000 7f8ac7e00700 1 -- 172.21.15.63:0/3838527938 >> 172.21.15.63:0/3838527938 conn(0x7f8ac807e240 msgr2=0x7f8ac807e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.156+0000 7f8ac7e00700 1 -- 172.21.15.63:0/3838527938 shutdown_connections 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.156+0000 7f8ac7e00700 1 -- 172.21.15.63:0/3838527938 wait complete. 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.157+0000 7f8ac7e00700 1 Processor -- start 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.158+0000 7f8ac7e00700 1 -- start start 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.158+0000 7f8ac7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.158+0000 7f8ac7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac810f180 con 0x7f8ac810adf0 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.158+0000 7f8ac6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.158+0000 7f8ac6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54582/0 (socket says 172.21.15.63:54582) 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.158+0000 7f8ac6a00700 1 -- 172.21.15.63:0/1856464240 learned_addr learned my addr 172.21.15.63:0/1856464240 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.159+0000 7f8ac6a00700 1 -- 172.21.15.63:0/1856464240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac819f0e0 con 0x7f8ac810adf0 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.159+0000 7f8ac6a00700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac819eba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80116a0 tx=0x7f8ab8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.159+0000 7f8abf400700 1 -- 172.21.15.63:0/1856464240 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ab80039f0 con 0x7f8ac810adf0 2024-05-11T13:17:52.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.159+0000 7f8abf400700 1 -- 172.21.15.63:0/1856464240 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8ab8003b90 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.159+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac819f370 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.159+0000 7f8abf400700 1 -- 172.21.15.63:0/1856464240 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ab801a600 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.159+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac81a2060 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.160+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac806a630 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.160+0000 7f8abf400700 1 -- 172.21.15.63:0/1856464240 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7f8ab8024a50 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.161+0000 7f8abf400700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f8aac03e310 0x7f8aac0407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.161+0000 7f8abf400700 1 -- 172.21.15.63:0/1856464240 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ab80538f0 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.161+0000 7f8abfe00700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f8aac03e310 0x7f8aac0407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.162+0000 7f8abfe00700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f8aac03e310 0x7f8aac0407d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.168+0000 7f8abf400700 1 -- 172.21.15.63:0/1856464240 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ab800fdd0 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.493+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8ac81a2230 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.494+0000 7f8abf400700 1 -- 172.21.15.63:0/1856464240 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8ac81a2230 con 0x7f8ac810adf0 2024-05-11T13:17:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f8aac03e310 msgr2=0x7f8aac0407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f8aac03e310 0x7f8aac0407d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).stop 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 msgr2=0x7f8ac819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac819eba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80116a0 tx=0x7f8ab8011a60 comp rx=0 tx=0).stop 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 shutdown_connections 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f8aac03e310 0x7f8aac0407d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 --2- 172.21.15.63:0/1856464240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ac810adf0 0x7f8ac819eba0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 >> 172.21.15.63:0/1856464240 conn(0x7f8ac807e240 msgr2=0x7f8ac8108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 shutdown_connections 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:51.497+0000 7f8ac7e00700 1 -- 172.21.15.63:0/1856464240 wait complete. 2024-05-11T13:17:52.170 INFO:teuthology.orchestra.run.smithi063.stdout:mgr is available 2024-05-11T13:17:52.750 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:52 smithi063 ceph-mon[29809]: mgrmap e4: x(active, since 2s) 2024-05-11T13:17:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-05-11T13:17:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:17:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-11T13:17:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-11T13:17:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.748+0000 7fda29c00700 1 Processor -- start 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.748+0000 7fda29c00700 1 -- start start 2024-05-11T13:17:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.749+0000 7fda29c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.749+0000 7fda29c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda241125e0 con 0x7fda2410fcb0 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.749+0000 7fda22a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.749+0000 7fda22a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54594/0 (socket says 172.21.15.63:54594) 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.749+0000 7fda22a00700 1 -- 172.21.15.63:0/3535080489 learned_addr learned my addr 172.21.15.63:0/3535080489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.750+0000 7fda22a00700 1 -- 172.21.15.63:0/3535080489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda24112760 con 0x7fda2410fcb0 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.750+0000 7fda22a00700 1 --2- 172.21.15.63:0/3535080489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241120a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda140098a0 tx=0x7fda14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895606f6b8707f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.751+0000 7fda21600700 1 -- 172.21.15.63:0/3535080489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda14004160 con 0x7fda2410fcb0 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.751+0000 7fda21600700 1 -- 172.21.15.63:0/3535080489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fda14004300 con 0x7fda2410fcb0 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.751+0000 7fda21600700 1 -- 172.21.15.63:0/3535080489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda140119e0 con 0x7fda2410fcb0 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.751+0000 7fda29c00700 1 -- 172.21.15.63:0/3535080489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 msgr2=0x7fda241120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.751+0000 7fda29c00700 1 --2- 172.21.15.63:0/3535080489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241120a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda140098a0 tx=0x7fda14009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.751+0000 7fda29c00700 1 -- 172.21.15.63:0/3535080489 shutdown_connections 2024-05-11T13:17:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.752+0000 7fda29c00700 1 --2- 172.21.15.63:0/3535080489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241120a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.752+0000 7fda29c00700 1 -- 172.21.15.63:0/3535080489 >> 172.21.15.63:0/3535080489 conn(0x7fda2407e420 msgr2=0x7fda24080840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.752+0000 7fda29c00700 1 -- 172.21.15.63:0/3535080489 shutdown_connections 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.752+0000 7fda29c00700 1 -- 172.21.15.63:0/3535080489 wait complete. 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.753+0000 7fda29c00700 1 Processor -- start 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.753+0000 7fda29c00700 1 -- start start 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.754+0000 7fda29c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.754+0000 7fda29c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda24113660 con 0x7fda2410fcb0 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.754+0000 7fda22a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.754+0000 7fda22a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54604/0 (socket says 172.21.15.63:54604) 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.754+0000 7fda22a00700 1 -- 172.21.15.63:0/1308686521 learned_addr learned my addr 172.21.15.63:0/1308686521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.754+0000 7fda22a00700 1 -- 172.21.15.63:0/1308686521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda241a3660 con 0x7fda2410fcb0 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.755+0000 7fda22a00700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241a3120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda14009130 tx=0x7fda14011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.755+0000 7fda1b400700 1 -- 172.21.15.63:0/1308686521 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda14003b20 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.755+0000 7fda1b400700 1 -- 172.21.15.63:0/1308686521 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fda1401a8a0 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.755+0000 7fda1b400700 1 -- 172.21.15.63:0/1308686521 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda14024810 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.755+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda241a38f0 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.755+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda241a65e0 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.756+0000 7fda1b400700 1 -- 172.21.15.63:0/1308686521 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fda14022020 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.756+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda24064bb0 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.757+0000 7fda1b400700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7fda0803e470 0x7fda08040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.757+0000 7fda1b400700 1 -- 172.21.15.63:0/1308686521 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda14053550 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.757+0000 7fda22000700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7fda0803e470 0x7fda08040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.758+0000 7fda22000700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7fda0803e470 0x7fda08040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda10006fd0 tx=0x7fda10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.763+0000 7fda1b400700 1 -- 172.21.15.63:0/1308686521 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda1400f2e0 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:52.999+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fda241a67b0 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.001+0000 7fda1b400700 1 -- 172.21.15.63:0/1308686521 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fda241a67b0 con 0x7fda2410fcb0 2024-05-11T13:17:53.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7fda0803e470 msgr2=0x7fda08040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:53.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7fda0803e470 0x7fda08040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda10006fd0 tx=0x7fda10006e40 comp rx=0 tx=0).stop 2024-05-11T13:17:53.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 msgr2=0x7fda241a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:53.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241a3120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda14009130 tx=0x7fda14011a60 comp rx=0 tx=0).stop 2024-05-11T13:17:53.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 shutdown_connections 2024-05-11T13:17:53.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7fda0803e470 0x7fda08040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:53.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 --2- 172.21.15.63:0/1308686521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda2410fcb0 0x7fda241a3120 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:53.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.005+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 >> 172.21.15.63:0/1308686521 conn(0x7fda2407e420 msgr2=0x7fda2407ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:53.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.006+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 shutdown_connections 2024-05-11T13:17:53.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:53.006+0000 7fda29c00700 1 -- 172.21.15.63:0/1308686521 wait complete. 2024-05-11T13:17:53.750 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling cephadm module... 2024-05-11T13:17:54.066 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:53 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1308686521' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-11T13:17:54.995 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:54 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1382217339' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-11T13:17:54.996 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: ignoring --setuser ceph since I am not root 2024-05-11T13:17:54.996 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: ignoring --setgroup ceph since I am not root 2024-05-11T13:17:54.996 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:54.994+0000 7f8f2d6da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:17:55.379 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:55.242+0000 7f8f2d6da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:17:55.379 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:55.377+0000 7f8f2d6da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:17:55.428 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.379+0000 7f79a6600700 1 Processor -- start 2024-05-11T13:17:55.428 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.379+0000 7f79a6600700 1 -- start start 2024-05-11T13:17:55.429 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.380+0000 7f79a6600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:55.429 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.380+0000 7f79a6600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010b8e0 con 0x7f79a010af80 2024-05-11T13:17:55.429 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.380+0000 7f799f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:55.429 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.380+0000 7f799f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54620/0 (socket says 172.21.15.63:54620) 2024-05-11T13:17:55.430 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.380+0000 7f799f400700 1 -- 172.21.15.63:0/438991972 learned_addr learned my addr 172.21.15.63:0/438991972 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:55.430 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.381+0000 7f799f400700 1 -- 172.21.15.63:0/438991972 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a010c090 con 0x7f79a010af80 2024-05-11T13:17:55.431 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.381+0000 7f799f400700 1 --2- 172.21.15.63:0/438991972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a010b3a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79900098a0 tx=0x7f7990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136487fc8ca5da08 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:55.431 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.381+0000 7f799e000700 1 -- 172.21.15.63:0/438991972 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990004160 con 0x7f79a010af80 2024-05-11T13:17:55.431 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.381+0000 7f799e000700 1 -- 172.21.15.63:0/438991972 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7990004300 con 0x7f79a010af80 2024-05-11T13:17:55.431 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.381+0000 7f799e000700 1 -- 172.21.15.63:0/438991972 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79900119e0 con 0x7f79a010af80 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.382+0000 7f79a6600700 1 -- 172.21.15.63:0/438991972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 msgr2=0x7f79a010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.382+0000 7f79a6600700 1 --2- 172.21.15.63:0/438991972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a010b3a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79900098a0 tx=0x7f7990009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.382+0000 7f79a6600700 1 -- 172.21.15.63:0/438991972 shutdown_connections 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.382+0000 7f79a6600700 1 --2- 172.21.15.63:0/438991972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a010b3a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.382+0000 7f79a6600700 1 -- 172.21.15.63:0/438991972 >> 172.21.15.63:0/438991972 conn(0x7f79a007e420 msgr2=0x7f79a007e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.382+0000 7f79a6600700 1 -- 172.21.15.63:0/438991972 shutdown_connections 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.382+0000 7f79a6600700 1 -- 172.21.15.63:0/438991972 wait complete. 2024-05-11T13:17:55.432 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.383+0000 7f79a6600700 1 Processor -- start 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.384+0000 7f79a6600700 1 -- start start 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.384+0000 7f79a6600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.384+0000 7f79a6600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010f350 con 0x7f79a010af80 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.384+0000 7f799f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.384+0000 7f799f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54626/0 (socket says 172.21.15.63:54626) 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.384+0000 7f799f400700 1 -- 172.21.15.63:0/1382217339 learned_addr learned my addr 172.21.15.63:0/1382217339 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.385+0000 7f799f400700 1 -- 172.21.15.63:0/1382217339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a019f280 con 0x7f79a010af80 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.385+0000 7f799f400700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a019ed40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7990009130 tx=0x7f7990011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:55.434 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.385+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990003b20 con 0x7f79a010af80 2024-05-11T13:17:55.435 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.385+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f799001a8a0 con 0x7f79a010af80 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.385+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990024810 con 0x7f79a010af80 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.385+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a019f510 con 0x7f79a010af80 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.385+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01a2200 con 0x7f79a010af80 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.387+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a0064bb0 con 0x7f79a010af80 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.387+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f7990022020 con 0x7f79a010af80 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.387+0000 7f7997e00700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f798403e470 0x7f7984040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.387+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7990053550 con 0x7f79a010af80 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.388+0000 7f799ea00700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f798403e470 0x7f7984040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:55.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.388+0000 7f799ea00700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f798403e470 0x7f7984040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f798c006fd0 tx=0x7f798c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:55.437 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.393+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f799000f2e0 con 0x7f79a010af80 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.693+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f79a01a2420 con 0x7f79a010af80 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.701+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f79a01a2420 con 0x7f79a010af80 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.703+0000 7f7997e00700 1 -- 172.21.15.63:0/1382217339 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f799001aa50 con 0x7f79a010af80 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f798403e470 msgr2=0x7f7984040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f798403e470 0x7f7984040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f798c006fd0 tx=0x7f798c006e40 comp rx=0 tx=0).stop 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 msgr2=0x7f79a019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a019ed40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7990009130 tx=0x7f7990011a60 comp rx=0 tx=0).stop 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 shutdown_connections 2024-05-11T13:17:55.438 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f798403e470 0x7f7984040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:55.439 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 --2- 172.21.15.63:0/1382217339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a010af80 0x7f79a019ed40 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7990009130 tx=0x7f7990011a60 comp rx=0 tx=0).stop 2024-05-11T13:17:55.439 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.706+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 >> 172.21.15.63:0/1382217339 conn(0x7f79a007e420 msgr2=0x7f79a0108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:55.439 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.707+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 shutdown_connections 2024-05-11T13:17:55.439 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:54.707+0000 7f79a6600700 1 -- 172.21.15.63:0/1382217339 wait complete. 2024-05-11T13:17:55.910 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:55 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1382217339' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-11T13:17:55.910 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:55 smithi063 ceph-mon[29809]: mgrmap e5: x(active, since 5s) 2024-05-11T13:17:55.910 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:55.799+0000 7f8f2d6da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:17:56.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:55.919+0000 7f8f2d6da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:17:57.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:17:56 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/83296575' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-11T13:17:57.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:17:57.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-11T13:17:57.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-11T13:17:57.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-11T13:17:57.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-11T13:17:57.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:17:57.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.066+0000 7f36a3e00700 1 Processor -- start 2024-05-11T13:17:57.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.066+0000 7f36a3e00700 1 -- start start 2024-05-11T13:17:57.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.066+0000 7f36a3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a4108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:57.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.067+0000 7f36a3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4109560 con 0x7f36a4108b70 2024-05-11T13:17:57.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.067+0000 7f36a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a4108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.067+0000 7f36a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a4108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54652/0 (socket says 172.21.15.63:54652) 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.067+0000 7f36a2a00700 1 -- 172.21.15.63:0/2440141273 learned_addr learned my addr 172.21.15.63:0/2440141273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.067+0000 7f36a2a00700 1 -- 172.21.15.63:0/2440141273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a4109d20 con 0x7f36a4108b70 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.068+0000 7f36a2a00700 1 --2- 172.21.15.63:0/2440141273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a4108f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f36940098a0 tx=0x7f3694009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9142ff374898e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.068+0000 7f36a1600700 1 -- 172.21.15.63:0/2440141273 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3694004160 con 0x7f36a4108b70 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.068+0000 7f36a1600700 1 -- 172.21.15.63:0/2440141273 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3694004300 con 0x7f36a4108b70 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.068+0000 7f36a1600700 1 -- 172.21.15.63:0/2440141273 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36940119e0 con 0x7f36a4108b70 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.069+0000 7f36a3e00700 1 -- 172.21.15.63:0/2440141273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 msgr2=0x7f36a4108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.069+0000 7f36a3e00700 1 --2- 172.21.15.63:0/2440141273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a4108f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f36940098a0 tx=0x7f3694009bb0 comp rx=0 tx=0).stop 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.069+0000 7f36a3e00700 1 -- 172.21.15.63:0/2440141273 shutdown_connections 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.069+0000 7f36a3e00700 1 --2- 172.21.15.63:0/2440141273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a4108f90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.069+0000 7f36a3e00700 1 -- 172.21.15.63:0/2440141273 >> 172.21.15.63:0/2440141273 conn(0x7f36a4104130 msgr2=0x7f36a4106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:57.223 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.069+0000 7f36a3e00700 1 -- 172.21.15.63:0/2440141273 shutdown_connections 2024-05-11T13:17:57.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.069+0000 7f36a3e00700 1 -- 172.21.15.63:0/2440141273 wait complete. 2024-05-11T13:17:57.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.071+0000 7f36a3e00700 1 Processor -- start 2024-05-11T13:17:57.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.071+0000 7f36a3e00700 1 -- start start 2024-05-11T13:17:57.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.071+0000 7f36a3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a407eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:57.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.071+0000 7f36a3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36a410cf50 con 0x7f36a4108b70 2024-05-11T13:17:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.071+0000 7f36a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a407eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:17:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.071+0000 7f36a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a407eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54654/0 (socket says 172.21.15.63:54654) 2024-05-11T13:17:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.071+0000 7f36a2a00700 1 -- 172.21.15.63:0/83296575 learned_addr learned my addr 172.21.15.63:0/83296575 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:17:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.072+0000 7f36a2a00700 1 -- 172.21.15.63:0/83296575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a407f090 con 0x7f36a4108b70 2024-05-11T13:17:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.072+0000 7f36a2a00700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a407eb50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36940119b0 tx=0x7f3694011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:17:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.072+0000 7f369b400700 1 -- 172.21.15.63:0/83296575 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3694003af0 con 0x7f36a4108b70 2024-05-11T13:17:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.072+0000 7f369b400700 1 -- 172.21.15.63:0/83296575 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f369401a8a0 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.072+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a407d280 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.073+0000 7f369b400700 1 -- 172.21.15.63:0/83296575 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3694024830 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.073+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a407d7c0 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.074+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a406a630 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.074+0000 7f369b400700 1 -- 172.21.15.63:0/83296575 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f3694022020 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.075+0000 7f369b400700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 0x7f3688040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.075+0000 7f369b400700 1 -- 172.21.15.63:0/83296575 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3694053590 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.075+0000 7f36a2000700 1 -- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 msgr2=0x7f3688040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.075+0000 7f36a2000700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 0x7f3688040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.080+0000 7f369b400700 1 -- 172.21.15.63:0/83296575 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3694053270 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.276+0000 7f36a2000700 1 -- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 msgr2=0x7f3688040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.276+0000 7f36a2000700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 0x7f3688040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.380+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f36a407dfb0 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.381+0000 7f369b400700 1 -- 172.21.15.63:0/83296575 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f369401aa50 con 0x7f36a4108b70 2024-05-11T13:17:57.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 msgr2=0x7f3688040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 0x7f3688040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 msgr2=0x7f36a407eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a407eb50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36940119b0 tx=0x7f3694011a90 comp rx=0 tx=0).stop 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 shutdown_connections 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f368803e470 0x7f3688040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 --2- 172.21.15.63:0/83296575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a4108b70 0x7f36a407eb50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 >> 172.21.15.63:0/83296575 conn(0x7f36a4104130 msgr2=0x7f36a4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.384+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 shutdown_connections 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:56.385+0000 7f36a3e00700 1 -- 172.21.15.63:0/83296575 wait complete. 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-05-11T13:17:57.229 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 5... 2024-05-11T13:17:57.782 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:57.524+0000 7f8f2d6da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:17:57.782 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:57.764+0000 7f8f2d6da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:17:58.071 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:57.985+0000 7f8f2d6da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:17:58.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:58.375+0000 7f8f2d6da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:17:58.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:58.501+0000 7f8f2d6da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:17:59.021 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:58.758+0000 7f8f2d6da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:17:59.021 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:59.019+0000 7f8f2d6da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:18:00.135 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:17:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:17:59.836+0000 7f8f2d6da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:18:00.135 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:00.133+0000 7f8f2d6da200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:18:01.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:00.705+0000 7f8f2d6da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:18:01.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:00.836+0000 7f8f2d6da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:18:01.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:01.309+0000 7f8f2d6da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:18:02.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:01.777+0000 7f8f2d6da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:18:02.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:01.905+0000 7f8f2d6da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:18:02.558 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:02.238+0000 7f8f2d6da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:18:02.559 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:02.425+0000 7f8f2d6da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:18:02.559 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:02.556+0000 7f8f2d6da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:18:03.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:02.962+0000 7f8f2d6da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:18:03.435 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:03.089+0000 7f8f2d6da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:18:04.169 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:03.873+0000 7f8f2d6da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:18:04.169 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:04 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:04.167+0000 7f8f2d6da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:18:04.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: Active manager daemon x restarted 2024-05-11T13:18:04.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: Activating manager daemon x 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: osdmap e2: 0 total, 0 up, 0 in 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: mgrmap e6: x(active, starting, since 0.00967009s) 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T13:18:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:04 smithi063 ceph-mon[29809]: Manager daemon x is now available 2024-05-11T13:18:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: Found migration_current of "None". Setting to last migration. 2024-05-11T13:18:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:05.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T13:18:05.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T13:18:05.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:05 smithi063 ceph-mon[29809]: mgrmap e7: x(active, since 1.01779s) 2024-05-11T13:18:05.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:18:05.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-11T13:18:05.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-11T13:18:05.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:18:05.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.858+0000 7f7bab600700 1 Processor -- start 2024-05-11T13:18:05.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.859+0000 7f7bab600700 1 -- start start 2024-05-11T13:18:05.928 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.859+0000 7f7bab600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba410a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:05.928 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.859+0000 7f7bab600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4108a00 con 0x7f7ba410a420 2024-05-11T13:18:05.928 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.859+0000 7f7ba8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba410a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:05.928 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.860+0000 7f7ba8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba410a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54670/0 (socket says 172.21.15.63:54670) 2024-05-11T13:18:05.928 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.860+0000 7f7ba8c00700 1 -- 172.21.15.63:0/3813660025 learned_addr learned my addr 172.21.15.63:0/3813660025 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.860+0000 7f7ba8c00700 1 -- 172.21.15.63:0/3813660025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba4108b80 con 0x7f7ba410a420 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.860+0000 7f7ba8c00700 1 --2- 172.21.15.63:0/3813660025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba410a840 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b980098a0 tx=0x7f7b98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8063378de62e68 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.860+0000 7f7ba3400700 1 -- 172.21.15.63:0/3813660025 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b98004160 con 0x7f7ba410a420 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.861+0000 7f7ba3400700 1 -- 172.21.15.63:0/3813660025 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7b98004300 con 0x7f7ba410a420 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.861+0000 7f7ba3400700 1 -- 172.21.15.63:0/3813660025 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b980119e0 con 0x7f7ba410a420 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.861+0000 7f7bab600700 1 -- 172.21.15.63:0/3813660025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 msgr2=0x7f7ba410a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.861+0000 7f7bab600700 1 --2- 172.21.15.63:0/3813660025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba410a840 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b980098a0 tx=0x7f7b98009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.862+0000 7f7bab600700 1 -- 172.21.15.63:0/3813660025 shutdown_connections 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.862+0000 7f7bab600700 1 --2- 172.21.15.63:0/3813660025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba410a840 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.862+0000 7f7bab600700 1 -- 172.21.15.63:0/3813660025 >> 172.21.15.63:0/3813660025 conn(0x7f7ba41043c0 msgr2=0x7f7ba41067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.862+0000 7f7bab600700 1 -- 172.21.15.63:0/3813660025 shutdown_connections 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.862+0000 7f7bab600700 1 -- 172.21.15.63:0/3813660025 wait complete. 2024-05-11T13:18:05.929 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.863+0000 7f7bab600700 1 Processor -- start 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.863+0000 7f7bab600700 1 -- start start 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.863+0000 7f7bab600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.864+0000 7f7bab600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba410d2f0 con 0x7f7ba410a420 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.864+0000 7f7ba8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.864+0000 7f7ba8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54682/0 (socket says 172.21.15.63:54682) 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.864+0000 7f7ba8c00700 1 -- 172.21.15.63:0/4202551722 learned_addr learned my addr 172.21.15.63:0/4202551722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.864+0000 7f7ba8c00700 1 -- 172.21.15.63:0/4202551722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba419f3e0 con 0x7f7ba410a420 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.865+0000 7f7ba8c00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba419eea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b980119b0 tx=0x7f7b98011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.865+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b98003af0 con 0x7f7ba410a420 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.865+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7b9801a8a0 con 0x7f7ba410a420 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.865+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b98024830 con 0x7f7ba410a420 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.865+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba419f670 con 0x7f7ba410a420 2024-05-11T13:18:05.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.865+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba41a2360 con 0x7f7ba410a420 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.867+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f7b98022020 con 0x7f7ba410a420 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.867+0000 7f7ba1600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.867+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 --> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b8c041090 con 0x7f7b8c03e4c0 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.867+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b98053500 con 0x7f7ba410a420 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.868+0000 7f7ba3e00700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 msgr2=0x7f7b8c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:57.868+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:58.069+0000 7f7ba3e00700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 msgr2=0x7f7b8c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:58.069+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:18:05.931 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:58.470+0000 7f7ba3e00700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 msgr2=0x7f7b8c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:58.470+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:59.271+0000 7f7ba3e00700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 msgr2=0x7f7b8c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:17:59.271+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:00.873+0000 7f7ba3e00700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 msgr2=0x7f7b8c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:00.873+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:04.076+0000 7f7ba3e00700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 msgr2=0x7f7b8c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1250505383 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:04.077+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:04.177+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f7b9802dcd0 con 0x7f7ba410a420 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:04.178+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 msgr2=0x7f7b8c040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:04.178+0000 7f7ba1600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.182+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f7b98055560 con 0x7f7ba410a420 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.183+0000 7f7ba1600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f7b8c041db0 0x7f7b8c0441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.183+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b8c041090 con 0x7f7b8c041db0 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.184+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f7b8c041db0 0x7f7b8c0441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.188+0000 7f7ba3e00700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f7b8c041db0 0x7f7b8c0441a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7b90003a10 tx=0x7f7b900092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.190+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7b8c041090 con 0x7f7b8c041db0 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.206+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7ba410d470 con 0x7f7b8c041db0 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.206+0000 7f7ba1600700 1 -- 172.21.15.63:0/4202551722 <== mgr.14120 v2:172.21.15.63:6800/405023443 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7ba410d470 con 0x7f7b8c041db0 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f7b8c041db0 msgr2=0x7f7b8c0441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f7b8c041db0 0x7f7b8c0441a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7b90003a10 tx=0x7f7b900092b0 comp rx=0 tx=0).stop 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 msgr2=0x7f7ba419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba419eea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b980119b0 tx=0x7f7b98011a90 comp rx=0 tx=0).stop 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 shutdown_connections 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f7b8c041db0 0x7f7b8c0441a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:05.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ba410a420 0x7f7ba419eea0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:05.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 --2- 172.21.15.63:0/4202551722 >> [v2:172.21.15.63:6800/1250505383,v1:172.21.15.63:6801/1250505383] conn(0x7f7b8c03e4c0 0x7f7b8c040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:05.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 >> 172.21.15.63:0/4202551722 conn(0x7f7ba41043c0 msgr2=0x7f7ba4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:05.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.207+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 shutdown_connections 2024-05-11T13:18:05.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:05.208+0000 7f7bab600700 1 -- 172.21.15.63:0/4202551722 wait complete. 2024-05-11T13:18:05.937 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 5 is available 2024-05-11T13:18:05.937 INFO:teuthology.orchestra.run.smithi063.stdout:Setting orchestrator backend to cephadm... 2024-05-11T13:18:06.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:06 smithi063 ceph-mon[29809]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-11T13:18:06.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:06 smithi063 ceph-mon[29809]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-11T13:18:06.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:06 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:06.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:06 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:07.504 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.515+0000 7efe7a800700 1 Processor -- start 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.515+0000 7efe7a800700 1 -- start start 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.516+0000 7efe7a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7407d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.516+0000 7efe7a800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe7407d7e0 con 0x7efe7407edf0 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.516+0000 7efe79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7407d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.516+0000 7efe79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7407d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59894/0 (socket says 172.21.15.63:59894) 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.517+0000 7efe79400700 1 -- 172.21.15.63:0/3357201392 learned_addr learned my addr 172.21.15.63:0/3357201392 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.517+0000 7efe79400700 1 -- 172.21.15.63:0/3357201392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7407d960 con 0x7efe7407edf0 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.517+0000 7efe79400700 1 --2- 172.21.15.63:0/3357201392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7407d2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe680098a0 tx=0x7efe68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f298305180bb7709 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.518+0000 7efe73e00700 1 -- 172.21.15.63:0/3357201392 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe68004160 con 0x7efe7407edf0 2024-05-11T13:18:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.518+0000 7efe73e00700 1 -- 172.21.15.63:0/3357201392 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7efe68004300 con 0x7efe7407edf0 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.518+0000 7efe73e00700 1 -- 172.21.15.63:0/3357201392 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe680119e0 con 0x7efe7407edf0 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.519+0000 7efe7a800700 1 -- 172.21.15.63:0/3357201392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 msgr2=0x7efe7407d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.519+0000 7efe7a800700 1 --2- 172.21.15.63:0/3357201392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7407d2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe680098a0 tx=0x7efe68009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.519+0000 7efe7a800700 1 -- 172.21.15.63:0/3357201392 shutdown_connections 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.519+0000 7efe7a800700 1 --2- 172.21.15.63:0/3357201392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7407d2a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.519+0000 7efe7a800700 1 -- 172.21.15.63:0/3357201392 >> 172.21.15.63:0/3357201392 conn(0x7efe74104130 msgr2=0x7efe74106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.519+0000 7efe7a800700 1 -- 172.21.15.63:0/3357201392 shutdown_connections 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.519+0000 7efe7a800700 1 -- 172.21.15.63:0/3357201392 wait complete. 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.520+0000 7efe7a800700 1 Processor -- start 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.520+0000 7efe7a800700 1 -- start start 2024-05-11T13:18:07.506 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.520+0000 7efe7a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.520+0000 7efe7a800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410f0a0 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.521+0000 7efe79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.521+0000 7efe79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59904/0 (socket says 172.21.15.63:59904) 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.521+0000 7efe79400700 1 -- 172.21.15.63:0/1975085431 learned_addr learned my addr 172.21.15.63:0/1975085431 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.521+0000 7efe79400700 1 -- 172.21.15.63:0/1975085431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7419f0d0 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.522+0000 7efe79400700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7419eb90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe68009130 tx=0x7efe68011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.522+0000 7efe72000700 1 -- 172.21.15.63:0/1975085431 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe68003b20 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.522+0000 7efe72000700 1 -- 172.21.15.63:0/1975085431 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7efe6801a8a0 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.522+0000 7efe72000700 1 -- 172.21.15.63:0/1975085431 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe68024810 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.522+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7419f360 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.522+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741a2050 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.523+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7406a630 con 0x7efe7407edf0 2024-05-11T13:18:07.507 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.524+0000 7efe72000700 1 -- 172.21.15.63:0/1975085431 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7efe68022020 con 0x7efe7407edf0 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.524+0000 7efe72000700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7efe5c03e3a0 0x7efe5c040860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.524+0000 7efe72000700 1 -- 172.21.15.63:0/1975085431 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe68053780 con 0x7efe7407edf0 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.525+0000 7efe78a00700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7efe5c03e3a0 0x7efe5c040860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.526+0000 7efe78a00700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7efe5c03e3a0 0x7efe5c040860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe60006fd0 tx=0x7efe60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.530+0000 7efe72000700 1 -- 172.21.15.63:0/1975085431 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe6800f2e0 con 0x7efe7407edf0 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.783+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7efe74105310 con 0x7efe5c03e3a0 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.794+0000 7efe72000700 1 -- 172.21.15.63:0/1975085431 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe74105310 con 0x7efe5c03e3a0 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.797+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7efe5c03e3a0 msgr2=0x7efe5c040860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.797+0000 7efe7a800700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7efe5c03e3a0 0x7efe5c040860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe60006fd0 tx=0x7efe60006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.797+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 msgr2=0x7efe7419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.797+0000 7efe7a800700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7419eb90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe68009130 tx=0x7efe68011a60 comp rx=0 tx=0).stop 2024-05-11T13:18:07.508 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.798+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 shutdown_connections 2024-05-11T13:18:07.509 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.798+0000 7efe7a800700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7efe5c03e3a0 0x7efe5c040860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:07.509 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.798+0000 7efe7a800700 1 --2- 172.21.15.63:0/1975085431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7407edf0 0x7efe7419eb90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:07.509 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.798+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 >> 172.21.15.63:0/1975085431 conn(0x7efe74104130 msgr2=0x7efe74104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:07.509 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.798+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 shutdown_connections 2024-05-11T13:18:07.509 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:06.798+0000 7efe7a800700 1 -- 172.21.15.63:0/1975085431 wait complete. 2024-05-11T13:18:08.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:07 smithi063 ceph-mon[29809]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:08.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:07 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:08.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:07 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:08.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:07 smithi063 ceph-mon[29809]: mgrmap e8: x(active, since 2s) 2024-05-11T13:18:08.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:07 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:08 smithi063 ceph-mon[29809]: [11/May/2024:13:18:06] ENGINE Bus STARTING 2024-05-11T13:18:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:08 smithi063 ceph-mon[29809]: [11/May/2024:13:18:07] ENGINE Serving on http://172.21.15.63:8765 2024-05-11T13:18:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:08 smithi063 ceph-mon[29809]: [11/May/2024:13:18:07] ENGINE Serving on https://172.21.15.63:7150 2024-05-11T13:18:08.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:08 smithi063 ceph-mon[29809]: [11/May/2024:13:18:07] ENGINE Bus STARTED 2024-05-11T13:18:08.976 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-11T13:18:08.976 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.108+0000 7fb49a600700 1 Processor -- start 2024-05-11T13:18:08.976 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.108+0000 7fb49a600700 1 -- start start 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.109+0000 7fb49a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.109+0000 7fb49a600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4940833c0 con 0x7fb49410fcb0 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.109+0000 7fb499200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.109+0000 7fb499200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59906/0 (socket says 172.21.15.63:59906) 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.109+0000 7fb499200700 1 -- 172.21.15.63:0/535099890 learned_addr learned my addr 172.21.15.63:0/535099890 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.110+0000 7fb499200700 1 -- 172.21.15.63:0/535099890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4941125e0 con 0x7fb49410fcb0 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.110+0000 7fb499200700 1 --2- 172.21.15.63:0/535099890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941120a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098a0 tx=0x7fb488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea073952586f8b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.110+0000 7fb493400700 1 -- 172.21.15.63:0/535099890 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488004160 con 0x7fb49410fcb0 2024-05-11T13:18:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.110+0000 7fb493400700 1 -- 172.21.15.63:0/535099890 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb488004300 con 0x7fb49410fcb0 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.110+0000 7fb493400700 1 -- 172.21.15.63:0/535099890 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4880119e0 con 0x7fb49410fcb0 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.111+0000 7fb49a600700 1 -- 172.21.15.63:0/535099890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 msgr2=0x7fb4941120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.111+0000 7fb49a600700 1 --2- 172.21.15.63:0/535099890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941120a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098a0 tx=0x7fb488009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.111+0000 7fb49a600700 1 -- 172.21.15.63:0/535099890 shutdown_connections 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.111+0000 7fb49a600700 1 --2- 172.21.15.63:0/535099890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941120a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.111+0000 7fb49a600700 1 -- 172.21.15.63:0/535099890 >> 172.21.15.63:0/535099890 conn(0x7fb49407e240 msgr2=0x7fb494080660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.111+0000 7fb49a600700 1 -- 172.21.15.63:0/535099890 shutdown_connections 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.111+0000 7fb49a600700 1 -- 172.21.15.63:0/535099890 wait complete. 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.112+0000 7fb49a600700 1 Processor -- start 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.113+0000 7fb49a600700 1 -- start start 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.113+0000 7fb49a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.113+0000 7fb49a600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb494113480 con 0x7fb49410fcb0 2024-05-11T13:18:08.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.113+0000 7fb499200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.113+0000 7fb499200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59916/0 (socket says 172.21.15.63:59916) 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.113+0000 7fb499200700 1 -- 172.21.15.63:0/589303928 learned_addr learned my addr 172.21.15.63:0/589303928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.114+0000 7fb499200700 1 -- 172.21.15.63:0/589303928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4941a3490 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.114+0000 7fb499200700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941a2f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb488009130 tx=0x7fb488011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.114+0000 7fb491600700 1 -- 172.21.15.63:0/589303928 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488003b20 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.114+0000 7fb491600700 1 -- 172.21.15.63:0/589303928 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb48801a8a0 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.114+0000 7fb491600700 1 -- 172.21.15.63:0/589303928 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488024810 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.114+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4941a3720 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.114+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941a6410 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.116+0000 7fb491600700 1 -- 172.21.15.63:0/589303928 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb488022020 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.116+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49406a630 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.116+0000 7fb491600700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fb47c03e4c0 0x7fb47c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.116+0000 7fb491600700 1 -- 172.21.15.63:0/589303928 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb488053980 con 0x7fb49410fcb0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.117+0000 7fb493e00700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fb47c03e4c0 0x7fb47c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:08.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.117+0000 7fb493e00700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fb47c03e4c0 0x7fb47c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb480006fd0 tx=0x7fb480006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.122+0000 7fb491600700 1 -- 172.21.15.63:0/589303928 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb48800f2e0 con 0x7fb49410fcb0 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.361+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb49407f380 con 0x7fb47c03e4c0 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.362+0000 7fb491600700 1 -- 172.21.15.63:0/589303928 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb49407f380 con 0x7fb47c03e4c0 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.365+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fb47c03e4c0 msgr2=0x7fb47c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.365+0000 7fb49a600700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fb47c03e4c0 0x7fb47c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb480006fd0 tx=0x7fb480006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.365+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 msgr2=0x7fb4941a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.365+0000 7fb49a600700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941a2f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb488009130 tx=0x7fb488011a60 comp rx=0 tx=0).stop 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.366+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 shutdown_connections 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.366+0000 7fb49a600700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fb47c03e4c0 0x7fb47c040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:08.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.366+0000 7fb49a600700 1 --2- 172.21.15.63:0/589303928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49410fcb0 0x7fb4941a2f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:08.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.366+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 >> 172.21.15.63:0/589303928 conn(0x7fb49407e240 msgr2=0x7fb49407ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:08.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.366+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 shutdown_connections 2024-05-11T13:18:08.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:08.366+0000 7fb49a600700 1 -- 172.21.15.63:0/589303928 wait complete. 2024-05-11T13:18:08.981 INFO:teuthology.orchestra.run.smithi063.stdout:Generating ssh key... 2024-05-11T13:18:09.895 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:09 smithi063 ceph-mon[29809]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:10.151 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: Generating public/private rsa key pair. 2024-05-11T13:18:10.151 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: Your identification has been saved in /tmp/tmpg2izv72i/key. 2024-05-11T13:18:10.151 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: Your public key has been saved in /tmp/tmpg2izv72i/key.pub. 2024-05-11T13:18:10.151 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: The key fingerprint is: 2024-05-11T13:18:10.151 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: SHA256:2Fm+Fkw94UzO3YmrZoi3k8szG1Zs59oPZq+0QyEGRQU ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:18:10.151 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: The key's randomart image is: 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: +---[RSA 3072]----+ 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | oE=. | 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | . B o...| 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | + B....| 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | o *.o o. | 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | . S =+.o. | 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | . +o+. | 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | . =++.* | 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | +*= *.+ | 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: | =*. ++o | 2024-05-11T13:18:10.152 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: +----[SHA256]-----+ 2024-05-11T13:18:10.863 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.641+0000 7f5dbfe00700 1 Processor -- start 2024-05-11T13:18:10.863 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.642+0000 7f5dbfe00700 1 -- start start 2024-05-11T13:18:10.863 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.642+0000 7f5dbfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc010b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:10.863 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.642+0000 7f5dbfe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010b810 con 0x7f5dc010ae20 2024-05-11T13:18:10.863 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.642+0000 7f5dbea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc010b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:10.865 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.643+0000 7f5dbea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc010b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59918/0 (socket says 172.21.15.63:59918) 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.643+0000 7f5dbea00700 1 -- 172.21.15.63:0/1673069829 learned_addr learned my addr 172.21.15.63:0/1673069829 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.643+0000 7f5dbea00700 1 -- 172.21.15.63:0/1673069829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc010bfd0 con 0x7f5dc010ae20 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.644+0000 7f5dbea00700 1 --2- 172.21.15.63:0/1673069829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc010b240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098a0 tx=0x7f5db0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71923bc2a9040ef server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.644+0000 7f5dbd600700 1 -- 172.21.15.63:0/1673069829 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db0004160 con 0x7f5dc010ae20 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.644+0000 7f5dbd600700 1 -- 172.21.15.63:0/1673069829 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5db0004300 con 0x7f5dc010ae20 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.644+0000 7f5dbd600700 1 -- 172.21.15.63:0/1673069829 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db00119e0 con 0x7f5dc010ae20 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.645+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1673069829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 msgr2=0x7f5dc010b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.645+0000 7f5dbfe00700 1 --2- 172.21.15.63:0/1673069829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc010b240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098a0 tx=0x7f5db0009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.645+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1673069829 shutdown_connections 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.645+0000 7f5dbfe00700 1 --2- 172.21.15.63:0/1673069829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc010b240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.645+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1673069829 >> 172.21.15.63:0/1673069829 conn(0x7f5dc007e2b0 msgr2=0x7f5dc007e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:10.866 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.645+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1673069829 shutdown_connections 2024-05-11T13:18:10.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.645+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1673069829 wait complete. 2024-05-11T13:18:10.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.646+0000 7f5dbfe00700 1 Processor -- start 2024-05-11T13:18:10.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.646+0000 7f5dbfe00700 1 -- start start 2024-05-11T13:18:10.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.647+0000 7f5dbfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:10.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.647+0000 7f5dbfe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010f200 con 0x7f5dc010ae20 2024-05-11T13:18:10.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.647+0000 7f5dbea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:10.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.647+0000 7f5dbea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59930/0 (socket says 172.21.15.63:59930) 2024-05-11T13:18:10.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.647+0000 7f5dbea00700 1 -- 172.21.15.63:0/1471291522 learned_addr learned my addr 172.21.15.63:0/1471291522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:10.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.648+0000 7f5dbea00700 1 -- 172.21.15.63:0/1471291522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc019f0d0 con 0x7f5dc010ae20 2024-05-11T13:18:10.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.648+0000 7f5dbea00700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc019eb90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5db00119b0 tx=0x7f5db0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:10.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.648+0000 7f5db7400700 1 -- 172.21.15.63:0/1471291522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db0003af0 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.648+0000 7f5db7400700 1 -- 172.21.15.63:0/1471291522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5db001a8a0 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.648+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc019f360 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.648+0000 7f5db7400700 1 -- 172.21.15.63:0/1471291522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db0024830 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.649+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01a2050 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.650+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc010f380 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.650+0000 7f5db7400700 1 -- 172.21.15.63:0/1471291522 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5db0022020 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.650+0000 7f5db7400700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f5da403e470 0x7f5da4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.650+0000 7f5db7400700 1 -- 172.21.15.63:0/1471291522 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5db0053950 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.651+0000 7f5dbe000700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f5da403e470 0x7f5da4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.651+0000 7f5dbe000700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f5da403e470 0x7f5da4040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5dac006fd0 tx=0x7f5dac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.656+0000 7f5db7400700 1 -- 172.21.15.63:0/1471291522 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5db00212c0 con 0x7f5dc010ae20 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:09.893+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5dc0108ab0 con 0x7f5da403e470 2024-05-11T13:18:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.147+0000 7f5db7400700 1 -- 172.21.15.63:0/1471291522 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5dc0108ab0 con 0x7f5da403e470 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.153+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f5da403e470 msgr2=0x7f5da4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.153+0000 7f5dbfe00700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f5da403e470 0x7f5da4040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5dac006fd0 tx=0x7f5dac006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.153+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 msgr2=0x7f5dc019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.153+0000 7f5dbfe00700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc019eb90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5db00119b0 tx=0x7f5db0011a90 comp rx=0 tx=0).stop 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.154+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 shutdown_connections 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.154+0000 7f5dbfe00700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f5da403e470 0x7f5da4040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.154+0000 7f5dbfe00700 1 --2- 172.21.15.63:0/1471291522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc010ae20 0x7f5dc019eb90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.154+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 >> 172.21.15.63:0/1471291522 conn(0x7f5dc007e2b0 msgr2=0x7f5dc01083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.154+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 shutdown_connections 2024-05-11T13:18:10.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:10.154+0000 7f5dbfe00700 1 -- 172.21.15.63:0/1471291522 wait complete. 2024-05-11T13:18:11.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:11 smithi063 ceph-mon[29809]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:11.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:11 smithi063 ceph-mon[29809]: Generating ssh key... 2024-05-11T13:18:11.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:11 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:11.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:11 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:12.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:12 smithi063 ceph-mon[29809]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:12.447 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8VnGcJKjYdsCmidM2pzJgZUDHyBEtefelvRHNLi3PUiwDnBT3kRBzYFCZNSzRbBjckeG1CyygLvU7U5YaXivjjnj5gu4VLvgqDM1S0aKnWvOppVQGu3OllGfz+MV6L4dmBdhH4r/1KZT56DdGz747sRknhWdKfvDkjX8YZBSxKuZC+qOiCu8kBCOev93mVnvLP8eF/tEIwo87pMo8W2b+BFzsRN0R/5TZ+dY8Ftl/s98cUh6iR8sQL0B9U8yYTwyGCgiFjz7hMC9JGVnXdhEU8MPgr/obJH0C8CKhBXZmqWVpzWg2VaQCnmd2X7EDxYFxa4lVu6FuuaI0zByLlMNnv5QnfSMqSdz2LGk2230CZUh6pNB3Y2I+CkeTpi8HKRATdGsJdLgez+nKJIYcOiG4NB2wF9I8oq4fOx1xpxNcUnHbuhs/YJxNaoQZpVElNAL0M5kx6ISFfbvDw8OI7fnFFZOP2acbwalAnDxznw81M9aq6FAJxhekOft0yV32Ac= ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:18:12.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.457+0000 7f8c9ae00700 1 Processor -- start 2024-05-11T13:18:12.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.457+0000 7f8c9ae00700 1 -- start start 2024-05-11T13:18:12.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.457+0000 7f8c9ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9410ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.457+0000 7f8c9ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9407cdd0 con 0x7f8c94109670 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.458+0000 7f8c99a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9410ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.458+0000 7f8c99a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9410ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38148/0 (socket says 172.21.15.63:38148) 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.458+0000 7f8c99a00700 1 -- 172.21.15.63:0/1993901320 learned_addr learned my addr 172.21.15.63:0/1993901320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.458+0000 7f8c99a00700 1 -- 172.21.15.63:0/1993901320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9410bfa0 con 0x7f8c94109670 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.458+0000 7f8c99a00700 1 --2- 172.21.15.63:0/1993901320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9410ba60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009e80 tx=0x7f8c8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3b46485e1ccc70 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.459+0000 7f8c93e00700 1 -- 172.21.15.63:0/1993901320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c88017070 con 0x7f8c94109670 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.459+0000 7f8c93e00700 1 -- 172.21.15.63:0/1993901320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c8800b8c0 con 0x7f8c94109670 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.459+0000 7f8c93e00700 1 -- 172.21.15.63:0/1993901320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c88004420 con 0x7f8c94109670 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.459+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1993901320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 msgr2=0x7f8c9410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.459+0000 7f8c9ae00700 1 --2- 172.21.15.63:0/1993901320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9410ba60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009e80 tx=0x7f8c8800b330 comp rx=0 tx=0).stop 2024-05-11T13:18:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.460+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1993901320 shutdown_connections 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.460+0000 7f8c9ae00700 1 --2- 172.21.15.63:0/1993901320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9410ba60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.460+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1993901320 >> 172.21.15.63:0/1993901320 conn(0x7f8c94102fe0 msgr2=0x7f8c94105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.460+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1993901320 shutdown_connections 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.460+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1993901320 wait complete. 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.461+0000 7f8c9ae00700 1 Processor -- start 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.461+0000 7f8c9ae00700 1 -- start start 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.461+0000 7f8c9ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.461+0000 7f8c9ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9410ce50 con 0x7f8c94109670 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.461+0000 7f8c99a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c99a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38152/0 (socket says 172.21.15.63:38152) 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c99a00700 1 -- 172.21.15.63:0/1696662655 learned_addr learned my addr 172.21.15.63:0/1696662655 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c99a00700 1 -- 172.21.15.63:0/1696662655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9419f160 con 0x7f8c94109670 2024-05-11T13:18:12.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c99a00700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9419ec20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c88007130 tx=0x7f8c8800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c92000700 1 -- 172.21.15.63:0/1696662655 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c88020020 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c92000700 1 -- 172.21.15.63:0/1696662655 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c88003bd0 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9419f3f0 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.462+0000 7f8c92000700 1 -- 172.21.15.63:0/1696662655 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c88017040 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.463+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c941a20e0 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.464+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9406a630 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.464+0000 7f8c92000700 1 -- 172.21.15.63:0/1696662655 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8c88023020 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.464+0000 7f8c92000700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f8c7c03e470 0x7f8c7c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.465+0000 7f8c92000700 1 -- 172.21.15.63:0/1696662655 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8c8800f050 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.465+0000 7f8c99000700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f8c7c03e470 0x7f8c7c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.466+0000 7f8c99000700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f8c7c03e470 0x7f8c7c040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006fd0 tx=0x7f8c80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.470+0000 7f8c92000700 1 -- 172.21.15.63:0/1696662655 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c8802a020 con 0x7f8c94109670 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.705+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8c94104150 con 0x7f8c7c03e470 2024-05-11T13:18:12.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.706+0000 7f8c92000700 1 -- 172.21.15.63:0/1696662655 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8c94104150 con 0x7f8c7c03e470 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f8c7c03e470 msgr2=0x7f8c7c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f8c7c03e470 0x7f8c7c040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006fd0 tx=0x7f8c80006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 msgr2=0x7f8c9419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9419ec20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c88007130 tx=0x7f8c8800bdc0 comp rx=0 tx=0).stop 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 shutdown_connections 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f8c7c03e470 0x7f8c7c040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 --2- 172.21.15.63:0/1696662655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c94109670 0x7f8c9419ec20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 >> 172.21.15.63:0/1696662655 conn(0x7f8c94102fe0 msgr2=0x7f8c94103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 shutdown_connections 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:11.709+0000 7f8c9ae00700 1 -- 172.21.15.63:0/1696662655 wait complete. 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:Adding key to root@localhost authorized_keys... 2024-05-11T13:18:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:Adding host smithi063... 2024-05-11T13:18:14.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:13 smithi063 ceph-mon[29809]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:15.062 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:14 smithi063 ceph-mon[29809]: Deploying cephadm binary to smithi063 2024-05-11T13:18:20.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Added host 'smithi063' with addr '172.21.15.63' 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.047+0000 7ff4bcc00700 1 Processor -- start 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.048+0000 7ff4bcc00700 1 -- start start 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.048+0000 7ff4bcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b810b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.048+0000 7ff4bcc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b810b7e0 con 0x7ff4b810adf0 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.049+0000 7ff4b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b810b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.049+0000 7ff4b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b810b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38166/0 (socket says 172.21.15.63:38166) 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.049+0000 7ff4b7400700 1 -- 172.21.15.63:0/1555794949 learned_addr learned my addr 172.21.15.63:0/1555794949 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.049+0000 7ff4b7400700 1 -- 172.21.15.63:0/1555794949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b810bf50 con 0x7ff4b810adf0 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.049+0000 7ff4b7400700 1 --2- 172.21.15.63:0/1555794949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b810b210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80098a0 tx=0x7ff4a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae1d750967876b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.050+0000 7ff4b6000700 1 -- 172.21.15.63:0/1555794949 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a8004160 con 0x7ff4b810adf0 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.050+0000 7ff4b6000700 1 -- 172.21.15.63:0/1555794949 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff4a8004300 con 0x7ff4b810adf0 2024-05-11T13:18:20.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.050+0000 7ff4b6000700 1 -- 172.21.15.63:0/1555794949 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a80119e0 con 0x7ff4b810adf0 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.050+0000 7ff4bcc00700 1 -- 172.21.15.63:0/1555794949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 msgr2=0x7ff4b810b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.050+0000 7ff4bcc00700 1 --2- 172.21.15.63:0/1555794949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b810b210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80098a0 tx=0x7ff4a8009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.051+0000 7ff4bcc00700 1 -- 172.21.15.63:0/1555794949 shutdown_connections 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.051+0000 7ff4bcc00700 1 --2- 172.21.15.63:0/1555794949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b810b210 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.051+0000 7ff4bcc00700 1 -- 172.21.15.63:0/1555794949 >> 172.21.15.63:0/1555794949 conn(0x7ff4b807e240 msgr2=0x7ff4b807e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.051+0000 7ff4bcc00700 1 -- 172.21.15.63:0/1555794949 shutdown_connections 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.051+0000 7ff4bcc00700 1 -- 172.21.15.63:0/1555794949 wait complete. 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.052+0000 7ff4bcc00700 1 Processor -- start 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.052+0000 7ff4bcc00700 1 -- start start 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.052+0000 7ff4bcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.052+0000 7ff4bcc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b810f180 con 0x7ff4b810adf0 2024-05-11T13:18:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.052+0000 7ff4b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.053+0000 7ff4b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38176/0 (socket says 172.21.15.63:38176) 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.053+0000 7ff4b7400700 1 -- 172.21.15.63:0/4145768028 learned_addr learned my addr 172.21.15.63:0/4145768028 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.053+0000 7ff4b7400700 1 -- 172.21.15.63:0/4145768028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b819f0e0 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.053+0000 7ff4b7400700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b819eba0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80119b0 tx=0x7ff4a8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.053+0000 7ff4afe00700 1 -- 172.21.15.63:0/4145768028 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a8003af0 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.054+0000 7ff4afe00700 1 -- 172.21.15.63:0/4145768028 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff4a801a8a0 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.054+0000 7ff4afe00700 1 -- 172.21.15.63:0/4145768028 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a8024830 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.054+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b819f370 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.054+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b81a2060 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.055+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b806a630 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.055+0000 7ff4afe00700 1 -- 172.21.15.63:0/4145768028 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff4a8022020 con 0x7ff4b810adf0 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.055+0000 7ff4afe00700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff49c03e4c0 0x7ff49c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:20.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.056+0000 7ff4afe00700 1 -- 172.21.15.63:0/4145768028 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff4a8053900 con 0x7ff4b810adf0 2024-05-11T13:18:20.425 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.056+0000 7ff4b6a00700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff49c03e4c0 0x7ff49c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:20.425 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.057+0000 7ff4b6a00700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff49c03e4c0 0x7ff49c040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4006fd0 tx=0x7ff4a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:20.425 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.061+0000 7ff4afe00700 1 -- 172.21.15.63:0/4145768028 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4a8053250 con 0x7ff4b810adf0 2024-05-11T13:18:20.425 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:13.303+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}) -- 0x7ff4b8108b10 con 0x7ff49c03e4c0 2024-05-11T13:18:20.425 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.657+0000 7ff4afe00700 1 -- 172.21.15.63:0/4145768028 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff4b8108b10 con 0x7ff49c03e4c0 2024-05-11T13:18:20.426 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff49c03e4c0 msgr2=0x7ff49c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:20.426 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff49c03e4c0 0x7ff49c040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4006fd0 tx=0x7ff4a4006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:20.426 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 msgr2=0x7ff4b819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:20.426 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b819eba0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80119b0 tx=0x7ff4a8011a90 comp rx=0 tx=0).stop 2024-05-11T13:18:20.427 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 shutdown_connections 2024-05-11T13:18:20.427 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff49c03e4c0 0x7ff49c040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:20.427 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 --2- 172.21.15.63:0/4145768028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b810adf0 0x7ff4b819eba0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:20.427 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 >> 172.21.15.63:0/4145768028 conn(0x7ff4b807e240 msgr2=0x7ff4b8108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:20.427 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.661+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 shutdown_connections 2024-05-11T13:18:20.427 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:19.662+0000 7ff4bcc00700 1 -- 172.21.15.63:0/4145768028 wait complete. 2024-05-11T13:18:20.427 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying unmanaged mon service... 2024-05-11T13:18:20.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:20 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:20.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:20 smithi063 ceph-mon[29809]: Added host smithi063 2024-05-11T13:18:20.924 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:20 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:22.509 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:22 smithi063 ceph-mon[29809]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:22.509 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:22 smithi063 ceph-mon[29809]: Saving service mon spec with placement count:5 2024-05-11T13:18:22.510 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:22 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:22.782 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.099+0000 7fdc4aa00700 1 Processor -- start 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.099+0000 7fdc4aa00700 1 -- start start 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.099+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc441091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.099+0000 7fdc4aa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441096e0 con 0x7fdc44108d80 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.100+0000 7fdc43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc441091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.100+0000 7fdc43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc441091a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36998/0 (socket says 172.21.15.63:36998) 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.100+0000 7fdc43e00700 1 -- 172.21.15.63:0/3119471841 learned_addr learned my addr 172.21.15.63:0/3119471841 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:22.783 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.101+0000 7fdc43e00700 1 -- 172.21.15.63:0/3119471841 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc44109860 con 0x7fdc44108d80 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.101+0000 7fdc43e00700 1 --2- 172.21.15.63:0/3119471841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc441091a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdc340098a0 tx=0x7fdc34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e31e9f62528e082 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.101+0000 7fdc43400700 1 -- 172.21.15.63:0/3119471841 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc34004160 con 0x7fdc44108d80 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.101+0000 7fdc43400700 1 -- 172.21.15.63:0/3119471841 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdc34004300 con 0x7fdc44108d80 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.101+0000 7fdc43400700 1 -- 172.21.15.63:0/3119471841 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc340119e0 con 0x7fdc44108d80 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.102+0000 7fdc4aa00700 1 -- 172.21.15.63:0/3119471841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 msgr2=0x7fdc441091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.102+0000 7fdc4aa00700 1 --2- 172.21.15.63:0/3119471841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc441091a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdc340098a0 tx=0x7fdc34009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.102+0000 7fdc4aa00700 1 -- 172.21.15.63:0/3119471841 shutdown_connections 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.102+0000 7fdc4aa00700 1 --2- 172.21.15.63:0/3119471841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc441091a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.102+0000 7fdc4aa00700 1 -- 172.21.15.63:0/3119471841 >> 172.21.15.63:0/3119471841 conn(0x7fdc441043c0 msgr2=0x7fdc44106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:22.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.102+0000 7fdc4aa00700 1 -- 172.21.15.63:0/3119471841 shutdown_connections 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.102+0000 7fdc4aa00700 1 -- 172.21.15.63:0/3119471841 wait complete. 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.104+0000 7fdc4aa00700 1 Processor -- start 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.104+0000 7fdc4aa00700 1 -- start start 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.104+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc4419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.104+0000 7fdc4aa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4410ad20 con 0x7fdc44108d80 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.104+0000 7fdc43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc4419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.104+0000 7fdc43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc4419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37002/0 (socket says 172.21.15.63:37002) 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.104+0000 7fdc43e00700 1 -- 172.21.15.63:0/2497649402 learned_addr learned my addr 172.21.15.63:0/2497649402 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.105+0000 7fdc43e00700 1 -- 172.21.15.63:0/2497649402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4419f330 con 0x7fdc44108d80 2024-05-11T13:18:22.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.105+0000 7fdc43e00700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc4419edf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdc340116a0 tx=0x7fdc340046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.105+0000 7fdc41600700 1 -- 172.21.15.63:0/2497649402 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc34003b70 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.105+0000 7fdc41600700 1 -- 172.21.15.63:0/2497649402 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdc3401a8a0 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.105+0000 7fdc41600700 1 -- 172.21.15.63:0/2497649402 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc34024890 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.106+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4419f5c0 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.106+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc441a22b0 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.107+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc44064bb0 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.107+0000 7fdc41600700 1 -- 172.21.15.63:0/2497649402 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fdc34022020 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.108+0000 7fdc41600700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fdc2803e4c0 0x7fdc28040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.108+0000 7fdc41600700 1 -- 172.21.15.63:0/2497649402 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc34053af0 con 0x7fdc44108d80 2024-05-11T13:18:22.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.108+0000 7fdc3be00700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fdc2803e4c0 0x7fdc28040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:22.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.109+0000 7fdc3be00700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fdc2803e4c0 0x7fdc28040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c006fd0 tx=0x7fdc2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:22.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.113+0000 7fdc41600700 1 -- 172.21.15.63:0/2497649402 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc340212c0 con 0x7fdc44108d80 2024-05-11T13:18:22.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.378+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdc44105520 con 0x7fdc2803e4c0 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.388+0000 7fdc41600700 1 -- 172.21.15.63:0/2497649402 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdc44105520 con 0x7fdc2803e4c0 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fdc2803e4c0 msgr2=0x7fdc28040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fdc2803e4c0 0x7fdc28040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c006fd0 tx=0x7fdc2c006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 msgr2=0x7fdc4419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc4419edf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdc340116a0 tx=0x7fdc340046a0 comp rx=0 tx=0).stop 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 shutdown_connections 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7fdc2803e4c0 0x7fdc28040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 --2- 172.21.15.63:0/2497649402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc44108d80 0x7fdc4419edf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 >> 172.21.15.63:0/2497649402 conn(0x7fdc441043c0 msgr2=0x7fdc44104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:22.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 shutdown_connections 2024-05-11T13:18:22.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:21.391+0000 7fdc4aa00700 1 -- 172.21.15.63:0/2497649402 wait complete. 2024-05-11T13:18:22.790 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying unmanaged mgr service... 2024-05-11T13:18:24.666 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-11T13:18:24.666 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.686+0000 7f2a3de00700 1 Processor -- start 2024-05-11T13:18:24.666 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.687+0000 7f2a3de00700 1 -- start start 2024-05-11T13:18:24.666 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.687+0000 7f2a3de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:24.666 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.687+0000 7f2a3de00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3807ce90 con 0x7f2a38107580 2024-05-11T13:18:24.666 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.687+0000 7f2a36a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:24.667 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.688+0000 7f2a36a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37012/0 (socket says 172.21.15.63:37012) 2024-05-11T13:18:24.667 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.688+0000 7f2a36a00700 1 -- 172.21.15.63:0/3200646953 learned_addr learned my addr 172.21.15.63:0/3200646953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:24.670 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.688+0000 7f2a36a00700 1 -- 172.21.15.63:0/3200646953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a38109ef0 con 0x7f2a38107580 2024-05-11T13:18:24.670 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.688+0000 7f2a36a00700 1 --2- 172.21.15.63:0/3200646953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381099b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2a280098a0 tx=0x7f2a28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a08bbe15eb43b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:24.670 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.689+0000 7f2a35600700 1 -- 172.21.15.63:0/3200646953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a28004160 con 0x7f2a38107580 2024-05-11T13:18:24.670 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.689+0000 7f2a35600700 1 -- 172.21.15.63:0/3200646953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2a28004300 con 0x7f2a38107580 2024-05-11T13:18:24.670 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.689+0000 7f2a35600700 1 -- 172.21.15.63:0/3200646953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a280119e0 con 0x7f2a38107580 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.690+0000 7f2a3de00700 1 -- 172.21.15.63:0/3200646953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 msgr2=0x7f2a381099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.690+0000 7f2a3de00700 1 --2- 172.21.15.63:0/3200646953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381099b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2a280098a0 tx=0x7f2a28009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.690+0000 7f2a3de00700 1 -- 172.21.15.63:0/3200646953 shutdown_connections 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.690+0000 7f2a3de00700 1 --2- 172.21.15.63:0/3200646953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381099b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.690+0000 7f2a3de00700 1 -- 172.21.15.63:0/3200646953 >> 172.21.15.63:0/3200646953 conn(0x7f2a381031e0 msgr2=0x7f2a38105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.690+0000 7f2a3de00700 1 -- 172.21.15.63:0/3200646953 shutdown_connections 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.690+0000 7f2a3de00700 1 -- 172.21.15.63:0/3200646953 wait complete. 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.691+0000 7f2a3de00700 1 Processor -- start 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.692+0000 7f2a3de00700 1 -- start start 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.692+0000 7f2a3de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.692+0000 7f2a3de00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3810aec0 con 0x7f2a38107580 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.692+0000 7f2a36a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.692+0000 7f2a36a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37028/0 (socket says 172.21.15.63:37028) 2024-05-11T13:18:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.692+0000 7f2a36a00700 1 -- 172.21.15.63:0/2333997994 learned_addr learned my addr 172.21.15.63:0/2333997994 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.693+0000 7f2a36a00700 1 -- 172.21.15.63:0/2333997994 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a381a35f0 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.693+0000 7f2a36a00700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381a30b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a280119b0 tx=0x7f2a28011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.693+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a280039f0 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.693+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2a28003b90 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.693+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a381a3880 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.693+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a2801a620 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.694+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a381a6570 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.695+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a38064bb0 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.695+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2a2801a8d0 con 0x7f2a38107580 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.696+0000 7f2a3ca00700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f2a1c03e530 0x7f2a1c0409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:24.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.696+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a28053cd0 con 0x7f2a38107580 2024-05-11T13:18:24.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.696+0000 7f2a36000700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f2a1c03e530 0x7f2a1c0409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:24.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.697+0000 7f2a36000700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f2a1c03e530 0x7f2a1c0409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006fd0 tx=0x7f2a24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:24.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.701+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a28059020 con 0x7f2a38107580 2024-05-11T13:18:24.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.753+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mon.0 v2:172.21.15.63:3300/0 7 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a28018af0 con 0x7f2a38107580 2024-05-11T13:18:24.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.962+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2a380008d0 con 0x7f2a1c03e530 2024-05-11T13:18:24.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.968+0000 7f2a3ca00700 1 -- 172.21.15.63:0/2333997994 <== mgr.14120 v2:172.21.15.63:6800/405023443 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2a380008d0 con 0x7f2a1c03e530 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f2a1c03e530 msgr2=0x7f2a1c0409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f2a1c03e530 0x7f2a1c0409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006fd0 tx=0x7f2a24006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 msgr2=0x7f2a381a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381a30b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a280119b0 tx=0x7f2a28011a90 comp rx=0 tx=0).stop 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 shutdown_connections 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f2a1c03e530 0x7f2a1c0409f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 --2- 172.21.15.63:0/2333997994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a38107580 0x7f2a381a30b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 >> 172.21.15.63:0/2333997994 conn(0x7f2a381031e0 msgr2=0x7f2a381055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 shutdown_connections 2024-05-11T13:18:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:23.971+0000 7f2a3de00700 1 -- 172.21.15.63:0/2333997994 wait complete. 2024-05-11T13:18:24.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:24 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:24.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:24 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:25.911 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:25 smithi063 ceph-mon[29809]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:25.911 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:25 smithi063 ceph-mon[29809]: Saving service mgr spec with placement count:2 2024-05-11T13:18:25.912 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:25 smithi063 ceph-mon[29809]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:18:25.912 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:25 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:25.912 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:25 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/511370710' entity='client.admin' 2024-05-11T13:18:26.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.295+0000 7f6bba800700 1 Processor -- start 2024-05-11T13:18:26.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.296+0000 7f6bba800700 1 -- start start 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.296+0000 7f6bba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb407d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.296+0000 7f6bba800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb407d800 con 0x7f6bb407edc0 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.297+0000 7f6bb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb407d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.297+0000 7f6bb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb407d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37030/0 (socket says 172.21.15.63:37030) 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.297+0000 7f6bb3400700 1 -- 172.21.15.63:0/631070547 learned_addr learned my addr 172.21.15.63:0/631070547 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.297+0000 7f6bb3400700 1 -- 172.21.15.63:0/631070547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb407d980 con 0x7f6bb407edc0 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.298+0000 7f6bb3400700 1 --2- 172.21.15.63:0/631070547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb407d2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098a0 tx=0x7f6ba4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ccd7994d8a422b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.298+0000 7f6bb2000700 1 -- 172.21.15.63:0/631070547 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba4004160 con 0x7f6bb407edc0 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.298+0000 7f6bb2000700 1 -- 172.21.15.63:0/631070547 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba4004300 con 0x7f6bb407edc0 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.298+0000 7f6bb2000700 1 -- 172.21.15.63:0/631070547 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba4011960 con 0x7f6bb407edc0 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.299+0000 7f6bba800700 1 -- 172.21.15.63:0/631070547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 msgr2=0x7f6bb407d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:26.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.299+0000 7f6bba800700 1 --2- 172.21.15.63:0/631070547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb407d2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098a0 tx=0x7f6ba4009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.299+0000 7f6bba800700 1 -- 172.21.15.63:0/631070547 shutdown_connections 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.299+0000 7f6bba800700 1 --2- 172.21.15.63:0/631070547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb407d2c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.299+0000 7f6bba800700 1 -- 172.21.15.63:0/631070547 >> 172.21.15.63:0/631070547 conn(0x7f6bb41043c0 msgr2=0x7f6bb41067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.299+0000 7f6bba800700 1 -- 172.21.15.63:0/631070547 shutdown_connections 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.299+0000 7f6bba800700 1 -- 172.21.15.63:0/631070547 wait complete. 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.300+0000 7f6bba800700 1 Processor -- start 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.300+0000 7f6bba800700 1 -- start start 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.301+0000 7f6bba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.301+0000 7f6bba800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb410f280 con 0x7f6bb407edc0 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.301+0000 7f6bb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.301+0000 7f6bb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37040/0 (socket says 172.21.15.63:37040) 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.301+0000 7f6bb3400700 1 -- 172.21.15.63:0/511370710 learned_addr learned my addr 172.21.15.63:0/511370710 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.301+0000 7f6bb3400700 1 -- 172.21.15.63:0/511370710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb419f320 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.302+0000 7f6bb3400700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb419ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4011670 tx=0x7f6ba4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.302+0000 7f6babe00700 1 -- 172.21.15.63:0/511370710 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba401f070 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.302+0000 7f6babe00700 1 -- 172.21.15.63:0/511370710 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba4003b20 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.302+0000 7f6babe00700 1 -- 172.21.15.63:0/511370710 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba40195f0 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.302+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb419f5b0 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.302+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb41a22a0 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.304+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb4064bb0 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.304+0000 7f6babe00700 1 -- 172.21.15.63:0/511370710 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6ba4022070 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.304+0000 7f6babe00700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f6b9803e4c0 0x7f6b98040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.304+0000 7f6babe00700 1 -- 172.21.15.63:0/511370710 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ba40539b0 con 0x7f6bb407edc0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.305+0000 7f6bb2a00700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f6b9803e4c0 0x7f6b98040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.305+0000 7f6bb2a00700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f6b9803e4c0 0x7f6b98040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006fd0 tx=0x7f6ba0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.310+0000 7f6babe00700 1 -- 172.21.15.63:0/511370710 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ba402cd20 con 0x7f6bb407edc0 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.551+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6bb41a2850 con 0x7f6bb407edc0 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.555+0000 7f6babe00700 1 -- 172.21.15.63:0/511370710 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6ba401b440 con 0x7f6bb407edc0 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.558+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f6b9803e4c0 msgr2=0x7f6b98040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.558+0000 7f6bba800700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f6b9803e4c0 0x7f6b98040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006fd0 tx=0x7f6ba0006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.558+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 msgr2=0x7f6bb419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.558+0000 7f6bba800700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb419ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4011670 tx=0x7f6ba4011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.559+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 shutdown_connections 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.559+0000 7f6bba800700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f6b9803e4c0 0x7f6b98040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.559+0000 7f6bba800700 1 --2- 172.21.15.63:0/511370710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bb407edc0 0x7f6bb419ede0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.559+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 >> 172.21.15.63:0/511370710 conn(0x7f6bb41043c0 msgr2=0x7f6bb4104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.559+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 shutdown_connections 2024-05-11T13:18:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:25.559+0000 7f6bba800700 1 -- 172.21.15.63:0/511370710 wait complete. 2024-05-11T13:18:28.166 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:27 smithi063 ceph-mon[29809]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:18:28.167 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:27 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2924810328' entity='client.admin' 2024-05-11T13:18:28.180 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.823+0000 7f392d600700 1 Processor -- start 2024-05-11T13:18:28.180 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.823+0000 7f392d600700 1 -- start start 2024-05-11T13:18:28.180 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.824+0000 7f392d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f3928109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:28.180 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.824+0000 7f392d600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39281096c0 con 0x7f3928108d60 2024-05-11T13:18:28.180 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.824+0000 7f3926a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f3928109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:28.180 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.824+0000 7f3926a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f3928109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37050/0 (socket says 172.21.15.63:37050) 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.824+0000 7f3926a00700 1 -- 172.21.15.63:0/1159520299 learned_addr learned my addr 172.21.15.63:0/1159520299 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.825+0000 7f3926a00700 1 -- 172.21.15.63:0/1159520299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3928109840 con 0x7f3928108d60 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.825+0000 7f3926a00700 1 --2- 172.21.15.63:0/1159520299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f3928109180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39180098a0 tx=0x7f3918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d7ed658ce8ee58 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f3925600700 1 -- 172.21.15.63:0/1159520299 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918004160 con 0x7f3928108d60 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f3925600700 1 -- 172.21.15.63:0/1159520299 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3918004300 con 0x7f3928108d60 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f3925600700 1 -- 172.21.15.63:0/1159520299 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918011960 con 0x7f3928108d60 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f392d600700 1 -- 172.21.15.63:0/1159520299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 msgr2=0x7f3928109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f392d600700 1 --2- 172.21.15.63:0/1159520299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f3928109180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39180098a0 tx=0x7f3918009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f392d600700 1 -- 172.21.15.63:0/1159520299 shutdown_connections 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f392d600700 1 --2- 172.21.15.63:0/1159520299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f3928109180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f392d600700 1 -- 172.21.15.63:0/1159520299 >> 172.21.15.63:0/1159520299 conn(0x7f3928104400 msgr2=0x7f3928106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.826+0000 7f392d600700 1 -- 172.21.15.63:0/1159520299 shutdown_connections 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.827+0000 7f392d600700 1 -- 172.21.15.63:0/1159520299 wait complete. 2024-05-11T13:18:28.181 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.828+0000 7f392d600700 1 Processor -- start 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.828+0000 7f392d600700 1 -- start start 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.828+0000 7f392d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f392819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.828+0000 7f392d600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f392810ad40 con 0x7f3928108d60 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.828+0000 7f3926a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f392819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.828+0000 7f3926a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f392819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37056/0 (socket says 172.21.15.63:37056) 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.829+0000 7f3926a00700 1 -- 172.21.15.63:0/2924810328 learned_addr learned my addr 172.21.15.63:0/2924810328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.829+0000 7f3926a00700 1 -- 172.21.15.63:0/2924810328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392819d0e0 con 0x7f3928108d60 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.829+0000 7f3926a00700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f392819cba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3918004660 tx=0x7f3918011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.829+0000 7f391f400700 1 -- 172.21.15.63:0/2924810328 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918003b20 con 0x7f3928108d60 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.829+0000 7f391f400700 1 -- 172.21.15.63:0/2924810328 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f391801b8d0 con 0x7f3928108d60 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.829+0000 7f391f400700 1 -- 172.21.15.63:0/2924810328 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918023890 con 0x7f3928108d60 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.829+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392819d370 con 0x7f3928108d60 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.830+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39281a0060 con 0x7f3928108d60 2024-05-11T13:18:28.182 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.831+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3928064bb0 con 0x7f3928108d60 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.831+0000 7f391f400700 1 -- 172.21.15.63:0/2924810328 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f391801ba80 con 0x7f3928108d60 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.831+0000 7f391f400700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f390c03e4c0 0x7f390c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.831+0000 7f391f400700 1 -- 172.21.15.63:0/2924810328 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3918053e20 con 0x7f3928108d60 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.832+0000 7f3926000700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f390c03e4c0 0x7f390c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.832+0000 7f3926000700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f390c03e4c0 0x7f390c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3914006fd0 tx=0x7f3914006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:26.837+0000 7f391f400700 1 -- 172.21.15.63:0/2924810328 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f391801f030 con 0x7f3928108d60 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.075+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f39281a0230 con 0x7f3928108d60 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.081+0000 7f391f400700 1 -- 172.21.15.63:0/2924810328 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f39180292a0 con 0x7f3928108d60 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f390c03e4c0 msgr2=0x7f390c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f390c03e4c0 0x7f390c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3914006fd0 tx=0x7f3914006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 msgr2=0x7f392819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f392819cba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3918004660 tx=0x7f3918011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:28.183 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 shutdown_connections 2024-05-11T13:18:28.184 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f390c03e4c0 0x7f390c040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:28.184 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 --2- 172.21.15.63:0/2924810328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3928108d60 0x7f392819cba0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:28.184 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 >> 172.21.15.63:0/2924810328 conn(0x7f3928104400 msgr2=0x7f3928104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:28.184 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 shutdown_connections 2024-05-11T13:18:28.184 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:27.085+0000 7f392d600700 1 -- 172.21.15.63:0/2924810328 wait complete. 2024-05-11T13:18:28.184 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling the dashboard module... 2024-05-11T13:18:29.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:29 smithi063 ceph-mon[29809]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:18:29.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:29 smithi063 ceph-mon[29809]: from='mgr.14120 172.21.15.63:0/722451051' entity='mgr.x' 2024-05-11T13:18:29.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:29 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/461154902' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-11T13:18:30.085 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: ignoring --setuser ceph since I am not root 2024-05-11T13:18:30.085 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: ignoring --setgroup ceph since I am not root 2024-05-11T13:18:30.085 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:30.083+0000 7fa6a6293200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:18:30.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.830+0000 7f979f800700 1 Processor -- start 2024-05-11T13:18:30.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.830+0000 7f979f800700 1 -- start start 2024-05-11T13:18:30.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.831+0000 7f979f800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:30.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.831+0000 7f979f800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f979807f120 con 0x7f979807ed00 2024-05-11T13:18:30.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.831+0000 7f979ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979807d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.831+0000 7f979ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979807d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37070/0 (socket says 172.21.15.63:37070) 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.831+0000 7f979ce00700 1 -- 172.21.15.63:0/225056906 learned_addr learned my addr 172.21.15.63:0/225056906 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.831+0000 7f979ce00700 1 -- 172.21.15.63:0/225056906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979807d800 con 0x7f979807ed00 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.832+0000 7f979ce00700 1 --2- 172.21.15.63:0/225056906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979807d2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f5348bbec7bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.832+0000 7f9797400700 1 -- 172.21.15.63:0/225056906 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c004160 con 0x7f979807ed00 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.832+0000 7f9797400700 1 -- 172.21.15.63:0/225056906 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f978c004300 con 0x7f979807ed00 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.832+0000 7f9797400700 1 -- 172.21.15.63:0/225056906 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c011960 con 0x7f979807ed00 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.833+0000 7f979f800700 1 -- 172.21.15.63:0/225056906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 msgr2=0x7f979807d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.833+0000 7f979f800700 1 --2- 172.21.15.63:0/225056906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979807d2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.833+0000 7f979f800700 1 -- 172.21.15.63:0/225056906 shutdown_connections 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.833+0000 7f979f800700 1 --2- 172.21.15.63:0/225056906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979807d2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.833+0000 7f979f800700 1 -- 172.21.15.63:0/225056906 >> 172.21.15.63:0/225056906 conn(0x7f9798104400 msgr2=0x7f9798106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.833+0000 7f979f800700 1 -- 172.21.15.63:0/225056906 shutdown_connections 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.833+0000 7f979f800700 1 -- 172.21.15.63:0/225056906 wait complete. 2024-05-11T13:18:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.834+0000 7f979f800700 1 Processor -- start 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.835+0000 7f979f800700 1 -- start start 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.835+0000 7f979f800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.835+0000 7f979f800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f979810f1b0 con 0x7f979807ed00 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.835+0000 7f979ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.835+0000 7f979ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37076/0 (socket says 172.21.15.63:37076) 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.835+0000 7f979ce00700 1 -- 172.21.15.63:0/461154902 learned_addr learned my addr 172.21.15.63:0/461154902 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.836+0000 7f979ce00700 1 -- 172.21.15.63:0/461154902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979819f310 con 0x7f979807ed00 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.836+0000 7f979ce00700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979819edd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f978c011670 tx=0x7f978c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.836+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c01f070 con 0x7f979807ed00 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.836+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f978c003b20 con 0x7f979807ed00 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.836+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979819f5a0 con 0x7f979807ed00 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.836+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c0195f0 con 0x7f979807ed00 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.837+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97981a2290 con 0x7f979807ed00 2024-05-11T13:18:30.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.838+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9798064bb0 con 0x7f979807ed00 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.838+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f978c022070 con 0x7f979807ed00 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.839+0000 7f9795600700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f978003e470 0x7f9780040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.839+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f978c053a10 con 0x7f979807ed00 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.839+0000 7f9797e00700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f978003e470 0x7f9780040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.840+0000 7f9797e00700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f978003e470 0x7f9780040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9784006fd0 tx=0x7f9784006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:28.844+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f978c02cd00 con 0x7f979807ed00 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.153+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f97981a2840 con 0x7f979807ed00 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.762+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f978c01b440 con 0x7f979807ed00 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.766+0000 7f9795600700 1 -- 172.21.15.63:0/461154902 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f978c01baf0 con 0x7f979807ed00 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.766+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f978003e470 msgr2=0x7f9780040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.766+0000 7f979f800700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f978003e470 0x7f9780040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9784006fd0 tx=0x7f9784006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.767+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 msgr2=0x7f979819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.767+0000 7f979f800700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979819edd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f978c011670 tx=0x7f978c011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.767+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 shutdown_connections 2024-05-11T13:18:30.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.767+0000 7f979f800700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f978003e470 0x7f9780040930 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9784006fd0 tx=0x7f9784006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:30.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.767+0000 7f979f800700 1 --2- 172.21.15.63:0/461154902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f979807ed00 0x7f979819edd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:30.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.767+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 >> 172.21.15.63:0/461154902 conn(0x7f9798104400 msgr2=0x7f9798104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:30.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.768+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 shutdown_connections 2024-05-11T13:18:30.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:29.768+0000 7f979f800700 1 -- 172.21.15.63:0/461154902 wait complete. 2024-05-11T13:18:30.651 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:30.351+0000 7fa6a6293200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:18:30.651 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:30.500+0000 7fa6a6293200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:18:30.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:30 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/461154902' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-11T13:18:30.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:30 smithi063 ceph-mon[29809]: mgrmap e9: x(active, since 25s) 2024-05-11T13:18:31.312 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:30.936+0000 7fa6a6293200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:18:31.313 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:31.056+0000 7fa6a6293200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:18:32.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:31 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3687711756' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-11T13:18:32.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:18:32.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.102+0000 7ff460e00700 1 Processor -- start 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.102+0000 7ff460e00700 1 -- start start 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.102+0000 7ff460e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c10bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.102+0000 7ff460e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c07ce10 con 0x7ff45c109830 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.102+0000 7ff459e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c10bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.102+0000 7ff459e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c10bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43578/0 (socket says 172.21.15.63:43578) 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.102+0000 7ff459e00700 1 -- 172.21.15.63:0/1837005826 learned_addr learned my addr 172.21.15.63:0/1837005826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.103+0000 7ff459e00700 1 -- 172.21.15.63:0/1837005826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c10c160 con 0x7ff45c109830 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.103+0000 7ff459e00700 1 --2- 172.21.15.63:0/1837005826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c10bc20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ba305646904b78 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:32.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.103+0000 7ff458a00700 1 -- 172.21.15.63:0/1837005826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c004160 con 0x7ff45c109830 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.103+0000 7ff458a00700 1 -- 172.21.15.63:0/1837005826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff44c004300 con 0x7ff45c109830 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.103+0000 7ff458a00700 1 -- 172.21.15.63:0/1837005826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c011960 con 0x7ff45c109830 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.104+0000 7ff460e00700 1 -- 172.21.15.63:0/1837005826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 msgr2=0x7ff45c10bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.104+0000 7ff460e00700 1 --2- 172.21.15.63:0/1837005826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c10bc20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.104+0000 7ff460e00700 1 -- 172.21.15.63:0/1837005826 shutdown_connections 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.104+0000 7ff460e00700 1 --2- 172.21.15.63:0/1837005826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c10bc20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.104+0000 7ff460e00700 1 -- 172.21.15.63:0/1837005826 >> 172.21.15.63:0/1837005826 conn(0x7ff45c103200 msgr2=0x7ff45c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.104+0000 7ff460e00700 1 -- 172.21.15.63:0/1837005826 shutdown_connections 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.105+0000 7ff460e00700 1 -- 172.21.15.63:0/1837005826 wait complete. 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.106+0000 7ff460e00700 1 Processor -- start 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.106+0000 7ff460e00700 1 -- start start 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.106+0000 7ff460e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:32.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.106+0000 7ff460e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10d0a0 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.106+0000 7ff459e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.106+0000 7ff459e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43580/0 (socket says 172.21.15.63:43580) 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.106+0000 7ff459e00700 1 -- 172.21.15.63:0/3687711756 learned_addr learned my addr 172.21.15.63:0/3687711756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.107+0000 7ff459e00700 1 -- 172.21.15.63:0/3687711756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c19f2e0 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.107+0000 7ff459e00700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c19eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff44c011670 tx=0x7ff44c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.107+0000 7ff452a00700 1 -- 172.21.15.63:0/3687711756 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c003a60 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.107+0000 7ff452a00700 1 -- 172.21.15.63:0/3687711756 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff44c01b8d0 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.107+0000 7ff452a00700 1 -- 172.21.15.63:0/3687711756 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c023790 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.107+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c19f570 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.107+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a2260 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.108+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c064bb0 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.109+0000 7ff452a00700 1 -- 172.21.15.63:0/3687711756 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7ff44c01b440 con 0x7ff45c109830 2024-05-11T13:18:32.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.110+0000 7ff452a00700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 0x7ff4400409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.110+0000 7ff452a00700 1 -- 172.21.15.63:0/3687711756 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff44c053f10 con 0x7ff45c109830 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.110+0000 7ff459400700 1 -- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 msgr2=0x7ff4400409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.110+0000 7ff459400700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 0x7ff4400409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.116+0000 7ff452a00700 1 -- 172.21.15.63:0/3687711756 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff44c01f030 con 0x7ff45c109830 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.311+0000 7ff459400700 1 -- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 msgr2=0x7ff4400409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.311+0000 7ff459400700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 0x7ff4400409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.414+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff45c1a2430 con 0x7ff45c109830 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.415+0000 7ff452a00700 1 -- 172.21.15.63:0/3687711756 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff44c01bda0 con 0x7ff45c109830 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 msgr2=0x7ff4400409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 0x7ff4400409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 msgr2=0x7ff45c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c19eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff44c011670 tx=0x7ff44c011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 shutdown_connections 2024-05-11T13:18:32.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7ff44003e520 0x7ff4400409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:32.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 --2- 172.21.15.63:0/3687711756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff45c109830 0x7ff45c19eda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:32.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 >> 172.21.15.63:0/3687711756 conn(0x7ff45c103200 msgr2=0x7ff45c103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:32.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 shutdown_connections 2024-05-11T13:18:32.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:31.418+0000 7ff460e00700 1 -- 172.21.15.63:0/3687711756 wait complete. 2024-05-11T13:18:32.092 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-05-11T13:18:32.092 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 9... 2024-05-11T13:18:32.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:32.669+0000 7fa6a6293200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:18:32.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:32.904+0000 7fa6a6293200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:18:33.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:33.123+0000 7fa6a6293200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:18:33.902 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:33.519+0000 7fa6a6293200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:18:33.902 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:33.646+0000 7fa6a6293200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:18:34.165 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:33.901+0000 7fa6a6293200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:18:34.165 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:34 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:34.164+0000 7fa6a6293200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:18:35.288 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:34 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:34.990+0000 7fa6a6293200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:18:35.288 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:35.287+0000 7fa6a6293200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:18:35.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:35.860+0000 7fa6a6293200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:18:36.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:35.992+0000 7fa6a6293200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:18:36.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:36.468+0000 7fa6a6293200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:18:37.407 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:36.939+0000 7fa6a6293200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:18:37.407 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:37.068+0000 7fa6a6293200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:18:37.407 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:37.406+0000 7fa6a6293200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:18:37.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:37.593+0000 7fa6a6293200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:18:38.132 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:37.725+0000 7fa6a6293200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:18:38.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:38.132+0000 7fa6a6293200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:18:38.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:38.261+0000 7fa6a6293200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:18:39.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:39.053+0000 7fa6a6293200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:18:39.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:18:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:18:39.350+0000 7fa6a6293200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:18:39.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: Active manager daemon x restarted 2024-05-11T13:18:39.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: Activating manager daemon x 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: osdmap e3: 0 total, 0 up, 0 in 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: mgrmap e10: x(active, starting, since 0.010919s) 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T13:18:39.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:39 smithi063 ceph-mon[29809]: Manager daemon x is now available 2024-05-11T13:18:40.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:40.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T13:18:40.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T13:18:40.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:40 smithi063 ceph-mon[29809]: mgrmap e11: x(active, since 1.01288s) 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.665+0000 7f61d7c00700 1 Processor -- start 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.665+0000 7f61d7c00700 1 -- start start 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.666+0000 7f61d7c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d0107a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.666+0000 7f61d7c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61d007cdd0 con 0x7f61d0107630 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.666+0000 7f61d5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d0107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.667+0000 7f61d5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d0107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43592/0 (socket says 172.21.15.63:43592) 2024-05-11T13:18:41.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.667+0000 7f61d5200700 1 -- 172.21.15.63:0/2724256821 learned_addr learned my addr 172.21.15.63:0/2724256821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.667+0000 7f61d5200700 1 -- 172.21.15.63:0/2724256821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d0107f90 con 0x7f61d0107630 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.667+0000 7f61d5200700 1 --2- 172.21.15.63:0/2724256821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d0107a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61c40098a0 tx=0x7f61c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1252a82897f1a9a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.668+0000 7f61cf400700 1 -- 172.21.15.63:0/2724256821 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61c4004160 con 0x7f61d0107630 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.668+0000 7f61cf400700 1 -- 172.21.15.63:0/2724256821 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61c4004300 con 0x7f61d0107630 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.668+0000 7f61cf400700 1 -- 172.21.15.63:0/2724256821 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61c4011960 con 0x7f61d0107630 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.668+0000 7f61d7c00700 1 -- 172.21.15.63:0/2724256821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 msgr2=0x7f61d0107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.668+0000 7f61d7c00700 1 --2- 172.21.15.63:0/2724256821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d0107a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61c40098a0 tx=0x7f61c4009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.669+0000 7f61d7c00700 1 -- 172.21.15.63:0/2724256821 shutdown_connections 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.669+0000 7f61d7c00700 1 --2- 172.21.15.63:0/2724256821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d0107a50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.669+0000 7f61d7c00700 1 -- 172.21.15.63:0/2724256821 >> 172.21.15.63:0/2724256821 conn(0x7f61d01031f0 msgr2=0x7f61d0105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.669+0000 7f61d7c00700 1 -- 172.21.15.63:0/2724256821 shutdown_connections 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.669+0000 7f61d7c00700 1 -- 172.21.15.63:0/2724256821 wait complete. 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.670+0000 7f61d7c00700 1 Processor -- start 2024-05-11T13:18:41.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.670+0000 7f61d7c00700 1 -- start start 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.670+0000 7f61d7c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.670+0000 7f61d7c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0108fa0 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.671+0000 7f61d5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.671+0000 7f61d5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43602/0 (socket says 172.21.15.63:43602) 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.671+0000 7f61d5200700 1 -- 172.21.15.63:0/848787447 learned_addr learned my addr 172.21.15.63:0/848787447 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.671+0000 7f61d5200700 1 -- 172.21.15.63:0/848787447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d019f300 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.671+0000 7f61d5200700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d019edc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61c4003e50 tx=0x7f61c4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.672+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61c4003a60 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.672+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61c401b8d0 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.672+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61c40237d0 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.672+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d019f590 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.672+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d01a2280 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.673+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f61c401b440 con 0x7f61d0107630 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.674+0000 7f61cd600700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.674+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 --> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f61b8041040 con 0x7f61b803e470 2024-05-11T13:18:41.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.674+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f61c40540d0 con 0x7f61d0107630 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.677+0000 7f61cfe00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 msgr2=0x7f61b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.677+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.878+0000 7f61cfe00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 msgr2=0x7f61b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:32.878+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:33.279+0000 7f61cfe00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 msgr2=0x7f61b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:33.279+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:34.080+0000 7f61cfe00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 msgr2=0x7f61b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:34.080+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:35.682+0000 7f61cfe00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 msgr2=0x7f61b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:35.682+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:38.885+0000 7f61cfe00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 msgr2=0x7f61b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/405023443 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:38.886+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:39.362+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f61c400b3e0 con 0x7f61d0107630 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:39.362+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 msgr2=0x7f61b8040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:18:41.044 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:39.362+0000 7f61cd600700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.363+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f61c40561b0 con 0x7f61d0107630 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.363+0000 7f61cd600700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f61b8041ef0 0x7f61b80442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.363+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f61c401acb0 con 0x7f61b8041ef0 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.366+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f61b8041ef0 0x7f61b80442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.368+0000 7f61cfe00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f61b8041ef0 0x7f61b80442e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0039c0 tx=0x7f61bc009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.369+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f61c401acb0 con 0x7f61b8041ef0 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.385+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f61d006a630 con 0x7f61b8041ef0 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.385+0000 7f61cd600700 1 -- 172.21.15.63:0/848787447 <== mgr.14152 v2:172.21.15.63:6800/1742389159 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f61d006a630 con 0x7f61b8041ef0 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f61b8041ef0 msgr2=0x7f61b80442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f61b8041ef0 0x7f61b80442e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0039c0 tx=0x7f61bc009270 comp rx=0 tx=0).stop 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 msgr2=0x7f61d019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d019edc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61c4003e50 tx=0x7f61c4011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 shutdown_connections 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f61b8041ef0 0x7f61b80442e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0107630 0x7f61d019edc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:41.045 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 --2- 172.21.15.63:0/848787447 >> [v2:172.21.15.63:6800/405023443,v1:172.21.15.63:6801/405023443] conn(0x7f61b803e470 0x7f61b8040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:41.046 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 >> 172.21.15.63:0/848787447 conn(0x7f61d01031f0 msgr2=0x7f61d0195a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:41.046 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 shutdown_connections 2024-05-11T13:18:41.046 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:40.386+0000 7f61d7c00700 1 -- 172.21.15.63:0/848787447 wait complete. 2024-05-11T13:18:41.046 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 9 is available 2024-05-11T13:18:41.046 INFO:teuthology.orchestra.run.smithi063.stdout:Generating a dashboard self-signed certificate... 2024-05-11T13:18:41.596 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:41 smithi063 ceph-mon[29809]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-11T13:18:41.597 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:41 smithi063 ceph-mon[29809]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-11T13:18:42.414 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: [11/May/2024:13:18:41] ENGINE Bus STARTING 2024-05-11T13:18:42.414 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: [11/May/2024:13:18:41] ENGINE Serving on https://172.21.15.63:7150 2024-05-11T13:18:42.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: [11/May/2024:13:18:41] ENGINE Serving on http://172.21.15.63:8765 2024-05-11T13:18:42.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: [11/May/2024:13:18:41] ENGINE Bus STARTED 2024-05-11T13:18:42.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: mgrmap e12: x(active, since 2s) 2024-05-11T13:18:42.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:42.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:42.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:43.031 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.902+0000 7fa283e00700 1 Processor -- start 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.902+0000 7fa283e00700 1 -- start start 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.902+0000 7fa283e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28407d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.902+0000 7fa283e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa28407d860 con 0x7fa28407ed60 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.903+0000 7fa282a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28407d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.903+0000 7fa282a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28407d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58342/0 (socket says 172.21.15.63:58342) 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.903+0000 7fa282a00700 1 -- 172.21.15.63:0/1212924879 learned_addr learned my addr 172.21.15.63:0/1212924879 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.903+0000 7fa282a00700 1 -- 172.21.15.63:0/1212924879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28407d9e0 con 0x7fa28407ed60 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.904+0000 7fa282a00700 1 --2- 172.21.15.63:0/1212924879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28407d320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2740098a0 tx=0x7fa274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515e8464e3134e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:43.032 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.904+0000 7fa282000700 1 -- 172.21.15.63:0/1212924879 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa274004160 con 0x7fa28407ed60 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.904+0000 7fa282000700 1 -- 172.21.15.63:0/1212924879 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa274004300 con 0x7fa28407ed60 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.904+0000 7fa282000700 1 -- 172.21.15.63:0/1212924879 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa274011960 con 0x7fa28407ed60 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.905+0000 7fa283e00700 1 -- 172.21.15.63:0/1212924879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 msgr2=0x7fa28407d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.905+0000 7fa283e00700 1 --2- 172.21.15.63:0/1212924879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28407d320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2740098a0 tx=0x7fa274009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.905+0000 7fa283e00700 1 -- 172.21.15.63:0/1212924879 shutdown_connections 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.905+0000 7fa283e00700 1 --2- 172.21.15.63:0/1212924879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28407d320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.905+0000 7fa283e00700 1 -- 172.21.15.63:0/1212924879 >> 172.21.15.63:0/1212924879 conn(0x7fa284104240 msgr2=0x7fa284106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.905+0000 7fa283e00700 1 -- 172.21.15.63:0/1212924879 shutdown_connections 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.905+0000 7fa283e00700 1 -- 172.21.15.63:0/1212924879 wait complete. 2024-05-11T13:18:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.906+0000 7fa283e00700 1 Processor -- start 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.906+0000 7fa283e00700 1 -- start start 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.907+0000 7fa283e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.907+0000 7fa283e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410eff0 con 0x7fa28407ed60 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.907+0000 7fa282a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.907+0000 7fa282a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58352/0 (socket says 172.21.15.63:58352) 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.907+0000 7fa282a00700 1 -- 172.21.15.63:0/3042323295 learned_addr learned my addr 172.21.15.63:0/3042323295 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.907+0000 7fa282a00700 1 -- 172.21.15.63:0/3042323295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28419f0d0 con 0x7fa28407ed60 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.908+0000 7fa282a00700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28419eb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa274011670 tx=0x7fa274011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.908+0000 7fa27b400700 1 -- 172.21.15.63:0/3042323295 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa274003a60 con 0x7fa28407ed60 2024-05-11T13:18:43.034 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.908+0000 7fa27b400700 1 -- 172.21.15.63:0/3042323295 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa27401b8d0 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.908+0000 7fa27b400700 1 -- 172.21.15.63:0/3042323295 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa274023790 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.908+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28419f360 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.908+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2841a2050 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.909+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28406a630 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.910+0000 7fa27b400700 1 -- 172.21.15.63:0/3042323295 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fa27401b440 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.911+0000 7fa27b400700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa26803e520 0x7fa2680409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.911+0000 7fa27b400700 1 -- 172.21.15.63:0/3042323295 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa274054560 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.911+0000 7fa27be00700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa26803e520 0x7fa2680409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.912+0000 7fa27be00700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa26803e520 0x7fa2680409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006fd0 tx=0x7fa26c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:41.916+0000 7fa27b400700 1 -- 172.21.15.63:0/3042323295 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa27401f030 con 0x7fa28407ed60 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.163+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fa2841053f0 con 0x7fa26803e520 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.300+0000 7fa27b400700 1 -- 172.21.15.63:0/3042323295 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fa2841053f0 con 0x7fa26803e520 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa26803e520 msgr2=0x7fa2680409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:43.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa26803e520 0x7fa2680409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006fd0 tx=0x7fa26c006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 msgr2=0x7fa28419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28419eb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa274011670 tx=0x7fa274011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 shutdown_connections 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa26803e520 0x7fa2680409e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 --2- 172.21.15.63:0/3042323295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa28407ed60 0x7fa28419eb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 >> 172.21.15.63:0/3042323295 conn(0x7fa284104240 msgr2=0x7fa284104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 shutdown_connections 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:42.303+0000 7fa283e00700 1 -- 172.21.15.63:0/3042323295 wait complete. 2024-05-11T13:18:43.036 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial admin user... 2024-05-11T13:18:43.588 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:43 smithi063 ceph-mon[29809]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:44.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:44 smithi063 ceph-mon[29809]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:44.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:44 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:44.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$uYpmE7x9e27o0H8gfbgl0ebqfN1fK9H8nj74PwSMIrG4EDyr/Dj8e", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715433524, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-11T13:18:44.807 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.587+0000 7fe514000700 1 Processor -- start 2024-05-11T13:18:44.807 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.588+0000 7fe514000700 1 -- start start 2024-05-11T13:18:44.807 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.588+0000 7fe514000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:44.807 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.588+0000 7fe514000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c108630 con 0x7fe50c107c40 2024-05-11T13:18:44.807 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.589+0000 7fe511600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.589+0000 7fe511600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58354/0 (socket says 172.21.15.63:58354) 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.589+0000 7fe511600700 1 -- 172.21.15.63:0/103750263 learned_addr learned my addr 172.21.15.63:0/103750263 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.589+0000 7fe511600700 1 -- 172.21.15.63:0/103750263 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c108e40 con 0x7fe50c107c40 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.590+0000 7fe511600700 1 --2- 172.21.15.63:0/103750263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c108060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe50000bb40 tx=0x7fe50000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb260953cff8d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.590+0000 7fe50be00700 1 -- 172.21.15.63:0/103750263 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe50000dbe0 con 0x7fe50c107c40 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.590+0000 7fe50be00700 1 -- 172.21.15.63:0/103750263 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5000044c0 con 0x7fe50c107c40 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.590+0000 7fe50be00700 1 -- 172.21.15.63:0/103750263 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe500003e20 con 0x7fe50c107c40 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.591+0000 7fe514000700 1 -- 172.21.15.63:0/103750263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 msgr2=0x7fe50c108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.591+0000 7fe514000700 1 --2- 172.21.15.63:0/103750263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c108060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe50000bb40 tx=0x7fe50000be50 comp rx=0 tx=0).stop 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.591+0000 7fe514000700 1 -- 172.21.15.63:0/103750263 shutdown_connections 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.591+0000 7fe514000700 1 --2- 172.21.15.63:0/103750263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c108060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:44.808 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.591+0000 7fe514000700 1 -- 172.21.15.63:0/103750263 >> 172.21.15.63:0/103750263 conn(0x7fe50c103200 msgr2=0x7fe50c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.591+0000 7fe514000700 1 -- 172.21.15.63:0/103750263 shutdown_connections 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.591+0000 7fe514000700 1 -- 172.21.15.63:0/103750263 wait complete. 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.592+0000 7fe514000700 1 Processor -- start 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.592+0000 7fe514000700 1 -- start start 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.593+0000 7fe514000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.593+0000 7fe514000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c07bfb0 con 0x7fe50c107c40 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.593+0000 7fe511600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.593+0000 7fe511600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58368/0 (socket says 172.21.15.63:58368) 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.593+0000 7fe511600700 1 -- 172.21.15.63:0/1219458473 learned_addr learned my addr 172.21.15.63:0/1219458473 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.593+0000 7fe511600700 1 -- 172.21.15.63:0/1219458473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c19f380 con 0x7fe50c107c40 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.594+0000 7fe511600700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c19ee40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe50000b340 tx=0x7fe500003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.594+0000 7fe50a000700 1 -- 172.21.15.63:0/1219458473 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe500010500 con 0x7fe50c107c40 2024-05-11T13:18:44.809 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.594+0000 7fe50a000700 1 -- 172.21.15.63:0/1219458473 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe50001d070 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.594+0000 7fe50a000700 1 -- 172.21.15.63:0/1219458473 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe500021900 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.594+0000 7fe514000700 1 -- 172.21.15.63:0/1219458473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c19f610 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.594+0000 7fe514000700 1 -- 172.21.15.63:0/1219458473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c1a2300 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.596+0000 7fe50a000700 1 -- 172.21.15.63:0/1219458473 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fe50001f030 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.596+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e8005330 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.596+0000 7fe50a000700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe4f403e510 0x7fe4f40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.596+0000 7fe50a000700 1 -- 172.21.15.63:0/1219458473 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe500053930 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.597+0000 7fe510c00700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe4f403e510 0x7fe4f40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.597+0000 7fe510c00700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe4f403e510 0x7fe4f40409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.606+0000 7fe50a000700 1 -- 172.21.15.63:0/1219458473 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe500015070 con 0x7fe50c107c40 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:43.848+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe4e8002400 con 0x7fe4f403e510 2024-05-11T13:18:44.810 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.099+0000 7fe50a000700 1 -- 172.21.15.63:0/1219458473 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe4e8002400 con 0x7fe4f403e510 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe4f403e510 msgr2=0x7fe4f40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe4f403e510 0x7fe4f40409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 msgr2=0x7fe50c19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c19ee40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe50000b340 tx=0x7fe500003f20 comp rx=0 tx=0).stop 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 shutdown_connections 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe4f403e510 0x7fe4f40409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 --2- 172.21.15.63:0/1219458473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe50c107c40 0x7fe50c19ee40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 >> 172.21.15.63:0/1219458473 conn(0x7fe50c103200 msgr2=0x7fe50c195490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.102+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 shutdown_connections 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:44.103+0000 7fe4ff400700 1 -- 172.21.15.63:0/1219458473 wait complete. 2024-05-11T13:18:44.811 INFO:teuthology.orchestra.run.smithi063.stdout:Fetching dashboard port number... 2024-05-11T13:18:46.301 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:46.301 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:46 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1680572775' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 8443 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.372+0000 7f82eb200700 1 Processor -- start 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.373+0000 7f82eb200700 1 -- start start 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.373+0000 7f82eb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.373+0000 7f82eb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec07ce90 con 0x7f82ec1073a0 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.373+0000 7f82e9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.374+0000 7f82e9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58384/0 (socket says 172.21.15.63:58384) 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.374+0000 7f82e9e00700 1 -- 172.21.15.63:0/2743200981 learned_addr learned my addr 172.21.15.63:0/2743200981 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.374+0000 7f82e9e00700 1 -- 172.21.15.63:0/2743200981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec109d10 con 0x7f82ec1073a0 2024-05-11T13:18:46.318 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.374+0000 7f82e9e00700 1 --2- 172.21.15.63:0/2743200981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1097d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0098a0 tx=0x7f82dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ab65b5e699967d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.375+0000 7f82e8a00700 1 -- 172.21.15.63:0/2743200981 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82dc004160 con 0x7f82ec1073a0 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.375+0000 7f82e8a00700 1 -- 172.21.15.63:0/2743200981 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82dc004300 con 0x7f82ec1073a0 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.375+0000 7f82e8a00700 1 -- 172.21.15.63:0/2743200981 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82dc011960 con 0x7f82ec1073a0 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.375+0000 7f82eb200700 1 -- 172.21.15.63:0/2743200981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 msgr2=0x7f82ec1097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.375+0000 7f82eb200700 1 --2- 172.21.15.63:0/2743200981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1097d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0098a0 tx=0x7f82dc009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.376+0000 7f82eb200700 1 -- 172.21.15.63:0/2743200981 shutdown_connections 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.376+0000 7f82eb200700 1 --2- 172.21.15.63:0/2743200981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1097d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.376+0000 7f82eb200700 1 -- 172.21.15.63:0/2743200981 >> 172.21.15.63:0/2743200981 conn(0x7f82ec102fe0 msgr2=0x7f82ec105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.376+0000 7f82eb200700 1 -- 172.21.15.63:0/2743200981 shutdown_connections 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.376+0000 7f82eb200700 1 -- 172.21.15.63:0/2743200981 wait complete. 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.377+0000 7f82eb200700 1 Processor -- start 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.377+0000 7f82eb200700 1 -- start start 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.377+0000 7f82eb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.377+0000 7f82eb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec10ad20 con 0x7f82ec1073a0 2024-05-11T13:18:46.319 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.378+0000 7f82e9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.378+0000 7f82e9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58398/0 (socket says 172.21.15.63:58398) 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.378+0000 7f82e9e00700 1 -- 172.21.15.63:0/1680572775 learned_addr learned my addr 172.21.15.63:0/1680572775 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.378+0000 7f82e9e00700 1 -- 172.21.15.63:0/1680572775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec1a34d0 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.378+0000 7f82e9e00700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1a2f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82dc011670 tx=0x7f82dc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.379+0000 7f82e2a00700 1 -- 172.21.15.63:0/1680572775 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82dc003a60 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.379+0000 7f82e2a00700 1 -- 172.21.15.63:0/1680572775 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82dc01b8d0 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.379+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec1a3760 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.379+0000 7f82e2a00700 1 -- 172.21.15.63:0/1680572775 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82dc023790 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.379+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec1a6450 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.380+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ec06a630 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.380+0000 7f82e2a00700 1 -- 172.21.15.63:0/1680572775 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f82dc01b440 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.381+0000 7f82e2a00700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82d003e520 0x7f82d00409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.381+0000 7f82e2a00700 1 -- 172.21.15.63:0/1680572775 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f82dc054120 con 0x7f82ec1073a0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.381+0000 7f82e9400700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82d003e520 0x7f82d00409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:46.320 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.382+0000 7f82e9400700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82d003e520 0x7f82d00409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f82d8006fd0 tx=0x7f82d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.388+0000 7f82e2a00700 1 -- 172.21.15.63:0/1680572775 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82dc01f030 con 0x7f82ec1073a0 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.624+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f82ec02e7c0 con 0x7f82ec1073a0 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.625+0000 7f82e2a00700 1 -- 172.21.15.63:0/1680572775 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f82dc016260 con 0x7f82ec1073a0 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82d003e520 msgr2=0x7f82d00409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82d003e520 0x7f82d00409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f82d8006fd0 tx=0x7f82d8006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 msgr2=0x7f82ec1a2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1a2f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82dc011670 tx=0x7f82dc011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 shutdown_connections 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82d003e520 0x7f82d00409e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 --2- 172.21.15.63:0/1680572775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82ec1073a0 0x7f82ec1a2f90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.628+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 >> 172.21.15.63:0/1680572775 conn(0x7f82ec102fe0 msgr2=0x7f82ec105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.629+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 shutdown_connections 2024-05-11T13:18:46.321 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:45.629+0000 7f82eb200700 1 -- 172.21.15.63:0/1680572775 wait complete. 2024-05-11T13:18:46.343 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T13:18:46.343 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-11T13:18:46.364 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T13:18:46.364 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-11T13:18:46.364 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-11T13:18:46.364 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-11T13:18:46.375 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph Dashboard is now available at: 2024-05-11T13:18:46.375 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:46.375 INFO:teuthology.orchestra.run.smithi063.stdout: URL: https://smithi063.front.sepia.ceph.com:8443/ 2024-05-11T13:18:46.375 INFO:teuthology.orchestra.run.smithi063.stdout: User: admin 2024-05-11T13:18:46.375 INFO:teuthology.orchestra.run.smithi063.stdout: Password: znjpq192hr 2024-05-11T13:18:46.375 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:46.377 INFO:teuthology.orchestra.run.smithi063.stdout:Saving cluster configuration to /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config directory 2024-05-11T13:18:46.380 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling autotune for osd_memory_target 2024-05-11T13:18:47.224 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:47 smithi063 ceph-mon[29809]: mgrmap e13: x(active, since 6s) 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.011+0000 7f0b47400700 1 Processor -- start 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.012+0000 7f0b47400700 1 -- start start 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.012+0000 7f0b47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.012+0000 7f0b47400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401125e0 con 0x7f0b4010fcb0 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.012+0000 7f0b44a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.013+0000 7f0b44a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58412/0 (socket says 172.21.15.63:58412) 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.013+0000 7f0b44a00700 1 -- 172.21.15.63:0/3563971684 learned_addr learned my addr 172.21.15.63:0/3563971684 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.013+0000 7f0b44a00700 1 -- 172.21.15.63:0/3563971684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b40112760 con 0x7f0b4010fcb0 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.013+0000 7f0b44a00700 1 --2- 172.21.15.63:0/3563971684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401120a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b340098a0 tx=0x7f0b34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c06560078222d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:48.063 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.014+0000 7f0b3f400700 1 -- 172.21.15.63:0/3563971684 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34004160 con 0x7f0b4010fcb0 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.014+0000 7f0b3f400700 1 -- 172.21.15.63:0/3563971684 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b34004300 con 0x7f0b4010fcb0 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.014+0000 7f0b3f400700 1 -- 172.21.15.63:0/3563971684 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34011960 con 0x7f0b4010fcb0 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.014+0000 7f0b47400700 1 -- 172.21.15.63:0/3563971684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 msgr2=0x7f0b401120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.014+0000 7f0b47400700 1 --2- 172.21.15.63:0/3563971684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401120a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b340098a0 tx=0x7f0b34009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.014+0000 7f0b47400700 1 -- 172.21.15.63:0/3563971684 shutdown_connections 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.014+0000 7f0b47400700 1 --2- 172.21.15.63:0/3563971684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401120a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.015+0000 7f0b47400700 1 -- 172.21.15.63:0/3563971684 >> 172.21.15.63:0/3563971684 conn(0x7f0b4007e420 msgr2=0x7f0b40080840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.015+0000 7f0b47400700 1 -- 172.21.15.63:0/3563971684 shutdown_connections 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.015+0000 7f0b47400700 1 -- 172.21.15.63:0/3563971684 wait complete. 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.016+0000 7f0b47400700 1 Processor -- start 2024-05-11T13:18:48.068 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.016+0000 7f0b47400700 1 -- start start 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.016+0000 7f0b47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.016+0000 7f0b47400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401136a0 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.016+0000 7f0b44a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.016+0000 7f0b44a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58422/0 (socket says 172.21.15.63:58422) 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.016+0000 7f0b44a00700 1 -- 172.21.15.63:0/3254578649 learned_addr learned my addr 172.21.15.63:0/3254578649 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.017+0000 7f0b44a00700 1 -- 172.21.15.63:0/3254578649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b401a3680 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.017+0000 7f0b44a00700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401a3140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b34003e50 tx=0x7f0b34011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.017+0000 7f0b3d600700 1 -- 172.21.15.63:0/3254578649 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34003a60 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.017+0000 7f0b3d600700 1 -- 172.21.15.63:0/3254578649 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b3401b8d0 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.017+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b401a3910 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.017+0000 7f0b3d600700 1 -- 172.21.15.63:0/3254578649 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b340237d0 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.018+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b401a6600 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.019+0000 7f0b3d600700 1 -- 172.21.15.63:0/3254578649 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0b3401b440 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.019+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b40064bb0 con 0x7f0b4010fcb0 2024-05-11T13:18:48.069 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.019+0000 7f0b3d600700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0b2803e510 0x7f0b280409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.019+0000 7f0b3d600700 1 -- 172.21.15.63:0/3254578649 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0b34054350 con 0x7f0b4010fcb0 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.020+0000 7f0b3fe00700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0b2803e510 0x7f0b280409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.020+0000 7f0b3fe00700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0b2803e510 0x7f0b280409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.025+0000 7f0b3d600700 1 -- 172.21.15.63:0/3254578649 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b3401f030 con 0x7f0b4010fcb0 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.276+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f0b401a6c20 con 0x7f0b4010fcb0 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.276+0000 7f0b3d600700 1 -- 172.21.15.63:0/3254578649 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f0b34017e90 con 0x7f0b4010fcb0 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.279+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0b2803e510 msgr2=0x7f0b280409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.279+0000 7f0b47400700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0b2803e510 0x7f0b280409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.279+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 msgr2=0x7f0b401a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.279+0000 7f0b47400700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401a3140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b34003e50 tx=0x7f0b34011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.280+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 shutdown_connections 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.280+0000 7f0b47400700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0b2803e510 0x7f0b280409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:48.070 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.280+0000 7f0b47400700 1 --2- 172.21.15.63:0/3254578649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4010fcb0 0x7f0b401a3140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:48.071 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.280+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 >> 172.21.15.63:0/3254578649 conn(0x7f0b4007e420 msgr2=0x7f0b4007ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:48.071 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.280+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 shutdown_connections 2024-05-11T13:18:48.071 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:47.280+0000 7f0b47400700 1 -- 172.21.15.63:0/3254578649 wait complete. 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.755+0000 7f8b1ee00700 1 Processor -- start 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.756+0000 7f8b1ee00700 1 -- start start 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.756+0000 7f8b1ee00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.756+0000 7f8b1ee00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1810b8e0 con 0x7f8b1810af80 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.756+0000 7f8b17e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.757+0000 7f8b17e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1810b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58436/0 (socket says 172.21.15.63:58436) 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.757+0000 7f8b17e00700 1 -- 172.21.15.63:0/4123427051 learned_addr learned my addr 172.21.15.63:0/4123427051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.757+0000 7f8b17e00700 1 -- 172.21.15.63:0/4123427051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1810c090 con 0x7f8b1810af80 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.758+0000 7f8b17e00700 1 --2- 172.21.15.63:0/4123427051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1810b3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b080098a0 tx=0x7f8b08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32794bb2a8cf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:49.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.758+0000 7f8b16a00700 1 -- 172.21.15.63:0/4123427051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b08004160 con 0x7f8b1810af80 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.758+0000 7f8b16a00700 1 -- 172.21.15.63:0/4123427051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b08004300 con 0x7f8b1810af80 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.758+0000 7f8b16a00700 1 -- 172.21.15.63:0/4123427051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b08011960 con 0x7f8b1810af80 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.759+0000 7f8b1ee00700 1 -- 172.21.15.63:0/4123427051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 msgr2=0x7f8b1810b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.759+0000 7f8b1ee00700 1 --2- 172.21.15.63:0/4123427051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1810b3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b080098a0 tx=0x7f8b08009bb0 comp rx=0 tx=0).stop 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.759+0000 7f8b1ee00700 1 -- 172.21.15.63:0/4123427051 shutdown_connections 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.759+0000 7f8b1ee00700 1 --2- 172.21.15.63:0/4123427051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1810b3a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.759+0000 7f8b1ee00700 1 -- 172.21.15.63:0/4123427051 >> 172.21.15.63:0/4123427051 conn(0x7f8b1807e420 msgr2=0x7f8b1807e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.759+0000 7f8b1ee00700 1 -- 172.21.15.63:0/4123427051 shutdown_connections 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.759+0000 7f8b1ee00700 1 -- 172.21.15.63:0/4123427051 wait complete. 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.760+0000 7f8b1ee00700 1 Processor -- start 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.760+0000 7f8b1ee00700 1 -- start start 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.760+0000 7f8b1ee00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.760+0000 7f8b1ee00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1810f350 con 0x7f8b1810af80 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.761+0000 7f8b17e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:49.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.761+0000 7f8b17e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58452/0 (socket says 172.21.15.63:58452) 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.761+0000 7f8b17e00700 1 -- 172.21.15.63:0/802575059 learned_addr learned my addr 172.21.15.63:0/802575059 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.761+0000 7f8b17e00700 1 -- 172.21.15.63:0/802575059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1819f480 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.761+0000 7f8b17e00700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1819ef40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b08011670 tx=0x7f8b08011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.762+0000 7f8b14c00700 1 -- 172.21.15.63:0/802575059 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b08003a60 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.762+0000 7f8b14c00700 1 -- 172.21.15.63:0/802575059 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b0801b8d0 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.762+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1819f710 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.762+0000 7f8b14c00700 1 -- 172.21.15.63:0/802575059 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b08023790 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.762+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b181a23b0 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.763+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b18064bb0 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.763+0000 7f8b14c00700 1 -- 172.21.15.63:0/802575059 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8b0801b440 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.764+0000 7f8b14c00700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8afc03e5c0 0x7f8afc040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.764+0000 7f8b14c00700 1 -- 172.21.15.63:0/802575059 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8b080541a0 con 0x7f8b1810af80 2024-05-11T13:18:49.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.764+0000 7f8b17400700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8afc03e5c0 0x7f8afc040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.765+0000 7f8b17400700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8afc03e5c0 0x7f8afc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006fd0 tx=0x7f8b04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:48.769+0000 7f8b14c00700 1 -- 172.21.15.63:0/802575059 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b0801f030 con 0x7f8b1810af80 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.133+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8b1810f4d0 con 0x7f8b1810af80 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.139+0000 7f8b14c00700 1 -- 172.21.15.63:0/802575059 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f8b080239f0 con 0x7f8b1810af80 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.142+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8afc03e5c0 msgr2=0x7f8afc040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.142+0000 7f8b1ee00700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8afc03e5c0 0x7f8afc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006fd0 tx=0x7f8b04006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 msgr2=0x7f8b1819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1819ef40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b08011670 tx=0x7f8b08011a30 comp rx=0 tx=0).stop 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 shutdown_connections 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8afc03e5c0 0x7f8afc040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 --2- 172.21.15.63:0/802575059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b1810af80 0x7f8b1819ef40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 >> 172.21.15.63:0/802575059 conn(0x7f8b1807e420 msgr2=0x7f8b181083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 shutdown_connections 2024-05-11T13:18:49.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-11T13:18:49.143+0000 7f8b1ee00700 1 -- 172.21.15.63:0/802575059 wait complete. 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout:Or, if you are only running a single cluster on this host: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: ceph telemetry on 2024-05-11T13:18:49.736 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.737 INFO:teuthology.orchestra.run.smithi063.stdout:For more information see: 2024-05-11T13:18:49.737 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.737 INFO:teuthology.orchestra.run.smithi063.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-11T13:18:49.737 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:18:49.737 INFO:teuthology.orchestra.run.smithi063.stdout:Bootstrap complete. 2024-05-11T13:18:49.819 INFO:tasks.cephadm:Fetching config... 2024-05-11T13:18:49.820 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:18:49.820 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-11T13:18:49.921 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-11T13:18:49.921 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:18:49.922 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-11T13:18:49.970 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-11T13:18:49.971 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:18:49.971 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-11T13:18:50.035 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-11T13:18:50.035 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:18:50.035 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-11T13:18:50.129 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-11T13:18:50.130 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8VnGcJKjYdsCmidM2pzJgZUDHyBEtefelvRHNLi3PUiwDnBT3kRBzYFCZNSzRbBjckeG1CyygLvU7U5YaXivjjnj5gu4VLvgqDM1S0aKnWvOppVQGu3OllGfz+MV6L4dmBdhH4r/1KZT56DdGz747sRknhWdKfvDkjX8YZBSxKuZC+qOiCu8kBCOev93mVnvLP8eF/tEIwo87pMo8W2b+BFzsRN0R/5TZ+dY8Ftl/s98cUh6iR8sQL0B9U8yYTwyGCgiFjz7hMC9JGVnXdhEU8MPgr/obJH0C8CKhBXZmqWVpzWg2VaQCnmd2X7EDxYFxa4lVu6FuuaI0zByLlMNnv5QnfSMqSdz2LGk2230CZUh6pNB3Y2I+CkeTpi8HKRATdGsJdLgez+nKJIYcOiG4NB2wF9I8oq4fOx1xpxNcUnHbuhs/YJxNaoQZpVElNAL0M5kx6ISFfbvDw8OI7fnFFZOP2acbwalAnDxznw81M9aq6FAJxhekOft0yV32Ac= ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-11T13:18:50.254 INFO:teuthology.orchestra.run.smithi063.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8VnGcJKjYdsCmidM2pzJgZUDHyBEtefelvRHNLi3PUiwDnBT3kRBzYFCZNSzRbBjckeG1CyygLvU7U5YaXivjjnj5gu4VLvgqDM1S0aKnWvOppVQGu3OllGfz+MV6L4dmBdhH4r/1KZT56DdGz747sRknhWdKfvDkjX8YZBSxKuZC+qOiCu8kBCOev93mVnvLP8eF/tEIwo87pMo8W2b+BFzsRN0R/5TZ+dY8Ftl/s98cUh6iR8sQL0B9U8yYTwyGCgiFjz7hMC9JGVnXdhEU8MPgr/obJH0C8CKhBXZmqWVpzWg2VaQCnmd2X7EDxYFxa4lVu6FuuaI0zByLlMNnv5QnfSMqSdz2LGk2230CZUh6pNB3Y2I+CkeTpi8HKRATdGsJdLgez+nKJIYcOiG4NB2wF9I8oq4fOx1xpxNcUnHbuhs/YJxNaoQZpVElNAL0M5kx6ISFfbvDw8OI7fnFFZOP2acbwalAnDxznw81M9aq6FAJxhekOft0yV32Ac= ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:18:50.277 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:50 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/802575059' entity='client.admin' 2024-05-11T13:18:50.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8VnGcJKjYdsCmidM2pzJgZUDHyBEtefelvRHNLi3PUiwDnBT3kRBzYFCZNSzRbBjckeG1CyygLvU7U5YaXivjjnj5gu4VLvgqDM1S0aKnWvOppVQGu3OllGfz+MV6L4dmBdhH4r/1KZT56DdGz747sRknhWdKfvDkjX8YZBSxKuZC+qOiCu8kBCOev93mVnvLP8eF/tEIwo87pMo8W2b+BFzsRN0R/5TZ+dY8Ftl/s98cUh6iR8sQL0B9U8yYTwyGCgiFjz7hMC9JGVnXdhEU8MPgr/obJH0C8CKhBXZmqWVpzWg2VaQCnmd2X7EDxYFxa4lVu6FuuaI0zByLlMNnv5QnfSMqSdz2LGk2230CZUh6pNB3Y2I+CkeTpi8HKRATdGsJdLgez+nKJIYcOiG4NB2wF9I8oq4fOx1xpxNcUnHbuhs/YJxNaoQZpVElNAL0M5kx6ISFfbvDw8OI7fnFFZOP2acbwalAnDxznw81M9aq6FAJxhekOft0yV32Ac= ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-11T13:18:50.405 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8VnGcJKjYdsCmidM2pzJgZUDHyBEtefelvRHNLi3PUiwDnBT3kRBzYFCZNSzRbBjckeG1CyygLvU7U5YaXivjjnj5gu4VLvgqDM1S0aKnWvOppVQGu3OllGfz+MV6L4dmBdhH4r/1KZT56DdGz747sRknhWdKfvDkjX8YZBSxKuZC+qOiCu8kBCOev93mVnvLP8eF/tEIwo87pMo8W2b+BFzsRN0R/5TZ+dY8Ftl/s98cUh6iR8sQL0B9U8yYTwyGCgiFjz7hMC9JGVnXdhEU8MPgr/obJH0C8CKhBXZmqWVpzWg2VaQCnmd2X7EDxYFxa4lVu6FuuaI0zByLlMNnv5QnfSMqSdz2LGk2230CZUh6pNB3Y2I+CkeTpi8HKRATdGsJdLgez+nKJIYcOiG4NB2wF9I8oq4fOx1xpxNcUnHbuhs/YJxNaoQZpVElNAL0M5kx6ISFfbvDw8OI7fnFFZOP2acbwalAnDxznw81M9aq6FAJxhekOft0yV32Ac= ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:18:50.455 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8VnGcJKjYdsCmidM2pzJgZUDHyBEtefelvRHNLi3PUiwDnBT3kRBzYFCZNSzRbBjckeG1CyygLvU7U5YaXivjjnj5gu4VLvgqDM1S0aKnWvOppVQGu3OllGfz+MV6L4dmBdhH4r/1KZT56DdGz747sRknhWdKfvDkjX8YZBSxKuZC+qOiCu8kBCOev93mVnvLP8eF/tEIwo87pMo8W2b+BFzsRN0R/5TZ+dY8Ftl/s98cUh6iR8sQL0B9U8yYTwyGCgiFjz7hMC9JGVnXdhEU8MPgr/obJH0C8CKhBXZmqWVpzWg2VaQCnmd2X7EDxYFxa4lVu6FuuaI0zByLlMNnv5QnfSMqSdz2LGk2230CZUh6pNB3Y2I+CkeTpi8HKRATdGsJdLgez+nKJIYcOiG4NB2wF9I8oq4fOx1xpxNcUnHbuhs/YJxNaoQZpVElNAL0M5kx6ISFfbvDw8OI7fnFFZOP2acbwalAnDxznw81M9aq6FAJxhekOft0yV32Ac= ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-11T13:18:50.572 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8VnGcJKjYdsCmidM2pzJgZUDHyBEtefelvRHNLi3PUiwDnBT3kRBzYFCZNSzRbBjckeG1CyygLvU7U5YaXivjjnj5gu4VLvgqDM1S0aKnWvOppVQGu3OllGfz+MV6L4dmBdhH4r/1KZT56DdGz747sRknhWdKfvDkjX8YZBSxKuZC+qOiCu8kBCOev93mVnvLP8eF/tEIwo87pMo8W2b+BFzsRN0R/5TZ+dY8Ftl/s98cUh6iR8sQL0B9U8yYTwyGCgiFjz7hMC9JGVnXdhEU8MPgr/obJH0C8CKhBXZmqWVpzWg2VaQCnmd2X7EDxYFxa4lVu6FuuaI0zByLlMNnv5QnfSMqSdz2LGk2230CZUh6pNB3Y2I+CkeTpi8HKRATdGsJdLgez+nKJIYcOiG4NB2wF9I8oq4fOx1xpxNcUnHbuhs/YJxNaoQZpVElNAL0M5kx6ISFfbvDw8OI7fnFFZOP2acbwalAnDxznw81M9aq6FAJxhekOft0yV32Ac= ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:18:50.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-11T13:18:51.025 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:18:53.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.517+0000 7fb237200700 1 -- 172.21.15.63:0/1498801926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 msgr2=0x7fb238105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:53.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.517+0000 7fb237200700 1 --2- 172.21.15.63:0/1498801926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb238105c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb228009910 tx=0x7fb228009c20 comp rx=0 tx=0).stop 2024-05-11T13:18:53.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.517+0000 7fb237200700 1 -- 172.21.15.63:0/1498801926 shutdown_connections 2024-05-11T13:18:53.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.517+0000 7fb237200700 1 --2- 172.21.15.63:0/1498801926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb238105c90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:53.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.517+0000 7fb237200700 1 -- 172.21.15.63:0/1498801926 >> 172.21.15.63:0/1498801926 conn(0x7fb2380fd2b0 msgr2=0x7fb2380ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:53.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.517+0000 7fb237200700 1 -- 172.21.15.63:0/1498801926 shutdown_connections 2024-05-11T13:18:53.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.517+0000 7fb237200700 1 -- 172.21.15.63:0/1498801926 wait complete. 2024-05-11T13:18:53.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.518+0000 7fb237200700 1 Processor -- start 2024-05-11T13:18:53.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.518+0000 7fb237200700 1 -- start start 2024-05-11T13:18:53.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.519+0000 7fb237200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb2381982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:53.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.519+0000 7fb237200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb238108ba0 con 0x7fb2381038a0 2024-05-11T13:18:53.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.519+0000 7fb235e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb2381982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:53.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.519+0000 7fb235e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb2381982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59474/0 (socket says 172.21.15.63:59474) 2024-05-11T13:18:53.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.519+0000 7fb235e00700 1 -- 172.21.15.63:0/4107124956 learned_addr learned my addr 172.21.15.63:0/4107124956 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:53.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.520+0000 7fb235e00700 1 -- 172.21.15.63:0/4107124956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2381987e0 con 0x7fb2381038a0 2024-05-11T13:18:53.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.520+0000 7fb235e00700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb2381982a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b5c0 tx=0x7fb228005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:53.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.520+0000 7fb22ea00700 1 -- 172.21.15.63:0/4107124956 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb228027070 con 0x7fb2381038a0 2024-05-11T13:18:53.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.520+0000 7fb22ea00700 1 -- 172.21.15.63:0/4107124956 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb22800f460 con 0x7fb2381038a0 2024-05-11T13:18:53.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.520+0000 7fb22ea00700 1 -- 172.21.15.63:0/4107124956 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2280175d0 con 0x7fb2381038a0 2024-05-11T13:18:53.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.520+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb238198ad0 con 0x7fb2381038a0 2024-05-11T13:18:53.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.520+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23819b760 con 0x7fb2381038a0 2024-05-11T13:18:53.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.522+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb238109320 con 0x7fb2381038a0 2024-05-11T13:18:53.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.522+0000 7fb22ea00700 1 -- 172.21.15.63:0/4107124956 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb22802a070 con 0x7fb2381038a0 2024-05-11T13:18:53.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.522+0000 7fb22ea00700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb21c03e570 0x7fb21c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:53.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.522+0000 7fb22ea00700 1 -- 172.21.15.63:0/4107124956 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb2280539f0 con 0x7fb2381038a0 2024-05-11T13:18:53.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.523+0000 7fb235400700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb21c03e570 0x7fb21c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:53.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.523+0000 7fb235400700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb21c03e570 0x7fb21c040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.529+0000 7fb22ea00700 1 -- 172.21.15.63:0/4107124956 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb228013070 con 0x7fb2381038a0 2024-05-11T13:18:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.769+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb23802e7c0 con 0x7fb2381038a0 2024-05-11T13:18:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.774+0000 7fb22ea00700 1 -- 172.21.15.63:0/4107124956 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb22802be40 con 0x7fb2381038a0 2024-05-11T13:18:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.778+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb21c03e570 msgr2=0x7fb21c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.778+0000 7fb237200700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb21c03e570 0x7fb21c040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.778+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 msgr2=0x7fb2381982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.778+0000 7fb237200700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb2381982a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b5c0 tx=0x7fb228005810 comp rx=0 tx=0).stop 2024-05-11T13:18:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.778+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 shutdown_connections 2024-05-11T13:18:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.778+0000 7fb237200700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb21c03e570 0x7fb21c040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.778+0000 7fb237200700 1 --2- 172.21.15.63:0/4107124956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2381038a0 0x7fb2381982a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.779+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 >> 172.21.15.63:0/4107124956 conn(0x7fb2380fd2b0 msgr2=0x7fb2380fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.779+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 shutdown_connections 2024-05-11T13:18:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:53.779+0000 7fb237200700 1 -- 172.21.15.63:0/4107124956 wait complete. 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4107124956' entity='client.admin' 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:54.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:54.585 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-11T13:18:54.586 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-11T13:18:55.038 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:18:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.015+0000 7fd74fa00700 1 -- 172.21.15.63:0/303679845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 msgr2=0x7fd7481033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.015+0000 7fd74fa00700 1 --2- 172.21.15.63:0/303679845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd7481033d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd738009910 tx=0x7fd738009c20 comp rx=0 tx=0).stop 2024-05-11T13:18:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.015+0000 7fd74fa00700 1 -- 172.21.15.63:0/303679845 shutdown_connections 2024-05-11T13:18:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.015+0000 7fd74fa00700 1 --2- 172.21.15.63:0/303679845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd7481033d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.015+0000 7fd74fa00700 1 -- 172.21.15.63:0/303679845 >> 172.21.15.63:0/303679845 conn(0x7fd7480fe550 msgr2=0x7fd748100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.016+0000 7fd74fa00700 1 -- 172.21.15.63:0/303679845 shutdown_connections 2024-05-11T13:18:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.016+0000 7fd74fa00700 1 -- 172.21.15.63:0/303679845 wait complete. 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.017+0000 7fd74fa00700 1 Processor -- start 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.017+0000 7fd74fa00700 1 -- start start 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.017+0000 7fd74fa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd74819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.017+0000 7fd74fa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd748108da0 con 0x7fd748102fb0 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.018+0000 7fd74d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd74819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.018+0000 7fd74d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd74819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59490/0 (socket says 172.21.15.63:59490) 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.018+0000 7fd74d000700 1 -- 172.21.15.63:0/697749341 learned_addr learned my addr 172.21.15.63:0/697749341 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:18:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.018+0000 7fd74d000700 1 -- 172.21.15.63:0/697749341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74819ac70 con 0x7fd748102fb0 2024-05-11T13:18:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.018+0000 7fd74d000700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd74819a730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd738005790 tx=0x7fd7380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.019+0000 7fd746000700 1 -- 172.21.15.63:0/697749341 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd738027070 con 0x7fd748102fb0 2024-05-11T13:18:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.019+0000 7fd746000700 1 -- 172.21.15.63:0/697749341 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd73800bdf0 con 0x7fd748102fb0 2024-05-11T13:18:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.019+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74819af00 con 0x7fd748102fb0 2024-05-11T13:18:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.019+0000 7fd746000700 1 -- 172.21.15.63:0/697749341 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd73800fe50 con 0x7fd748102fb0 2024-05-11T13:18:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.019+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74819dbf0 con 0x7fd748102fb0 2024-05-11T13:18:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.020+0000 7fd746000700 1 -- 172.21.15.63:0/697749341 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd73802a070 con 0x7fd748102fb0 2024-05-11T13:18:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.020+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd748109520 con 0x7fd748102fb0 2024-05-11T13:18:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.021+0000 7fd746000700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd72c03e710 0x7fd72c040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:18:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.021+0000 7fd746000700 1 -- 172.21.15.63:0/697749341 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd7380538e0 con 0x7fd748102fb0 2024-05-11T13:18:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.021+0000 7fd73fe00700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd72c03e710 0x7fd72c040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:18:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.022+0000 7fd73fe00700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd72c03e710 0x7fd72c040bd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd734006fd0 tx=0x7fd734006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:18:57.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.027+0000 7fd746000700 1 -- 172.21.15.63:0/697749341 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd738013070 con 0x7fd748102fb0 2024-05-11T13:18:57.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.268+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fd74802e680 con 0x7fd72c03e710 2024-05-11T13:18:57.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.276+0000 7fd746000700 1 -- 172.21.15.63:0/697749341 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd74802e680 con 0x7fd72c03e710 2024-05-11T13:18:57.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd72c03e710 msgr2=0x7fd72c040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:57.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd72c03e710 0x7fd72c040bd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd734006fd0 tx=0x7fd734006e40 comp rx=0 tx=0).stop 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 msgr2=0x7fd74819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd74819a730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd738005790 tx=0x7fd7380057c0 comp rx=0 tx=0).stop 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 shutdown_connections 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd72c03e710 0x7fd72c040bd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 --2- 172.21.15.63:0/697749341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd748102fb0 0x7fd74819a730 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 >> 172.21.15.63:0/697749341 conn(0x7fd7480fe550 msgr2=0x7fd748100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 shutdown_connections 2024-05-11T13:18:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:18:57.279+0000 7fd74fa00700 1 -- 172.21.15.63:0/697749341 wait complete. 2024-05-11T13:18:58.024 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-05-11T13:18:58.024 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:18:58.024 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-05-11T13:18:58.080 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:18:58.080 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:18:58.173 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-05-11T13:18:58.173 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch host add smithi071 2024-05-11T13:18:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:18:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:58 smithi063 ceph-mon[29809]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:18:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:18:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:18:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:18:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:18:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:58 smithi063 ceph-mon[29809]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-11T13:18:59.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:59 smithi063 ceph-mon[29809]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:18:59.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:18:59 smithi063 ceph-mon[29809]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:00.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:00 smithi063 ceph-mon[29809]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:19:00.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:00 smithi063 ceph-mon[29809]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:00.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:00 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:00.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:00 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:00.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:00 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:00.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.662+0000 7f56c0400700 1 -- 172.21.15.63:0/4126851819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 msgr2=0x7f56b81057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:00.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.662+0000 7f56c0400700 1 --2- 172.21.15.63:0/4126851819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81057d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00b0a0 tx=0x7f56ac00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:19:00.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.662+0000 7f56c0400700 1 -- 172.21.15.63:0/4126851819 shutdown_connections 2024-05-11T13:19:00.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.662+0000 7f56c0400700 1 --2- 172.21.15.63:0/4126851819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81057d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:00.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.662+0000 7f56c0400700 1 -- 172.21.15.63:0/4126851819 >> 172.21.15.63:0/4126851819 conn(0x7f56b8100c60 msgr2=0x7f56b8103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:00.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.662+0000 7f56c0400700 1 -- 172.21.15.63:0/4126851819 shutdown_connections 2024-05-11T13:19:00.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.662+0000 7f56c0400700 1 -- 172.21.15.63:0/4126851819 wait complete. 2024-05-11T13:19:00.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.663+0000 7f56c0400700 1 Processor -- start 2024-05-11T13:19:00.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.663+0000 7f56c0400700 1 -- start start 2024-05-11T13:19:00.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.664+0000 7f56c0400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:00.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.664+0000 7f56c0400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56b810f520 con 0x7f56b81053f0 2024-05-11T13:19:00.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.664+0000 7f56bda00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:00.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.664+0000 7f56bda00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41318/0 (socket says 172.21.15.63:41318) 2024-05-11T13:19:00.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.664+0000 7f56bda00700 1 -- 172.21.15.63:0/471831563 learned_addr learned my addr 172.21.15.63:0/471831563 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:19:00.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.665+0000 7f56bda00700 1 -- 172.21.15.63:0/471831563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b81a0d90 con 0x7f56b81053f0 2024-05-11T13:19:00.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.665+0000 7f56bda00700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81a0850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f56ac007600 tx=0x7f56ac00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:00.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.665+0000 7f56b6000700 1 -- 172.21.15.63:0/471831563 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac017040 con 0x7f56b81053f0 2024-05-11T13:19:00.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.665+0000 7f56b6000700 1 -- 172.21.15.63:0/471831563 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56ac022070 con 0x7f56b81053f0 2024-05-11T13:19:00.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.665+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b81a1020 con 0x7f56b81053f0 2024-05-11T13:19:00.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.665+0000 7f56b6000700 1 -- 172.21.15.63:0/471831563 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac004870 con 0x7f56b81053f0 2024-05-11T13:19:00.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.666+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b819aa10 con 0x7f56b81053f0 2024-05-11T13:19:00.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.667+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b81068d0 con 0x7f56b81053f0 2024-05-11T13:19:00.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.667+0000 7f56b6000700 1 -- 172.21.15.63:0/471831563 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f56ac003a00 con 0x7f56b81053f0 2024-05-11T13:19:00.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.667+0000 7f56b6000700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56a003e5b0 0x7f56a0040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:00.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.667+0000 7f56b6000700 1 -- 172.21.15.63:0/471831563 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56ac00f050 con 0x7f56b81053f0 2024-05-11T13:19:00.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.668+0000 7f56bd000700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56a003e5b0 0x7f56a0040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:00.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.668+0000 7f56bd000700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56a003e5b0 0x7f56a0040a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006fd0 tx=0x7f56a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:00.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.673+0000 7f56b6000700 1 -- 172.21.15.63:0/471831563 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56ac025db0 con 0x7f56b81053f0 2024-05-11T13:19:00.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:00.916+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) -- 0x7f56b819bf40 con 0x7f56a003e5b0 2024-05-11T13:19:02.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:02 smithi063 ceph-mon[29809]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:19:02.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:02 smithi063 ceph-mon[29809]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:03.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:03 smithi063 ceph-mon[29809]: Deploying cephadm binary to smithi071 2024-05-11T13:19:04.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:04 smithi063 ceph-mon[29809]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:06 smithi063 ceph-mon[29809]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:07.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.405+0000 7f56b6000700 1 -- 172.21.15.63:0/471831563 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f56b819bf40 con 0x7f56a003e5b0 2024-05-11T13:19:07.406 INFO:teuthology.orchestra.run.smithi063.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-05-11T13:19:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.409+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56a003e5b0 msgr2=0x7f56a0040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.409+0000 7f56c0400700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56a003e5b0 0x7f56a0040a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006fd0 tx=0x7f56a4006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.409+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 msgr2=0x7f56b81a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.409+0000 7f56c0400700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81a0850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f56ac007600 tx=0x7f56ac00bd40 comp rx=0 tx=0).stop 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.410+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 shutdown_connections 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.410+0000 7f56c0400700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56a003e5b0 0x7f56a0040a70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.410+0000 7f56c0400700 1 --2- 172.21.15.63:0/471831563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b81053f0 0x7f56b81a0850 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.410+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 >> 172.21.15.63:0/471831563 conn(0x7f56b8100c60 msgr2=0x7f56b810cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.410+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 shutdown_connections 2024-05-11T13:19:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:07.410+0000 7f56c0400700 1 -- 172.21.15.63:0/471831563 wait complete. 2024-05-11T13:19:08.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-11T13:19:08.631 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:19:08.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:08 smithi063 ceph-mon[29809]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:08.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:08.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:08 smithi063 ceph-mon[29809]: Added host smithi071 2024-05-11T13:19:08.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:08.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:10.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:10.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:10 smithi063 ceph-mon[29809]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:10.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.656+0000 7f230b600700 1 -- 172.21.15.63:0/1223398213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 msgr2=0x7f23041057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:10.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.656+0000 7f230b600700 1 --2- 172.21.15.63:0/1223398213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041057c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009910 tx=0x7f22f8009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:10.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.656+0000 7f230b600700 1 -- 172.21.15.63:0/1223398213 shutdown_connections 2024-05-11T13:19:10.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.656+0000 7f230b600700 1 --2- 172.21.15.63:0/1223398213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041057c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:10.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.656+0000 7f230b600700 1 -- 172.21.15.63:0/1223398213 >> 172.21.15.63:0/1223398213 conn(0x7f2304100c70 msgr2=0x7f2304103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:10.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.656+0000 7f230b600700 1 -- 172.21.15.63:0/1223398213 shutdown_connections 2024-05-11T13:19:10.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.656+0000 7f230b600700 1 -- 172.21.15.63:0/1223398213 wait complete. 2024-05-11T13:19:10.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.658+0000 7f230b600700 1 Processor -- start 2024-05-11T13:19:10.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.658+0000 7f230b600700 1 -- start start 2024-05-11T13:19:10.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.658+0000 7f230b600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:10.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.658+0000 7f230b600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f230410f510 con 0x7f23041053e0 2024-05-11T13:19:10.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.659+0000 7f2308c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:10.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.659+0000 7f2308c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45268/0 (socket says 172.21.15.63:45268) 2024-05-11T13:19:10.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.659+0000 7f2308c00700 1 -- 172.21.15.63:0/2471941734 learned_addr learned my addr 172.21.15.63:0/2471941734 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:19:10.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.659+0000 7f2308c00700 1 -- 172.21.15.63:0/2471941734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23041a0c50 con 0x7f23041053e0 2024-05-11T13:19:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.659+0000 7f2308c00700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041a0710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22f8006010 tx=0x7f22f8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.660+0000 7f2301600700 1 -- 172.21.15.63:0/2471941734 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22f8026070 con 0x7f23041053e0 2024-05-11T13:19:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.660+0000 7f2301600700 1 -- 172.21.15.63:0/2471941734 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22f800b6f0 con 0x7f23041053e0 2024-05-11T13:19:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.660+0000 7f2301600700 1 -- 172.21.15.63:0/2471941734 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22f8003810 con 0x7f23041053e0 2024-05-11T13:19:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.660+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23041a0f40 con 0x7f23041053e0 2024-05-11T13:19:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.660+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230419a8d0 con 0x7f23041053e0 2024-05-11T13:19:10.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.661+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23041068c0 con 0x7f23041053e0 2024-05-11T13:19:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.661+0000 7f2301600700 1 -- 172.21.15.63:0/2471941734 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f22f8029070 con 0x7f23041053e0 2024-05-11T13:19:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.662+0000 7f2301600700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f22ec03e510 0x7f22ec0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.662+0000 7f2301600700 1 -- 172.21.15.63:0/2471941734 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f22f801e070 con 0x7f23041053e0 2024-05-11T13:19:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.662+0000 7f2303e00700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f22ec03e510 0x7f22ec0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.663+0000 7f2303e00700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f22ec03e510 0x7f22ec0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22f0006fd0 tx=0x7f22f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.668+0000 7f2301600700 1 -- 172.21.15.63:0/2471941734 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22f8013070 con 0x7f23041053e0 2024-05-11T13:19:10.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.916+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f230419bf10 con 0x7f22ec03e510 2024-05-11T13:19:10.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.919+0000 7f2301600700 1 -- 172.21.15.63:0/2471941734 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f230419bf10 con 0x7f22ec03e510 2024-05-11T13:19:10.920 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:19:10.920 INFO:teuthology.orchestra.run.smithi063.stdout:[{"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-05-11T13:19:10.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f22ec03e510 msgr2=0x7f22ec0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:10.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f22ec03e510 0x7f22ec0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22f0006fd0 tx=0x7f22f0006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 msgr2=0x7f23041a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041a0710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22f8006010 tx=0x7f22f8005710 comp rx=0 tx=0).stop 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 shutdown_connections 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f22ec03e510 0x7f22ec0409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 --2- 172.21.15.63:0/2471941734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23041053e0 0x7f23041a0710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 >> 172.21.15.63:0/2471941734 conn(0x7f2304100c70 msgr2=0x7f2304106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.923+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 shutdown_connections 2024-05-11T13:19:10.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:10.924+0000 7f230b600700 1 -- 172.21.15.63:0/2471941734 wait complete. 2024-05-11T13:19:11.672 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-05-11T13:19:11.672 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:19:11.672 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-05-11T13:19:11.732 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:19:11.732 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:11.829 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-05-11T13:19:11.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch host add smithi165 2024-05-11T13:19:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:19:12.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:12 smithi063 ceph-mon[29809]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:19:12.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:12 smithi063 ceph-mon[29809]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:12.641 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:12 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:14.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.280+0000 7fb4c1e00700 1 -- 172.21.15.63:0/79242422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 msgr2=0x7fb4bc075d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:14.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.280+0000 7fb4c1e00700 1 --2- 172.21.15.63:0/79242422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc075d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009910 tx=0x7fb4b0009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:14.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.280+0000 7fb4c1e00700 1 -- 172.21.15.63:0/79242422 shutdown_connections 2024-05-11T13:19:14.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.280+0000 7fb4c1e00700 1 --2- 172.21.15.63:0/79242422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc075d60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:14.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.280+0000 7fb4c1e00700 1 -- 172.21.15.63:0/79242422 >> 172.21.15.63:0/79242422 conn(0x7fb4bc0fe8b0 msgr2=0x7fb4bc100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:14.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.280+0000 7fb4c1e00700 1 -- 172.21.15.63:0/79242422 shutdown_connections 2024-05-11T13:19:14.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.280+0000 7fb4c1e00700 1 -- 172.21.15.63:0/79242422 wait complete. 2024-05-11T13:19:14.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.282+0000 7fb4c1e00700 1 Processor -- start 2024-05-11T13:19:14.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.282+0000 7fb4c1e00700 1 -- start start 2024-05-11T13:19:14.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.282+0000 7fb4c1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:14.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.282+0000 7fb4c1e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc10f3b0 con 0x7fb4bc075980 2024-05-11T13:19:14.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.283+0000 7fb4c0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:14.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.283+0000 7fb4c0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45290/0 (socket says 172.21.15.63:45290) 2024-05-11T13:19:14.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.283+0000 7fb4c0a00700 1 -- 172.21.15.63:0/586677200 learned_addr learned my addr 172.21.15.63:0/586677200 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:19:14.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.283+0000 7fb4c0a00700 1 -- 172.21.15.63:0/586677200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc1a0b80 con 0x7fb4bc075980 2024-05-11T13:19:14.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.284+0000 7fb4c0a00700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc1a0640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0006010 tx=0x7fb4b0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:14.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.284+0000 7fb4b9600700 1 -- 172.21.15.63:0/586677200 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0026070 con 0x7fb4bc075980 2024-05-11T13:19:14.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.284+0000 7fb4b9600700 1 -- 172.21.15.63:0/586677200 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b000b750 con 0x7fb4bc075980 2024-05-11T13:19:14.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.284+0000 7fb4b9600700 1 -- 172.21.15.63:0/586677200 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b00038a0 con 0x7fb4bc075980 2024-05-11T13:19:14.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.284+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc1a0e10 con 0x7fb4bc075980 2024-05-11T13:19:14.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.284+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc19a7c0 con 0x7fb4bc075980 2024-05-11T13:19:14.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.285+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc077030 con 0x7fb4bc075980 2024-05-11T13:19:14.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.285+0000 7fb4b9600700 1 -- 172.21.15.63:0/586677200 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb4b0029070 con 0x7fb4bc075980 2024-05-11T13:19:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.286+0000 7fb4b9600700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4a403e570 0x7fb4a4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.286+0000 7fb4b9600700 1 -- 172.21.15.63:0/586677200 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb4b001e070 con 0x7fb4bc075980 2024-05-11T13:19:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.286+0000 7fb4bbe00700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4a403e570 0x7fb4a4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:14.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.287+0000 7fb4bbe00700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4a403e570 0x7fb4a4040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006fd0 tx=0x7fb4a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:14.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.292+0000 7fb4b9600700 1 -- 172.21.15.63:0/586677200 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4b0013070 con 0x7fb4bc075980 2024-05-11T13:19:14.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:14.538+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}) -- 0x7fb4bc19bcf0 con 0x7fb4a403e570 2024-05-11T13:19:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:14 smithi063 ceph-mon[29809]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:15.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:15 smithi063 ceph-mon[29809]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:19:16.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:16 smithi063 ceph-mon[29809]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:17.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: Deploying cephadm binary to smithi165 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:19:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:17 smithi063 ceph-mon[29809]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:19:18.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:18 smithi063 ceph-mon[29809]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:18.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:18 smithi063 ceph-mon[29809]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:19:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:20 smithi063 ceph-mon[29809]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:21.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.266+0000 7fb4b9600700 1 -- 172.21.15.63:0/586677200 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb4bc19bcf0 con 0x7fb4a403e570 2024-05-11T13:19:21.267 INFO:teuthology.orchestra.run.smithi063.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-05-11T13:19:21.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4a403e570 msgr2=0x7fb4a4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:21.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4a403e570 0x7fb4a4040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006fd0 tx=0x7fb4a8006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:21.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 msgr2=0x7fb4bc1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:21.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc1a0640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0006010 tx=0x7fb4b0005710 comp rx=0 tx=0).stop 2024-05-11T13:19:21.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 shutdown_connections 2024-05-11T13:19:21.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4a403e570 0x7fb4a4040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:21.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 --2- 172.21.15.63:0/586677200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4bc075980 0x7fb4bc1a0640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:21.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.270+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 >> 172.21.15.63:0/586677200 conn(0x7fb4bc0fe8b0 msgr2=0x7fb4bc109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:21.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.271+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 shutdown_connections 2024-05-11T13:19:21.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:21.271+0000 7fb4c1e00700 1 -- 172.21.15.63:0/586677200 wait complete. 2024-05-11T13:19:21.670 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:21 smithi063 ceph-mon[29809]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:21.670 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:21 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:22.097 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-11T13:19:22.553 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:19:22.904 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:22 smithi063 ceph-mon[29809]: Added host smithi165 2024-05-11T13:19:22.904 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:22 smithi063 ceph-mon[29809]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:22.904 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:22 smithi063 ceph-mon[29809]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:19:22.904 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:22.904 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:22.904 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:23.760 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:23 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:24.650 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:24 smithi063 ceph-mon[29809]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:24.650 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:24.650 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:24.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.698+0000 7f4301000700 1 -- 172.21.15.63:0/2486550663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 msgr2=0x7f42fc10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:24.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.698+0000 7f4301000700 1 --2- 172.21.15.63:0/2486550663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc10e970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009910 tx=0x7f42ec009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:24.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.698+0000 7f4301000700 1 -- 172.21.15.63:0/2486550663 shutdown_connections 2024-05-11T13:19:24.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.698+0000 7f4301000700 1 --2- 172.21.15.63:0/2486550663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc10e970 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:24.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.698+0000 7f4301000700 1 -- 172.21.15.63:0/2486550663 >> 172.21.15.63:0/2486550663 conn(0x7f42fc078cf0 msgr2=0x7f42fc07b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:24.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.698+0000 7f4301000700 1 -- 172.21.15.63:0/2486550663 shutdown_connections 2024-05-11T13:19:24.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.698+0000 7f4301000700 1 -- 172.21.15.63:0/2486550663 wait complete. 2024-05-11T13:19:24.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.699+0000 7f4301000700 1 Processor -- start 2024-05-11T13:19:24.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.699+0000 7f4301000700 1 -- start start 2024-05-11T13:19:24.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.700+0000 7f4301000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:24.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.700+0000 7f4301000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc113960 con 0x7f42fc10e590 2024-05-11T13:19:24.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.700+0000 7f42f9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:24.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.700+0000 7f42f9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42000/0 (socket says 172.21.15.63:42000) 2024-05-11T13:19:24.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.700+0000 7f42f9e00700 1 -- 172.21.15.63:0/771328543 learned_addr learned my addr 172.21.15.63:0/771328543 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:19:24.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.700+0000 7f42f9e00700 1 -- 172.21.15.63:0/771328543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc19f490 con 0x7f42fc10e590 2024-05-11T13:19:24.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.701+0000 7f42f9e00700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc19ef50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0092e0 tx=0x7f42ec005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:24.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.701+0000 7f42f2a00700 1 -- 172.21.15.63:0/771328543 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42ec026070 con 0x7f42fc10e590 2024-05-11T13:19:24.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.701+0000 7f42f2a00700 1 -- 172.21.15.63:0/771328543 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42ec00b750 con 0x7f42fc10e590 2024-05-11T13:19:24.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.701+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc19f720 con 0x7f42fc10e590 2024-05-11T13:19:24.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.701+0000 7f42f2a00700 1 -- 172.21.15.63:0/771328543 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42ec003810 con 0x7f42fc10e590 2024-05-11T13:19:24.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.701+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc19fb80 con 0x7f42fc10e590 2024-05-11T13:19:24.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.702+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42fc10ac90 con 0x7f42fc10e590 2024-05-11T13:19:24.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.702+0000 7f42f2a00700 1 -- 172.21.15.63:0/771328543 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f42ec029070 con 0x7f42fc10e590 2024-05-11T13:19:24.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.703+0000 7f42f2a00700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f42e003e570 0x7f42e0040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:24.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.703+0000 7f42f2a00700 1 -- 172.21.15.63:0/771328543 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f42ec01e070 con 0x7f42fc10e590 2024-05-11T13:19:24.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.703+0000 7f42f9400700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f42e003e570 0x7f42e0040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:24.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.704+0000 7f42f9400700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f42e003e570 0x7f42e0040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f42e8006fd0 tx=0x7f42e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:24.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.709+0000 7f42f2a00700 1 -- 172.21.15.63:0/771328543 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42ec013070 con 0x7f42fc10e590 2024-05-11T13:19:24.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.953+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42fc1a02b0 con 0x7f42e003e570 2024-05-11T13:19:24.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.955+0000 7f42f2a00700 1 -- 172.21.15.63:0/771328543 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f42fc1a02b0 con 0x7f42e003e570 2024-05-11T13:19:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:19:24.956 INFO:teuthology.orchestra.run.smithi063.stdout:[{"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-05-11T13:19:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.958+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f42e003e570 msgr2=0x7f42e0040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f42e003e570 0x7f42e0040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f42e8006fd0 tx=0x7f42e8006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 msgr2=0x7f42fc19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc19ef50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0092e0 tx=0x7f42ec005710 comp rx=0 tx=0).stop 2024-05-11T13:19:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 shutdown_connections 2024-05-11T13:19:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f42e003e570 0x7f42e0040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 --2- 172.21.15.63:0/771328543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42fc10e590 0x7f42fc19ef50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 >> 172.21.15.63:0/771328543 conn(0x7f42fc078cf0 msgr2=0x7f42fc079a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 shutdown_connections 2024-05-11T13:19:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:24.959+0000 7f4301000700 1 -- 172.21.15.63:0/771328543 wait complete. 2024-05-11T13:19:25.659 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-11T13:19:25.659 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-11T13:19:26.113 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:19:26.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:26 smithi063 ceph-mon[29809]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:19:26.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:26 smithi063 ceph-mon[29809]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:27.648 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:28.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.235+0000 7fcc15800700 1 -- 172.21.15.63:0/1270522628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 msgr2=0x7fcc101035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:28.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.235+0000 7fcc15800700 1 --2- 172.21.15.63:0/1270522628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101035a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009910 tx=0x7fcc00009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:28.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.236+0000 7fcc15800700 1 -- 172.21.15.63:0/1270522628 shutdown_connections 2024-05-11T13:19:28.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.236+0000 7fcc15800700 1 --2- 172.21.15.63:0/1270522628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101035a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:28.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.236+0000 7fcc15800700 1 -- 172.21.15.63:0/1270522628 >> 172.21.15.63:0/1270522628 conn(0x7fcc100fea90 msgr2=0x7fcc10100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:28.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.236+0000 7fcc15800700 1 -- 172.21.15.63:0/1270522628 shutdown_connections 2024-05-11T13:19:28.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.236+0000 7fcc15800700 1 -- 172.21.15.63:0/1270522628 wait complete. 2024-05-11T13:19:28.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.237+0000 7fcc15800700 1 Processor -- start 2024-05-11T13:19:28.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.237+0000 7fcc15800700 1 -- start start 2024-05-11T13:19:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.238+0000 7fcc15800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.238+0000 7fcc15800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1010d2f0 con 0x7fcc101031c0 2024-05-11T13:19:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.238+0000 7fcc0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.238+0000 7fcc0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42022/0 (socket says 172.21.15.63:42022) 2024-05-11T13:19:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.238+0000 7fcc0ea00700 1 -- 172.21.15.63:0/2426585924 learned_addr learned my addr 172.21.15.63:0/2426585924 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:19:28.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.239+0000 7fcc0ea00700 1 -- 172.21.15.63:0/2426585924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc101a0c50 con 0x7fcc101031c0 2024-05-11T13:19:28.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.239+0000 7fcc0ea00700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101a0710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006010 tx=0x7fcc00005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:28.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.239+0000 7fcc07400700 1 -- 172.21.15.63:0/2426585924 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00026070 con 0x7fcc101031c0 2024-05-11T13:19:28.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.240+0000 7fcc07400700 1 -- 172.21.15.63:0/2426585924 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc0000b6f0 con 0x7fcc101031c0 2024-05-11T13:19:28.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.240+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc101a0f40 con 0x7fcc101031c0 2024-05-11T13:19:28.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.240+0000 7fcc07400700 1 -- 172.21.15.63:0/2426585924 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00003810 con 0x7fcc101031c0 2024-05-11T13:19:28.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.240+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1019a8d0 con 0x7fcc101031c0 2024-05-11T13:19:28.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.241+0000 7fcc07400700 1 -- 172.21.15.63:0/2426585924 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fcc00029070 con 0x7fcc101031c0 2024-05-11T13:19:28.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.241+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc101046a0 con 0x7fcc101031c0 2024-05-11T13:19:28.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.242+0000 7fcc07400700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcbf403e510 0x7fcbf40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:28.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.242+0000 7fcc07400700 1 -- 172.21.15.63:0/2426585924 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc0001e070 con 0x7fcc101031c0 2024-05-11T13:19:28.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.242+0000 7fcc0e000700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcbf403e510 0x7fcbf40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:28.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.243+0000 7fcc0e000700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcbf403e510 0x7fcbf40409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc006fd0 tx=0x7fcbfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:28.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.247+0000 7fcc07400700 1 -- 172.21.15.63:0/2426585924 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc00013070 con 0x7fcc101031c0 2024-05-11T13:19:28.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.487+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcc1019bea0 con 0x7fcc101031c0 2024-05-11T13:19:28.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.573+0000 7fcc07400700 1 -- 172.21.15.63:0/2426585924 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcc0000b8a0 con 0x7fcc101031c0 2024-05-11T13:19:28.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.577+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcbf403e510 msgr2=0x7fcbf40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:28.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.577+0000 7fcc15800700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcbf403e510 0x7fcbf40409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc006fd0 tx=0x7fcbfc006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:28.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.577+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 msgr2=0x7fcc101a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:28.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.577+0000 7fcc15800700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101a0710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006010 tx=0x7fcc00005710 comp rx=0 tx=0).stop 2024-05-11T13:19:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.578+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 shutdown_connections 2024-05-11T13:19:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.578+0000 7fcc15800700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcbf403e510 0x7fcbf40409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.578+0000 7fcc15800700 1 --2- 172.21.15.63:0/2426585924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc101031c0 0x7fcc101a0710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.578+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 >> 172.21.15.63:0/2426585924 conn(0x7fcc100fea90 msgr2=0x7fcc10075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.578+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 shutdown_connections 2024-05-11T13:19:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:19:28.578+0000 7fcc15800700 1 -- 172.21.15.63:0/2426585924 wait complete. 2024-05-11T13:19:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:adjusted tunables profile to default 2024-05-11T13:19:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:28 smithi063 ceph-mon[29809]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:28 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2426585924' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-11T13:19:29.352 INFO:tasks.cephadm:Adding mon.a on smithi063 2024-05-11T13:19:29.353 INFO:tasks.cephadm:Adding mon.b on smithi071 2024-05-11T13:19:29.353 INFO:tasks.cephadm:Adding mon.c on smithi165 2024-05-11T13:19:29.353 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c' 2024-05-11T13:19:29.764 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-11T13:19:29.872 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-11T13:19:29.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:29 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2426585924' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-11T13:19:29.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:29 smithi063 ceph-mon[29809]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:30.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:30 smithi063 ceph-mon[29809]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:32.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.363+0000 7fb4bc400700 1 -- 172.21.15.165:0/419122095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 msgr2=0x7fb4b4105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.363+0000 7fb4bc400700 1 --2- 172.21.15.165:0/419122095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b4105620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009910 tx=0x7fb4a8009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.363+0000 7fb4bc400700 1 -- 172.21.15.165:0/419122095 shutdown_connections 2024-05-11T13:19:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.364+0000 7fb4bc400700 1 --2- 172.21.15.165:0/419122095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b4105620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.364+0000 7fb4bc400700 1 -- 172.21.15.165:0/419122095 >> 172.21.15.165:0/419122095 conn(0x7fb4b4100780 msgr2=0x7fb4b4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.364+0000 7fb4bc400700 1 -- 172.21.15.165:0/419122095 shutdown_connections 2024-05-11T13:19:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.364+0000 7fb4bc400700 1 -- 172.21.15.165:0/419122095 wait complete. 2024-05-11T13:19:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.365+0000 7fb4bc400700 1 Processor -- start 2024-05-11T13:19:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.365+0000 7fb4bc400700 1 -- start start 2024-05-11T13:19:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.366+0000 7fb4bc400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.366+0000 7fb4bc400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410aff0 con 0x7fb4b4105200 2024-05-11T13:19:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.366+0000 7fb4b9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.367+0000 7fb4b9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.165:48978/0 (socket says 172.21.15.165:48978) 2024-05-11T13:19:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.367+0000 7fb4b9a00700 1 -- 172.21.15.165:0/1784740214 learned_addr learned my addr 172.21.15.165:0/1784740214 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:19:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.367+0000 7fb4b9a00700 1 -- 172.21.15.165:0/1784740214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419ac90 con 0x7fb4b4105200 2024-05-11T13:19:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.368+0000 7fb4b9a00700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b419a750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8005120 tx=0x7fb4a8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.368+0000 7fb4b2000700 1 -- 172.21.15.165:0/1784740214 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a8027070 con 0x7fb4b4105200 2024-05-11T13:19:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.368+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419af20 con 0x7fb4b4105200 2024-05-11T13:19:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.368+0000 7fb4b2000700 1 -- 172.21.15.165:0/1784740214 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4a800f460 con 0x7fb4b4105200 2024-05-11T13:19:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.368+0000 7fb4b2000700 1 -- 172.21.15.165:0/1784740214 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a8017440 con 0x7fb4b4105200 2024-05-11T13:19:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.368+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b419dc10 con 0x7fb4b4105200 2024-05-11T13:19:32.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.370+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b410b770 con 0x7fb4b4105200 2024-05-11T13:19:32.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.370+0000 7fb4b2000700 1 -- 172.21.15.165:0/1784740214 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb4a802a070 con 0x7fb4b4105200 2024-05-11T13:19:32.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.371+0000 7fb4b2000700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb49c03e570 0x7fb49c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:32.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.371+0000 7fb4b2000700 1 -- 172.21.15.165:0/1784740214 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb4a80539e0 con 0x7fb4b4105200 2024-05-11T13:19:32.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.372+0000 7fb4b9000700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb49c03e570 0x7fb49c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:32.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.373+0000 7fb4b9000700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb49c03e570 0x7fb49c040a30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006fd0 tx=0x7fb4a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:32.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.387+0000 7fb4b2000700 1 -- 172.21.15.165:0/1784740214 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4a8013070 con 0x7fb4b4105200 2024-05-11T13:19:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.643+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}) -- 0x7fb4b402e800 con 0x7fb49c03e570 2024-05-11T13:19:32.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.660+0000 7fb4b2000700 1 -- 172.21.15.165:0/1784740214 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb4b402e800 con 0x7fb49c03e570 2024-05-11T13:19:32.660 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mon update... 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb49c03e570 msgr2=0x7fb49c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb49c03e570 0x7fb49c040a30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006fd0 tx=0x7fb4a0006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 msgr2=0x7fb4b419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b419a750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8005120 tx=0x7fb4a8005810 comp rx=0 tx=0).stop 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 shutdown_connections 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb49c03e570 0x7fb49c040a30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 --2- 172.21.15.165:0/1784740214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4b4105200 0x7fb4b419a750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 >> 172.21.15.165:0/1784740214 conn(0x7fb4b4100780 msgr2=0x7fb4b41012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 shutdown_connections 2024-05-11T13:19:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:32.663+0000 7fb4bc400700 1 -- 172.21.15.165:0/1784740214 wait complete. 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:19:33.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:33.456 DEBUG:teuthology.orchestra.run.smithi071:mon.b> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.b.service 2024-05-11T13:19:33.464 DEBUG:teuthology.orchestra.run.smithi165:mon.c> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.c.service 2024-05-11T13:19:33.469 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T13:19:33.469 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T13:19:33.896 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:19:34.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:33 smithi063 ceph-mon[29809]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:19:34.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:33 smithi063 ceph-mon[29809]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:19:34.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:33 smithi063 ceph-mon[29809]: Saving service mon spec with placement smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c;count:3 2024-05-11T13:19:35.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:34 smithi063 ceph-mon[29809]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:35.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:34 smithi063 ceph-mon[29809]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:35.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:35.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:35.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:35.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:19:35.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.137+0000 7f49d8000700 1 -- 172.21.15.165:0/3388131270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 msgr2=0x7f49d01057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.137+0000 7f49d8000700 1 --2- 172.21.15.165:0/3388131270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01057c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009910 tx=0x7f49c4009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.137+0000 7f49d8000700 1 -- 172.21.15.165:0/3388131270 shutdown_connections 2024-05-11T13:19:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.137+0000 7f49d8000700 1 --2- 172.21.15.165:0/3388131270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01057c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.137+0000 7f49d8000700 1 -- 172.21.15.165:0/3388131270 >> 172.21.15.165:0/3388131270 conn(0x7f49d0100c70 msgr2=0x7f49d0103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.138+0000 7f49d8000700 1 -- 172.21.15.165:0/3388131270 shutdown_connections 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.138+0000 7f49d8000700 1 -- 172.21.15.165:0/3388131270 wait complete. 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.139+0000 7f49d8000700 1 Processor -- start 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.139+0000 7f49d8000700 1 -- start start 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.140+0000 7f49d8000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.140+0000 7f49d8000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49d010f510 con 0x7f49d01053e0 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.140+0000 7f49d5600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.140+0000 7f49d5600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.165:54238/0 (socket says 172.21.15.165:54238) 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.140+0000 7f49d5600700 1 -- 172.21.15.165:0/1231232441 learned_addr learned my addr 172.21.15.165:0/1231232441 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.141+0000 7f49d5600700 1 -- 172.21.15.165:0/1231232441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d01a0c50 con 0x7f49d01053e0 2024-05-11T13:19:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.141+0000 7f49d5600700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01a0710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49c4006010 tx=0x7f49c4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.142+0000 7f49ce000700 1 -- 172.21.15.165:0/1231232441 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4026070 con 0x7f49d01053e0 2024-05-11T13:19:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.142+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d01a0f40 con 0x7f49d01053e0 2024-05-11T13:19:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.142+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d019a8d0 con 0x7f49d01053e0 2024-05-11T13:19:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.142+0000 7f49ce000700 1 -- 172.21.15.165:0/1231232441 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49c400b6f0 con 0x7f49d01053e0 2024-05-11T13:19:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.142+0000 7f49ce000700 1 -- 172.21.15.165:0/1231232441 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c40037f0 con 0x7f49d01053e0 2024-05-11T13:19:36.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.143+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b0005330 con 0x7f49d01053e0 2024-05-11T13:19:36.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.144+0000 7f49ce000700 1 -- 172.21.15.165:0/1231232441 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f49c4029070 con 0x7f49d01053e0 2024-05-11T13:19:36.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.144+0000 7f49ce000700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f49b803e570 0x7f49b8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:36.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.144+0000 7f49ce000700 1 -- 172.21.15.165:0/1231232441 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f49c401e070 con 0x7f49d01053e0 2024-05-11T13:19:36.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.145+0000 7f49d4c00700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f49b803e570 0x7f49b8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:36.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.146+0000 7f49d4c00700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f49b803e570 0x7f49b8040a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49bc006fd0 tx=0x7f49bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:36.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.160+0000 7f49ce000700 1 -- 172.21.15.165:0/1231232441 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49c4013070 con 0x7f49d01053e0 2024-05-11T13:19:36.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:35 smithi063 ceph-mon[29809]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:19:36.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:35 smithi063 ceph-mon[29809]: Deploying daemon mon.c on smithi165 2024-05-11T13:19:36.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.490+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f49b00058f0 con 0x7f49d01053e0 2024-05-11T13:19:36.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.491+0000 7f49ce000700 1 -- 172.21.15.165:0/1231232441 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f49c4021e20 con 0x7f49d01053e0 2024-05-11T13:19:36.491 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:19:36.492 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":1,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","modified":"2024-05-11T13:17:18.239963Z","created":"2024-05-11T13:17:18.239963Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-11T13:19:36.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f49b803e570 msgr2=0x7f49b8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:36.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f49b803e570 0x7f49b8040a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49bc006fd0 tx=0x7f49bc006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 msgr2=0x7f49d01a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01a0710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49c4006010 tx=0x7f49c4005710 comp rx=0 tx=0).stop 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 shutdown_connections 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f49b803e570 0x7f49b8040a30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 --2- 172.21.15.165:0/1231232441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d01053e0 0x7f49d01a0710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.494+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 >> 172.21.15.165:0/1231232441 conn(0x7f49d0100c70 msgr2=0x7f49d0106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.495+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 shutdown_connections 2024-05-11T13:19:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:36.495+0000 7f49d8000700 1 -- 172.21.15.165:0/1231232441 wait complete. 2024-05-11T13:19:36.496 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 1 2024-05-11T13:19:37.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:36 smithi063 ceph-mon[29809]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:37.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:36 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/1231232441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:39.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:38 smithi063 ceph-mon[29809]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:39.457 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T13:19:39.457 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T13:19:39.910 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:19:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:40 smithi063 ceph-mon[29809]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.221 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 podman[33188]: 2024-05-11 13:19:42.026059942 +0000 UTC m=+0.211817515 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:19:42.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.387+0000 7fda17e00700 1 -- 172.21.15.165:0/3463509921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 msgr2=0x7fda101045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.387+0000 7fda17e00700 1 --2- 172.21.15.165:0/3463509921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda101045e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda04009910 tx=0x7fda04009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.387+0000 7fda17e00700 1 -- 172.21.15.165:0/3463509921 shutdown_connections 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.387+0000 7fda17e00700 1 --2- 172.21.15.165:0/3463509921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda101045e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.387+0000 7fda17e00700 1 -- 172.21.15.165:0/3463509921 >> 172.21.15.165:0/3463509921 conn(0x7fda100fda30 msgr2=0x7fda100ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.387+0000 7fda17e00700 1 -- 172.21.15.165:0/3463509921 shutdown_connections 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.387+0000 7fda17e00700 1 -- 172.21.15.165:0/3463509921 wait complete. 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.388+0000 7fda17e00700 1 Processor -- start 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.389+0000 7fda17e00700 1 -- start start 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.389+0000 7fda17e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda10101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.389+0000 7fda17e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda1010d450 con 0x7fda10104200 2024-05-11T13:19:42.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.390+0000 7fda15400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda10101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.390+0000 7fda15400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda10101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.165:54246/0 (socket says 172.21.15.165:54246) 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.390+0000 7fda15400700 1 -- 172.21.15.165:0/3207614344 learned_addr learned my addr 172.21.15.165:0/3207614344 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.390+0000 7fda15400700 1 -- 172.21.15.165:0/3207614344 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda10103d50 con 0x7fda10104200 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.391+0000 7fda15400700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda10101e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda04006010 tx=0x7fda04005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.391+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda04026070 con 0x7fda10104200 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.391+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda101023a0 con 0x7fda10104200 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.391+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda0400b750 con 0x7fda10104200 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.391+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda040038a0 con 0x7fda10104200 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.391+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda10102860 con 0x7fda10104200 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.392+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1006b7b0 con 0x7fda10104200 2024-05-11T13:19:42.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.393+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fda04029070 con 0x7fda10104200 2024-05-11T13:19:42.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.393+0000 7fda0e000700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd9f803e570 0x7fd9f8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:42.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.393+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fda0401e070 con 0x7fda10104200 2024-05-11T13:19:42.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.394+0000 7fda14a00700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd9f803e570 0x7fd9f8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:42.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.395+0000 7fda14a00700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd9f803e570 0x7fd9f8040a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc006fd0 tx=0x7fd9fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:42.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.404+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda04013070 con 0x7fda10104200 2024-05-11T13:19:42.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 podman[33188]: 2024-05-11 13:19:42.488813507 +0000 UTC m=+0.674571117 container init a0e327cdd6b074b910834abbfcc18dfa31cb407673967d2ecc17c8367a4b16a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:19:42.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.600+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fda040142b0 con 0x7fda10104200 2024-05-11T13:19:42.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:42.738+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fda10103430 con 0x7fda10104200 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 podman[33188]: 2024-05-11 13:19:42.499218597 +0000 UTC m=+0.684976230 container start a0e327cdd6b074b910834abbfcc18dfa31cb407673967d2ecc17c8367a4b16a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: set uid:gid to 167:167 (ceph:ceph) 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev), process ceph-mon, pid 2 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pidfile_write: ignore empty --pid-file 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: load: jerasure load: lrc 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: RocksDB version: 7.9.2 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Git sha 0 2024-05-11T13:19:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Compile date 2024-05-10 10:41:35 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: DB SUMMARY 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: DB Session ID: DD2QZ2H9C6K8TKBF6NOR 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: CURRENT file: CURRENT 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: IDENTITY file: IDENTITY 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.error_if_exists: 0 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.create_if_missing: 0 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.paranoid_checks: 1 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-11T13:19:42.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.env: 0x562419766cc0 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.fs: PosixFileSystem 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.info_log: 0x562450e87e00 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_file_opening_threads: 16 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.statistics: (nil) 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.use_fsync: 0 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_log_file_size: 0 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.keep_log_file_num: 1000 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.recycle_log_file_num: 0 2024-05-11T13:19:42.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.allow_fallocate: 1 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.allow_mmap_reads: 0 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.allow_mmap_writes: 0 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.use_direct_reads: 0 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.create_missing_column_families: 0 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.db_log_dir: 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.wal_dir: 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-11T13:19:42.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.advise_random_on_open: 1 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.db_write_buffer_size: 0 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.write_buffer_manager: 0x562450e590e0 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.rate_limiter: (nil) 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.wal_recovery_mode: 2 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.enable_thread_tracking: 0 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.enable_pipelined_write: 0 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.unordered_write: 0 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-11T13:19:42.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.row_cache: None 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.wal_filter: None 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.allow_ingest_behind: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.two_write_queues: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.manual_wal_flush: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.wal_compression: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.atomic_flush: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-11T13:19:42.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.log_readahead_size: 0 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.best_efforts_recovery: 0 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.allow_data_in_errors: 0 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.db_host_id: __hostname__ 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_background_jobs: 2 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_background_compactions: -1 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_subcompactions: 1 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-11T13:19:42.823 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_total_wal_size: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_open_files: -1 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bytes_per_sync: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_readahead_size: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_background_flushes: -1 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Compression algorithms supported: 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kZSTD supported: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kXpressCompression supported: 0 2024-05-11T13:19:42.824 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kLZ4HCCompression supported: 1 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kZlibCompression supported: 1 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kSnappyCompression supported: 1 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kLZ4Compression supported: 1 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: kBZip2Compression supported: 0 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-11T13:19:42.825 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.merge_operator: 2024-05-11T13:19:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_filter: None 2024-05-11T13:19:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_filter_factory: None 2024-05-11T13:19:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.sst_partitioner_factory: None 2024-05-11T13:19:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562450e87fc0) 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: cache_index_and_filter_blocks: 1 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: pin_top_level_index_and_filter: 1 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: index_type: 0 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: data_block_index_type: 0 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: index_shortening: 1 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: checksum: 4 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: no_block_cache: 0 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache: 0x562450e8b1f0 2024-05-11T13:19:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache_name: BinnedLRUCache 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache_options: 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: capacity : 536870912 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: num_shard_bits : 4 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: strict_capacity_limit : 0 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: high_pri_pool_ratio: 0.000 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache_compressed: (nil) 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: persistent_cache: (nil) 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_size: 4096 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_size_deviation: 10 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_restart_interval: 16 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: index_block_restart_interval: 1 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: metadata_block_size: 4096 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: partition_filters: 0 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: use_delta_encoding: 1 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: filter_policy: bloomfilter 2024-05-11T13:19:42.828 INFO:journalctl@ceph.mon.c.smithi165.stdout: whole_key_filtering: 1 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: verify_compression: 0 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: read_amp_bytes_per_bit: 0 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: format_version: 5 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: enable_index_compression: 1 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_align: 0 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: max_auto_readahead_size: 262144 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: prepopulate_block_cache: 0 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: initial_auto_readahead_size: 8192 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout: num_file_reads_for_auto_readahead: 2 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.write_buffer_size: 33554432 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_write_buffer_number: 2 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression: NoCompression 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression: Disabled 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.prefix_extractor: nullptr 2024-05-11T13:19:42.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.num_levels: 7 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-11T13:19:42.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.level: 32767 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.strategy: 0 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.enabled: false 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.target_file_size_base: 67108864 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-11T13:19:42.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.arena_block_size: 1048576 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.disable_auto_compactions: 0 2024-05-11T13:19:42.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.inplace_update_support: 0 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-11T13:19:42.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.bloom_locality: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.max_successive_merges: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.paranoid_file_checks: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.force_consistency_checks: 1 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.report_bg_io_stats: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.ttl: 2592000 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.enable_blob_files: false 2024-05-11T13:19:42.834 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.min_blob_size: 0 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.blob_file_size: 268435456 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.blob_file_starting_level: 0 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5903d257-9704-4f31-848e-0aa5404ec542 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433582553560, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433582554806, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715433582, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5903d257-9704-4f31-848e-0aa5404ec542", "db_session_id": "DD2QZ2H9C6K8TKBF6NOR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-11T13:19:42.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433582555052, "job": 1, "event": "recovery_finished"} 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562450fa8000 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: DB pointer 0x562450f94000 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** DB Stats ** 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T13:19:42.836 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** Compaction Stats [default] ** 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** Compaction Stats [default] ** 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-05-11T13:19:42.837 INFO:journalctl@ceph.mon.c.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: Block cache BinnedLRUCache@0x562450e8b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: using public_addr v2:172.21.15.165:0/0 -> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-05-11T13:19:42.838 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: starting mon.c rank -1 at public addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] at bind addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(???) e0 preinit fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 bash[33188]: a0e327cdd6b074b910834abbfcc18dfa31cb407673967d2ecc17c8367a4b16a9 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).mds e1 new map 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout: e1 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout: legacy client fscid: -1 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout: No filesystems configured 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-11T13:19:42.839 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Added host smithi071 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:19:42.840 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Deploying cephadm binary to smithi165 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:19:42.841 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Added host smithi165 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.842 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2426585924' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2426585924' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.843 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Saving service mon spec with placement smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c;count:3 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.844 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: Deploying daemon mon.c on smithi165 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/1231232441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 systemd[1]: Started Ceph mon.c for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:19:42.845 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:42 smithi165 ceph-mon[33235]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-11T13:19:47.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.628+0000 7fda0e000700 1 -- 172.21.15.165:0/3207614344 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fda04030090 con 0x7fda10104200 2024-05-11T13:19:47.630 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:19:47.630 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","modified":"2024-05-11T13:19:42.600437Z","created":"2024-05-11T13:17:18.239963Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-11T13:19:47.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd9f803e570 msgr2=0x7fd9f8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:47.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd9f803e570 0x7fd9f8040a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc006fd0 tx=0x7fd9fc006e40 comp rx=0 tx=0).stop 2024-05-11T13:19:47.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 msgr2=0x7fda10101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:47.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda10101e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda04006010 tx=0x7fda04005710 comp rx=0 tx=0).stop 2024-05-11T13:19:47.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 shutdown_connections 2024-05-11T13:19:47.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd9f803e570 0x7fd9f8040a30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:47.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 --2- 172.21.15.165:0/3207614344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda10104200 0x7fda10101e60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:47.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.632+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 >> 172.21.15.165:0/3207614344 conn(0x7fda100fda30 msgr2=0x7fda10106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:47.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.633+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 shutdown_connections 2024-05-11T13:19:47.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:47.633+0000 7fda17e00700 1 -- 172.21.15.165:0/3207614344 wait complete. 2024-05-11T13:19:47.635 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-05-11T13:19:47.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:19:47.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: mon.a calling monitor election 2024-05-11T13:19:47.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/3207614344' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: mon.c calling monitor election 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: monmap epoch 2 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: last_changed 2024-05-11T13:19:42.600437+0000 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:19:47.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: min_mon_release 19 (squid) 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: election_strategy: 1 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: fsmap 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: mgrmap e13: x(active, since 68s) 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: overall HEALTH_OK 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:47.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:47.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:19:47.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: mon.a calling monitor election 2024-05-11T13:19:47.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/3207614344' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:47.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:47.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: mon.c calling monitor election 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: monmap epoch 2 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: last_changed 2024-05-11T13:19:42.600437+0000 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: min_mon_release 19 (squid) 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: election_strategy: 1 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:19:47.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-11T13:19:47.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: fsmap 2024-05-11T13:19:47.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:47.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: mgrmap e13: x(active, since 68s) 2024-05-11T13:19:47.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: overall HEALTH_OK 2024-05-11T13:19:47.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:47.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:48 smithi165 ceph-mon[33235]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:19:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:48 smithi165 ceph-mon[33235]: Deploying daemon mon.b on smithi071 2024-05-11T13:19:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:48 smithi063 ceph-mon[29809]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:19:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:48 smithi063 ceph-mon[29809]: Deploying daemon mon.b on smithi071 2024-05-11T13:19:49.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:49.301 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T13:19:49.302 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T13:19:49.736 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:19:49.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:19:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:19:49.600+0000 7fa66b200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-11T13:19:50.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:50 smithi165 ceph-mon[33235]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:51.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:50 smithi063 ceph-mon[29809]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:51.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.682+0000 7fada6a00700 1 -- 172.21.15.165:0/2068874523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0103290 msgr2=0x7fada0103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:51.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.682+0000 7fada6a00700 1 --2- 172.21.15.165:0/2068874523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0103290 0x7fada0103670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad90009910 tx=0x7fad90009c20 comp rx=0 tx=0).stop 2024-05-11T13:19:51.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.682+0000 7fada6a00700 1 -- 172.21.15.165:0/2068874523 shutdown_connections 2024-05-11T13:19:51.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.682+0000 7fada6a00700 1 --2- 172.21.15.165:0/2068874523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0103290 0x7fada0103670 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:51.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.682+0000 7fada6a00700 1 -- 172.21.15.165:0/2068874523 >> 172.21.15.165:0/2068874523 conn(0x7fada00feb20 msgr2=0x7fada0100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:51.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.682+0000 7fada6a00700 1 -- 172.21.15.165:0/2068874523 shutdown_connections 2024-05-11T13:19:51.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.682+0000 7fada6a00700 1 -- 172.21.15.165:0/2068874523 wait complete. 2024-05-11T13:19:51.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.684+0000 7fada6a00700 1 Processor -- start 2024-05-11T13:19:51.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.684+0000 7fada6a00700 1 -- start start 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.684+0000 7fada6a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 0x7fada0075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.684+0000 7fada6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0075d90 0x7fada0076210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.684+0000 7fada6a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fada010d3c0 con 0x7fada0075d90 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.684+0000 7fada6a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fada010d240 con 0x7fada0103290 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0075d90 0x7fada0076210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 0x7fada0075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0075d90 0x7fada0076210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.165:46154/0 (socket says 172.21.15.165:46154) 2024-05-11T13:19:51.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9f400700 1 -- 172.21.15.165:0/948669563 learned_addr learned my addr 172.21.15.165:0/948669563 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:19:51.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9fe00700 1 -- 172.21.15.165:0/948669563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 msgr2=0x7fada0075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-11T13:19:51.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9fe00700 1 -- 172.21.15.165:0/948669563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 msgr2=0x7fada0075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-11T13:19:51.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9fe00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 0x7fada0075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T13:19:51.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.685+0000 7fad9fe00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 0x7fada0075850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.686+0000 7fad9f400700 1 -- 172.21.15.165:0/948669563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 msgr2=0x7fada0075850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.686+0000 7fad9f400700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 0x7fada0075850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.686+0000 7fad9f400700 1 -- 172.21.15.165:0/948669563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada0077720 con 0x7fada0075d90 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.686+0000 7fad9f400700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0075d90 0x7fada0076210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00b700 tx=0x7fad8c00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.686+0000 7fad9cc00700 1 -- 172.21.15.165:0/948669563 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fad8c011630 con 0x7fada0075d90 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.687+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01a4e70 con 0x7fada0075d90 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.687+0000 7fad9cc00700 1 -- 172.21.15.165:0/948669563 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad8c011c60 con 0x7fada0075d90 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.687+0000 7fad9cc00700 1 -- 172.21.15.165:0/948669563 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fad8c010c90 con 0x7fada0075d90 2024-05-11T13:19:51.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.687+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01a53d0 con 0x7fada0075d90 2024-05-11T13:19:51.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.688+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada0104770 con 0x7fada0075d90 2024-05-11T13:19:51.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.688+0000 7fad9cc00700 1 -- 172.21.15.165:0/948669563 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fad8c018020 con 0x7fada0075d90 2024-05-11T13:19:51.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.689+0000 7fad9cc00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad8403e300 0x7fad840407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:51.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.689+0000 7fad9cc00700 1 -- 172.21.15.165:0/948669563 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad8c052230 con 0x7fada0075d90 2024-05-11T13:19:51.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.690+0000 7fad9fe00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad8403e300 0x7fad840407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:51.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.691+0000 7fad9fe00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad8403e300 0x7fad840407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b5c0 tx=0x7fad90005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:51.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:51.705+0000 7fad9cc00700 1 -- 172.21.15.165:0/948669563 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad8c01f020 con 0x7fada0075d90 2024-05-11T13:19:52.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.017+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fada01a56e0 con 0x7fada0075d90 2024-05-11T13:19:52.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.018+0000 7fad9cc00700 1 -- 172.21.15.165:0/948669563 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fad8c022d60 con 0x7fada0075d90 2024-05-11T13:19:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:19:52.020 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","modified":"2024-05-11T13:19:42.600437Z","created":"2024-05-11T13:17:18.239963Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-11T13:19:52.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.021+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad8403e300 msgr2=0x7fad840407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:52.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.021+0000 7fada6a00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad8403e300 0x7fad840407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b5c0 tx=0x7fad90005a90 comp rx=0 tx=0).stop 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.021+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0075d90 msgr2=0x7fada0076210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.021+0000 7fada6a00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0075d90 0x7fada0076210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00b700 tx=0x7fad8c00bac0 comp rx=0 tx=0).stop 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.022+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 shutdown_connections 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.022+0000 7fada6a00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad8403e300 0x7fad840407c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.022+0000 7fada6a00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103290 0x7fada0075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.022+0000 7fada6a00700 1 --2- 172.21.15.165:0/948669563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0075d90 0x7fada0076210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.022+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 >> 172.21.15.165:0/948669563 conn(0x7fada00feb20 msgr2=0x7fada0105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.022+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 shutdown_connections 2024-05-11T13:19:52.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:52.022+0000 7fada6a00700 1 -- 172.21.15.165:0/948669563 wait complete. 2024-05-11T13:19:52.024 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-05-11T13:19:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:52 smithi165 ceph-mon[33235]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:52 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/948669563' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:53.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:52 smithi063 ceph-mon[29809]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:52 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/948669563' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:53.278 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 podman[33274]: 2024-05-11 13:19:53.274624106 +0000 UTC m=+0.395946214 container init 48814d2bd8db14edce7d0ec0bb86d4244d10ffac176bbacaa22dd6e88ed71275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD) 2024-05-11T13:19:53.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 podman[33274]: 2024-05-11 13:19:53.286646131 +0000 UTC m=+0.407968236 container start 48814d2bd8db14edce7d0ec0bb86d4244d10ffac176bbacaa22dd6e88ed71275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS) 2024-05-11T13:19:53.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: set uid:gid to 167:167 (ceph:ceph) 2024-05-11T13:19:53.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev), process ceph-mon, pid 2 2024-05-11T13:19:53.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pidfile_write: ignore empty --pid-file 2024-05-11T13:19:53.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: load: jerasure load: lrc 2024-05-11T13:19:53.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: RocksDB version: 7.9.2 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Git sha 0 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Compile date 2024-05-10 10:41:35 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: DB SUMMARY 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: DB Session ID: AKITPDRUB2K9NNGCDL5O 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: CURRENT file: CURRENT 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: IDENTITY file: IDENTITY 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.error_if_exists: 0 2024-05-11T13:19:53.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.create_if_missing: 0 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.paranoid_checks: 1 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.env: 0x55ce7b4b6cc0 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.fs: PosixFileSystem 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.info_log: 0x55ce9b8e7e00 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_file_opening_threads: 16 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.statistics: (nil) 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.use_fsync: 0 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_log_file_size: 0 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-11T13:19:53.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.keep_log_file_num: 1000 2024-05-11T13:19:53.708 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.recycle_log_file_num: 0 2024-05-11T13:19:53.708 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.allow_fallocate: 1 2024-05-11T13:19:53.708 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.allow_mmap_reads: 0 2024-05-11T13:19:53.708 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.allow_mmap_writes: 0 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.use_direct_reads: 0 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.create_missing_column_families: 0 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.db_log_dir: 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.wal_dir: 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-11T13:19:53.709 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.advise_random_on_open: 1 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.db_write_buffer_size: 0 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.write_buffer_manager: 0x55ce9b8b90e0 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.rate_limiter: (nil) 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.wal_recovery_mode: 2 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.enable_thread_tracking: 0 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.enable_pipelined_write: 0 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.unordered_write: 0 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-11T13:19:53.710 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.row_cache: None 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.wal_filter: None 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.allow_ingest_behind: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.two_write_queues: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.manual_wal_flush: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.wal_compression: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.atomic_flush: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-11T13:19:53.711 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.log_readahead_size: 0 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.best_efforts_recovery: 0 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.allow_data_in_errors: 0 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.db_host_id: __hostname__ 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_background_jobs: 2 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_background_compactions: -1 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_subcompactions: 1 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-11T13:19:53.712 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_total_wal_size: 0 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_open_files: -1 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bytes_per_sync: 0 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_readahead_size: 0 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_background_flushes: -1 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Compression algorithms supported: 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-11T13:19:53.713 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kZSTD supported: 0 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kXpressCompression supported: 0 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kLZ4HCCompression supported: 1 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kZlibCompression supported: 1 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kSnappyCompression supported: 1 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kLZ4Compression supported: 1 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: kBZip2Compression supported: 0 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.merge_operator: 2024-05-11T13:19:53.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_filter: None 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_filter_factory: None 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.sst_partitioner_factory: None 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ce9b8e7fc0) 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: index_type: 0 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: data_block_index_type: 0 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: index_shortening: 1 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-11T13:19:53.715 INFO:journalctl@ceph.mon.b.smithi071.stdout: checksum: 4 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: no_block_cache: 0 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache: 0x55ce9b8eb1f0 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache_options: 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: capacity : 536870912 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: num_shard_bits : 4 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: strict_capacity_limit : 0 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache_compressed: (nil) 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: persistent_cache: (nil) 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_size: 4096 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_size_deviation: 10 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_restart_interval: 16 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: index_block_restart_interval: 1 2024-05-11T13:19:53.716 INFO:journalctl@ceph.mon.b.smithi071.stdout: metadata_block_size: 4096 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: partition_filters: 0 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: use_delta_encoding: 1 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: filter_policy: bloomfilter 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: whole_key_filtering: 1 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: verify_compression: 0 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: format_version: 5 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: enable_index_compression: 1 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_align: 0 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: max_auto_readahead_size: 262144 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: prepopulate_block_cache: 0 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: initial_auto_readahead_size: 8192 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-05-11T13:19:53.717 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.write_buffer_size: 33554432 2024-05-11T13:19:53.718 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_write_buffer_number: 2 2024-05-11T13:19:53.718 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression: NoCompression 2024-05-11T13:19:53.718 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression: Disabled 2024-05-11T13:19:53.718 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.prefix_extractor: nullptr 2024-05-11T13:19:53.718 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-11T13:19:53.718 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.num_levels: 7 2024-05-11T13:19:53.718 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-11T13:19:53.719 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-11T13:19:53.719 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-11T13:19:53.719 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.level: 32767 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.strategy: 0 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-11T13:19:53.720 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.enabled: false 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.target_file_size_base: 67108864 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-11T13:19:53.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.arena_block_size: 1048576 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.disable_auto_compactions: 0 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-11T13:19:53.722 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.inplace_update_support: 0 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-11T13:19:53.723 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.bloom_locality: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.max_successive_merges: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.paranoid_file_checks: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.force_consistency_checks: 1 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.report_bg_io_stats: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.ttl: 2592000 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.enable_blob_files: false 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.min_blob_size: 0 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.blob_file_size: 268435456 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-11T13:19:53.724 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.blob_file_starting_level: 0 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 75c971d6-e709-4bb6-b2a0-0cb2e5b99b3b 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433593343854, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433593345075, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715433593, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "75c971d6-e709-4bb6-b2a0-0cb2e5b99b3b", "db_session_id": "AKITPDRUB2K9NNGCDL5O", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715433593345415, "job": 1, "event": "recovery_finished"} 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-11T13:19:53.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ce9ba08000 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: DB pointer 0x55ce9b9f4000 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** DB Stats ** 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T13:19:53.726 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T13:19:53.729 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** Compaction Stats [default] ** 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** Compaction Stats [default] ** 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-11T13:19:53.730 INFO:journalctl@ceph.mon.b.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: Block cache BinnedLRUCache@0x55ce9b8eb1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-11T13:19:53.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: using public_addr v2:172.21.15.71:0/0 -> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: starting mon.b rank -1 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(???) e0 preinit fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).mds e1 new map 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout: e1 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout: legacy client fscid: -1 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout: No filesystems configured 2024-05-11T13:19:53.732 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Added host smithi071 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Deploying cephadm binary to smithi165 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.734 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Added host smithi165 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.735 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2426585924' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2426585924' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.736 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Saving service mon spec with placement smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c;count:3 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:19:53.737 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Deploying daemon mon.c on smithi165 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/1231232441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.a calling monitor election 2024-05-11T13:19:53.738 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/3207614344' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.c calling monitor election 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: monmap epoch 2 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: last_changed 2024-05-11T13:19:42.600437+0000 2024-05-11T13:19:53.739 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: min_mon_release 19 (squid) 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: election_strategy: 1 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: fsmap 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mgrmap e13: x(active, since 68s) 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: overall HEALTH_OK 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:19:53.740 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: Deploying daemon mon.b on smithi071 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/948669563' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 ceph-mon[33293]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 bash[33274]: 48814d2bd8db14edce7d0ec0bb86d4244d10ffac176bbacaa22dd6e88ed71275 2024-05-11T13:19:53.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:53 smithi071 systemd[1]: Started Ceph mon.b for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:19:53.771 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T13:19:53.772 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T13:19:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:19:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:19:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: mon.a calling monitor election 2024-05-11T13:19:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: mon.c calling monitor election 2024-05-11T13:19:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: mon.b calling monitor election 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: monmap epoch 3 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: last_changed 2024-05-11T13:19:53.401188+0000 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: min_mon_release 19 (squid) 2024-05-11T13:19:58.819 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: election_strategy: 1 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: 2: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.b 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: fsmap 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: mgrmap e13: x(active, since 79s) 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: overall HEALTH_OK 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:58.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:19:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: mon.a calling monitor election 2024-05-11T13:19:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: mon.c calling monitor election 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: mon.b calling monitor election 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: monmap epoch 3 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: last_changed 2024-05-11T13:19:53.401188+0000 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:19:58.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: min_mon_release 19 (squid) 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: election_strategy: 1 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: 2: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.b 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: fsmap 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: mgrmap e13: x(active, since 79s) 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: overall HEALTH_OK 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:58.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:19:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: mon.a calling monitor election 2024-05-11T13:19:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:19:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: mon.c calling monitor election 2024-05-11T13:19:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: mon.b calling monitor election 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: monmap epoch 3 2024-05-11T13:19:58.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: last_changed 2024-05-11T13:19:53.401188+0000 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: created 2024-05-11T13:17:18.239963+0000 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: min_mon_release 19 (squid) 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: election_strategy: 1 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: 2: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.b 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: fsmap 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: mgrmap e13: x(active, since 79s) 2024-05-11T13:19:58.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: overall HEALTH_OK 2024-05-11T13:19:58.958 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:58.958 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.184+0000 7f984a200700 1 -- 172.21.15.165:0/3312870433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 msgr2=0x7f982c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:59.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.184+0000 7f984a200700 1 --2- 172.21.15.165:0/3312870433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 0x7f982c005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9838005fa0 tx=0x7f983800ffc0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.184+0000 7f984a200700 1 -- 172.21.15.165:0/3312870433 shutdown_connections 2024-05-11T13:19:59.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.184+0000 7f984a200700 1 --2- 172.21.15.165:0/3312870433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 0x7f982c005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.184+0000 7f984a200700 1 -- 172.21.15.165:0/3312870433 >> 172.21.15.165:0/3312870433 conn(0x7f9844100b50 msgr2=0x7f9844102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:59.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.184+0000 7f984a200700 1 -- 172.21.15.165:0/3312870433 shutdown_connections 2024-05-11T13:19:59.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.184+0000 7f984a200700 1 -- 172.21.15.165:0/3312870433 wait complete. 2024-05-11T13:19:59.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.187+0000 7f984a200700 1 Processor -- start 2024-05-11T13:19:59.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.188+0000 7f984a200700 1 -- start start 2024-05-11T13:19:59.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f984a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 0x7f98441a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:59.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f984a200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98441a0d20 0x7f984419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:59.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f984a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f984419aee0 0x7f984419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:59.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f984a200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f984410ce20 con 0x7f98441052e0 2024-05-11T13:19:59.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f984a200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f984410cca0 con 0x7f984419aee0 2024-05-11T13:19:59.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f984a200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f984410cfa0 con 0x7f98441a0d20 2024-05-11T13:19:59.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f9843e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98441a0d20 0x7f984419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:59.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.189+0000 7f9848e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 0x7f98441a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:59.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9843e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98441a0d20 0x7f984419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58806/0 (socket says 172.21.15.165:58806) 2024-05-11T13:19:59.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9843e00700 1 -- 172.21.15.165:0/3496376085 learned_addr learned my addr 172.21.15.165:0/3496376085 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:19:59.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9849800700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f984419aee0 0x7f984419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:59.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9843e00700 1 -- 172.21.15.165:0/3496376085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f984419aee0 msgr2=0x7f984419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:59.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9843e00700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f984419aee0 0x7f984419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9843e00700 1 -- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 msgr2=0x7f98441a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:59.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9843e00700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 0x7f98441a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9843e00700 1 -- 172.21.15.165:0/3496376085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98441a76b0 con 0x7f98441a0d20 2024-05-11T13:19:59.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.190+0000 7f9849800700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f984419aee0 0x7f984419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:19:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.191+0000 7f9843e00700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98441a0d20 0x7f984419a9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f983000b780 tx=0x7f983000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.191+0000 7f9841600700 1 -- 172.21.15.165:0/3496376085 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830004230 con 0x7f98441a0d20 2024-05-11T13:19:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.191+0000 7f9841600700 1 -- 172.21.15.165:0/3496376085 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98300043d0 con 0x7f98441a0d20 2024-05-11T13:19:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.191+0000 7f9841600700 1 -- 172.21.15.165:0/3496376085 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830011a50 con 0x7f98441a0d20 2024-05-11T13:19:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.192+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98441a78e0 con 0x7f98441a0d20 2024-05-11T13:19:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.192+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a7dd0 con 0x7f98441a0d20 2024-05-11T13:19:59.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.193+0000 7f9841600700 1 -- 172.21.15.165:0/3496376085 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f98300036a0 con 0x7f98441a0d20 2024-05-11T13:19:59.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.194+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9844064bb0 con 0x7f98441a0d20 2024-05-11T13:19:59.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.194+0000 7f9841600700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f982403e9a0 0x7f9824040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:19:59.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.194+0000 7f9841600700 1 -- 172.21.15.165:0/3496376085 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98300523e0 con 0x7f98441a0d20 2024-05-11T13:19:59.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.195+0000 7f9848e00700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f982403e9a0 0x7f9824040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:19:59.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.196+0000 7f9848e00700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f982403e9a0 0x7f9824040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9838001eb0 tx=0x7f983801f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:19:59.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.201+0000 7f9841600700 1 -- 172.21.15.165:0/3496376085 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9830020740 con 0x7f98441a0d20 2024-05-11T13:19:59.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.534+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f98441a8190 con 0x7f98441a0d20 2024-05-11T13:19:59.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.534+0000 7f9841600700 1 -- 172.21.15.165:0/3496376085 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f9830015df0 con 0x7f98441a0d20 2024-05-11T13:19:59.536 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-11T13:19:59.536 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":3,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","modified":"2024-05-11T13:19:53.401188Z","created":"2024-05-11T13:17:18.239963Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-11T13:19:59.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f982403e9a0 msgr2=0x7f9824040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:59.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f982403e9a0 0x7f9824040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9838001eb0 tx=0x7f983801f040 comp rx=0 tx=0).stop 2024-05-11T13:19:59.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98441a0d20 msgr2=0x7f984419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:19:59.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98441a0d20 0x7f984419a9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f983000b780 tx=0x7f983000ba90 comp rx=0 tx=0).stop 2024-05-11T13:19:59.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 shutdown_connections 2024-05-11T13:19:59.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f982403e9a0 0x7f9824040e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f984419aee0 0x7f984419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98441052e0 0x7f98441a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 --2- 172.21.15.165:0/3496376085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98441a0d20 0x7f984419a9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:19:59.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.538+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 >> 172.21.15.165:0/3496376085 conn(0x7f9844100b50 msgr2=0x7f9844108cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:19:59.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.539+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 shutdown_connections 2024-05-11T13:19:59.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:19:59.539+0000 7f984a200700 1 -- 172.21.15.165:0/3496376085 wait complete. 2024-05-11T13:19:59.541 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 3 2024-05-11T13:19:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:19:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:59.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:59.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:19:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:19:59.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:19:59.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:19:59.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:19:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:00.244 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-11T13:20:00.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-11T13:20:00.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:00 smithi063 ceph-mon[29809]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:00.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:00 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/3496376085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:20:00.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:00 smithi063 ceph-mon[29809]: overall HEALTH_OK 2024-05-11T13:20:00.691 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:20:00.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:00 smithi165 ceph-mon[33235]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:00.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:00 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/3496376085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:20:00.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:00 smithi165 ceph-mon[33235]: overall HEALTH_OK 2024-05-11T13:20:00.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:00 smithi071 ceph-mon[33293]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:00.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:00 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/3496376085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T13:20:00.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:00 smithi071 ceph-mon[33293]: overall HEALTH_OK 2024-05-11T13:20:02.045 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:01 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.045 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:01 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.045 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:01 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.045 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:01 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.045 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:01 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:02.045 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:01 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:02.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:01 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:01 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:01 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:01 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:01 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:02.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:01 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:02.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:01 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:01 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:01 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:01 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:01 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:01 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:02.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.840+0000 7febe2a00700 1 -- 172.21.15.63:0/543003425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc1031c0 msgr2=0x7febdc1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:02.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.840+0000 7febe2a00700 1 --2- 172.21.15.63:0/543003425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc1031c0 0x7febdc1035a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7febcc009910 tx=0x7febcc009c20 comp rx=0 tx=0).stop 2024-05-11T13:20:02.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.841+0000 7febe2a00700 1 -- 172.21.15.63:0/543003425 shutdown_connections 2024-05-11T13:20:02.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.841+0000 7febe2a00700 1 --2- 172.21.15.63:0/543003425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc1031c0 0x7febdc1035a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:02.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.841+0000 7febe2a00700 1 -- 172.21.15.63:0/543003425 >> 172.21.15.63:0/543003425 conn(0x7febdc0fea90 msgr2=0x7febdc100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:20:02.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.841+0000 7febe2a00700 1 -- 172.21.15.63:0/543003425 shutdown_connections 2024-05-11T13:20:02.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.841+0000 7febe2a00700 1 -- 172.21.15.63:0/543003425 wait complete. 2024-05-11T13:20:02.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.842+0000 7febe2a00700 1 Processor -- start 2024-05-11T13:20:02.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.842+0000 7febe2a00700 1 -- start start 2024-05-11T13:20:02.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febe2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febdc1031c0 0x7febdc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:02.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febe2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc19ecd0 0x7febdc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:02.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febe2a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febdc198e90 0x7febdc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febe2a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10d2f0 con 0x7febdc19ecd0 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febe2a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10d170 con 0x7febdc1031c0 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febe2a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10d470 con 0x7febdc198e90 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febdb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc19ecd0 0x7febdc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febdb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc19ecd0 0x7febdc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33092/0 (socket says 172.21.15.63:33092) 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febdb400700 1 -- 172.21.15.63:0/165919693 learned_addr learned my addr 172.21.15.63:0/165919693 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.843+0000 7febdbe00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febdc1031c0 0x7febdc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febe0a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febdc198e90 0x7febdc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febdb400700 1 -- 172.21.15.63:0/165919693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febdc198e90 msgr2=0x7febdc199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febdb400700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febdc198e90 0x7febdc199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febdb400700 1 -- 172.21.15.63:0/165919693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febdc1031c0 msgr2=0x7febdc19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febdb400700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febdc1031c0 0x7febdc19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:02.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febdb400700 1 -- 172.21.15.63:0/165919693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc1a5660 con 0x7febdc19ecd0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febe0a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febdc198e90 0x7febdc199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.844+0000 7febdb400700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc19ecd0 0x7febdc198950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7febc800b780 tx=0x7febc800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.845+0000 7febd8c00700 1 -- 172.21.15.63:0/165919693 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8015020 con 0x7febdc19ecd0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.845+0000 7febd8c00700 1 -- 172.21.15.63:0/165919693 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febc8011cc0 con 0x7febdc19ecd0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.845+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc1a5890 con 0x7febdc19ecd0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.845+0000 7febd8c00700 1 -- 172.21.15.63:0/165919693 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8010d00 con 0x7febdc19ecd0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.845+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc1a5e20 con 0x7febdc19ecd0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.846+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febdc064bb0 con 0x7febdc19ecd0 2024-05-11T13:20:02.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.846+0000 7febd8c00700 1 -- 172.21.15.63:0/165919693 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7febc8018020 con 0x7febdc19ecd0 2024-05-11T13:20:02.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.847+0000 7febd8c00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7febc003e9a0 0x7febc0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:02.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.847+0000 7febd8c00700 1 -- 172.21.15.63:0/165919693 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7febc80522e0 con 0x7febdc19ecd0 2024-05-11T13:20:02.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.847+0000 7febdbe00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7febc003e9a0 0x7febc0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:02.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.848+0000 7febdbe00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7febc003e9a0 0x7febc0040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febcc005920 tx=0x7febcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:02.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:02.853+0000 7febd8c00700 1 -- 172.21.15.63:0/165919693 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febc801f020 con 0x7febdc19ecd0 2024-05-11T13:20:02.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-11T13:20:02.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:20:02.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:20:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-11T13:20:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:20:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:20:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.092+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7febdc1a6140 con 0x7febdc19ecd0 2024-05-11T13:20:03.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.093+0000 7febd8c00700 1 -- 172.21.15.63:0/165919693 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7febc8021300 con 0x7febdc19ecd0 2024-05-11T13:20:03.095 INFO:teuthology.orchestra.run.smithi063.stdout:# minimal ceph.conf for a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:20:03.095 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-11T13:20:03.095 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:20:03.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon_host = [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-05-11T13:20:03.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7febc003e9a0 msgr2=0x7febc0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:03.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7febc003e9a0 0x7febc0040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febcc005920 tx=0x7febcc005fb0 comp rx=0 tx=0).stop 2024-05-11T13:20:03.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc19ecd0 msgr2=0x7febdc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:03.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc19ecd0 0x7febdc198950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7febc800b780 tx=0x7febc800bb40 comp rx=0 tx=0).stop 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 shutdown_connections 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7febc003e9a0 0x7febc0040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febdc198e90 0x7febdc199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febdc1031c0 0x7febdc19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 --2- 172.21.15.63:0/165919693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febdc19ecd0 0x7febdc198950 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 >> 172.21.15.63:0/165919693 conn(0x7febdc0fea90 msgr2=0x7febdc104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 shutdown_connections 2024-05-11T13:20:03.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:03.097+0000 7febe2a00700 1 -- 172.21.15.63:0/165919693 wait complete. 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:03.903 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-11T13:20:03.903 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:20:03.903 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.conf 2024-05-11T13:20:03.971 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:20:03.971 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:20:04.064 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.064 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.064 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.064 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/165919693' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:04.065 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/165919693' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:04.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:04.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:04.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:04.073 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:20:04.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/165919693' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.152 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:04.153 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:04.153 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:04.153 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:04.166 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:20:04.166 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:20:04.292 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:20:04.293 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-05-11T13:20:04.384 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:20:04.384 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:20:04.504 INFO:tasks.cephadm:Adding mgr.x on smithi063 2024-05-11T13:20:04.505 INFO:tasks.cephadm:Adding mgr.y on smithi071 2024-05-11T13:20:04.505 INFO:tasks.cephadm:Adding mgr.z on smithi165 2024-05-11T13:20:04.505 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi063=x;smithi071=y;smithi165=z' 2024-05-11T13:20:04.976 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:20:05.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:04 smithi165 ceph-mon[33235]: Reconfiguring mon.a (unknown last config time)... 2024-05-11T13:20:05.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:04 smithi165 ceph-mon[33235]: Reconfiguring daemon mon.a on smithi063 2024-05-11T13:20:05.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:04 smithi165 ceph-mon[33235]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:05.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:04 smithi063 ceph-mon[29809]: Reconfiguring mon.a (unknown last config time)... 2024-05-11T13:20:05.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:04 smithi063 ceph-mon[29809]: Reconfiguring daemon mon.a on smithi063 2024-05-11T13:20:05.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:04 smithi063 ceph-mon[29809]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:05.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:04 smithi071 ceph-mon[33293]: Reconfiguring mon.a (unknown last config time)... 2024-05-11T13:20:05.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:04 smithi071 ceph-mon[33293]: Reconfiguring daemon mon.a on smithi063 2024-05-11T13:20:05.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:04 smithi071 ceph-mon[33293]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: Reconfiguring mon.b (monmap changed)... 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:06.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:06 smithi165 ceph-mon[33235]: Reconfiguring daemon mon.b on smithi071 2024-05-11T13:20:06.647 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:06.647 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:06.647 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:06.647 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: Reconfiguring mon.b (monmap changed)... 2024-05-11T13:20:06.647 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:06.648 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:06.648 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:06.648 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:06 smithi071 ceph-mon[33293]: Reconfiguring daemon mon.b on smithi071 2024-05-11T13:20:06.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:06.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:06.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:06.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: Reconfiguring mon.b (monmap changed)... 2024-05-11T13:20:06.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:06.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:06.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:06.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:06 smithi063 ceph-mon[29809]: Reconfiguring daemon mon.b on smithi071 2024-05-11T13:20:06.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.993+0000 7fa1f5200700 1 -- 172.21.15.165:0/739163738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f01030f0 msgr2=0x7fa1f0103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:06.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.993+0000 7fa1f5200700 1 --2- 172.21.15.165:0/739163738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f01030f0 0x7fa1f0103510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009910 tx=0x7fa1e0009c20 comp rx=0 tx=0).stop 2024-05-11T13:20:06.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.994+0000 7fa1f5200700 1 -- 172.21.15.165:0/739163738 shutdown_connections 2024-05-11T13:20:06.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.994+0000 7fa1f5200700 1 --2- 172.21.15.165:0/739163738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f01030f0 0x7fa1f0103510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:06.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.994+0000 7fa1f5200700 1 -- 172.21.15.165:0/739163738 >> 172.21.15.165:0/739163738 conn(0x7fa1f00fe690 msgr2=0x7fa1f0100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:20:06.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.994+0000 7fa1f5200700 1 -- 172.21.15.165:0/739163738 shutdown_connections 2024-05-11T13:20:06.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.994+0000 7fa1f5200700 1 -- 172.21.15.165:0/739163738 wait complete. 2024-05-11T13:20:06.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.995+0000 7fa1f5200700 1 Processor -- start 2024-05-11T13:20:06.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.995+0000 7fa1f5200700 1 -- start start 2024-05-11T13:20:06.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.995+0000 7fa1f5200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f01030f0 0x7fa1f019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:06.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.995+0000 7fa1f5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f019ae70 0x7fa1f019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:06.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.995+0000 7fa1f5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f019b2f0 0x7fa1f01a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:06.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.996+0000 7fa1f5200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0108e90 con 0x7fa1f019ae70 2024-05-11T13:20:06.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.996+0000 7fa1f5200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0108d10 con 0x7fa1f019b2f0 2024-05-11T13:20:06.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.996+0000 7fa1f5200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0109010 con 0x7fa1f01030f0 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.996+0000 7fa1ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f019ae70 0x7fa1f019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.996+0000 7fa1ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f019ae70 0x7fa1f019fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.165:43574/0 (socket says 172.21.15.165:43574) 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.996+0000 7fa1ed400700 1 -- 172.21.15.165:0/1777903038 learned_addr learned my addr 172.21.15.165:0/1777903038 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.996+0000 7fa1ede00700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f01030f0 0x7fa1f019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1ee800700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f019b2f0 0x7fa1f01a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1ede00700 1 -- 172.21.15.165:0/1777903038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f019b2f0 msgr2=0x7fa1f01a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1ede00700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f019b2f0 0x7fa1f01a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1ede00700 1 -- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f019ae70 msgr2=0x7fa1f019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1ede00700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f019ae70 0x7fa1f019fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1ede00700 1 -- 172.21.15.165:0/1777903038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f01a2d40 con 0x7fa1f01030f0 2024-05-11T13:20:06.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1ede00700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f01030f0 0x7fa1f019a930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005920 tx=0x7fa1e0005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:06.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1e6a00700 1 -- 172.21.15.165:0/1777903038 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0027070 con 0x7fa1f01030f0 2024-05-11T13:20:06.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.997+0000 7fa1e6a00700 1 -- 172.21.15.165:0/1777903038 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1e000bdf0 con 0x7fa1f01030f0 2024-05-11T13:20:06.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.998+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01a2fd0 con 0x7fa1f01030f0 2024-05-11T13:20:06.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.998+0000 7fa1e6a00700 1 -- 172.21.15.165:0/1777903038 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0022ca0 con 0x7fa1f01030f0 2024-05-11T13:20:06.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.998+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01a34a0 con 0x7fa1f01030f0 2024-05-11T13:20:07.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.999+0000 7fa1e6a00700 1 -- 172.21.15.165:0/1777903038 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa1e002a070 con 0x7fa1f01030f0 2024-05-11T13:20:07.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.999+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f002e570 con 0x7fa1f01030f0 2024-05-11T13:20:07.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:06.999+0000 7fa1e6a00700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa1d403e9a0 0x7fa1d4040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:07.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.000+0000 7fa1e6a00700 1 -- 172.21.15.165:0/1777903038 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa1e0053a40 con 0x7fa1f01030f0 2024-05-11T13:20:07.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.000+0000 7fa1ed400700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa1d403e9a0 0x7fa1d4040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:07.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.001+0000 7fa1ed400700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa1d403e9a0 0x7fa1d4040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009b80 tx=0x7fa1dc0092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:07.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.006+0000 7fa1e6a00700 1 -- 172.21.15.165:0/1777903038 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1e0013070 con 0x7fa1f01030f0 2024-05-11T13:20:07.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.254+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi071=y;smithi165=z", "target": ["mon-mgr", ""]}) -- 0x7fa1f0063a20 con 0x7fa1d403e9a0 2024-05-11T13:20:07.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.269+0000 7fa1e6a00700 1 -- 172.21.15.165:0/1777903038 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa1f0063a20 con 0x7fa1d403e9a0 2024-05-11T13:20:07.270 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mgr update... 2024-05-11T13:20:07.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.272+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa1d403e9a0 msgr2=0x7fa1d4040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:07.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.272+0000 7fa1f5200700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa1d403e9a0 0x7fa1d4040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009b80 tx=0x7fa1dc0092c0 comp rx=0 tx=0).stop 2024-05-11T13:20:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.272+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f01030f0 msgr2=0x7fa1f019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.272+0000 7fa1f5200700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f01030f0 0x7fa1f019a930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005920 tx=0x7fa1e0005000 comp rx=0 tx=0).stop 2024-05-11T13:20:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 shutdown_connections 2024-05-11T13:20:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa1d403e9a0 0x7fa1d4040e60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f019b2f0 0x7fa1f01a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:07.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f01030f0 0x7fa1f019a930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:07.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 --2- 172.21.15.165:0/1777903038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1f019ae70 0x7fa1f019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:07.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 >> 172.21.15.165:0/1777903038 conn(0x7fa1f00fe690 msgr2=0x7fa1f00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:20:07.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 shutdown_connections 2024-05-11T13:20:07.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:20:07.273+0000 7fa1f5200700 1 -- 172.21.15.165:0/1777903038 wait complete. 2024-05-11T13:20:07.994 DEBUG:teuthology.orchestra.run.smithi071:mgr.y> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.y.service 2024-05-11T13:20:08.001 DEBUG:teuthology.orchestra.run.smithi165:mgr.z> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.z.service 2024-05-11T13:20:08.008 INFO:tasks.cephadm:Deploying OSDs... 2024-05-11T13:20:08.008 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:20:08.008 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T13:20:08.071 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T13:20:08.072 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-11 13:18:51.420336728 +0000 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-11 13:13:14.726516109 +0000 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-11 13:13:14.726516109 +0000 2024-05-11T13:20:08.126 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-11 13:13:14.726516109 +0000 2024-05-11T13:20:08.126 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T13:20:08.204 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-11T13:20:08.205 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-11T13:20:08.205 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.00036504 s, 1.4 MB/s 2024-05-11T13:20:08.208 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T13:20:08.266 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-05-11T13:20:08.315 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-11 13:18:51.506335903 +0000 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-11 13:13:15.355510131 +0000 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-11 13:13:15.355510131 +0000 2024-05-11T13:20:08.316 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-11 13:13:15.355510131 +0000 2024-05-11T13:20:08.316 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T13:20:08.398 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-11T13:20:08.398 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-11T13:20:08.398 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000378238 s, 1.4 MB/s 2024-05-11T13:20:08.401 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi071=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: Saving service mgr spec with placement smithi063=x;smithi071=y;smithi165=z;count:3 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: Reconfiguring mon.c (monmap changed)... 2024-05-11T13:20:08.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:08.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:08.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:08.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:08 smithi071 ceph-mon[33293]: Reconfiguring daemon mon.c on smithi165 2024-05-11T13:20:08.462 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-05-11T13:20:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-11 13:18:51.569335299 +0000 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-11 13:13:15.976504230 +0000 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-11 13:13:15.976504230 +0000 2024-05-11T13:20:08.519 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-11 13:13:15.976504230 +0000 2024-05-11T13:20:08.519 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T13:20:08.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi071=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:08.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: Saving service mgr spec with placement smithi063=x;smithi071=y;smithi165=z;count:3 2024-05-11T13:20:08.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:08.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: Reconfiguring mon.c (monmap changed)... 2024-05-11T13:20:08.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:08.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:08.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:08.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:08 smithi165 ceph-mon[33235]: Reconfiguring daemon mon.c on smithi165 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi071=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: Saving service mgr spec with placement smithi063=x;smithi071=y;smithi165=z;count:3 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: Reconfiguring mon.c (monmap changed)... 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:08.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:08 smithi063 ceph-mon[29809]: Reconfiguring daemon mon.c on smithi165 2024-05-11T13:20:08.601 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-11T13:20:08.602 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-11T13:20:08.602 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000378997 s, 1.4 MB/s 2024-05-11T13:20:08.606 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T13:20:08.666 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-05-11T13:20:08.720 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-11 13:18:51.622334791 +0000 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-11 13:13:16.622498090 +0000 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-11 13:13:16.622498090 +0000 2024-05-11T13:20:08.721 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-11 13:13:16.622498090 +0000 2024-05-11T13:20:08.722 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T13:20:08.806 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-11T13:20:08.807 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-11T13:20:08.807 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000358307 s, 1.4 MB/s 2024-05-11T13:20:08.810 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T13:20:08.870 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-11T13:20:08.870 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T13:20:08.927 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T13:20:08.927 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-05-11T13:20:08.979 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-11 13:19:52.452396813 +0000 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-11 13:13:05.858397847 +0000 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-11 13:13:05.858397847 +0000 2024-05-11T13:20:08.980 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-05-11 13:13:05.858397847 +0000 2024-05-11T13:20:08.980 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T13:20:09.056 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-11T13:20:09.056 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-11T13:20:09.056 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000360323 s, 1.4 MB/s 2024-05-11T13:20:09.059 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T13:20:09.109 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-11 13:19:52.452396813 +0000 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-11 13:13:06.453387635 +0000 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-11 13:13:06.453387635 +0000 2024-05-11T13:20:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-05-11 13:13:06.453387635 +0000 2024-05-11T13:20:09.157 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T13:20:09.278 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-11T13:20:09.278 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-11T13:20:09.278 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000346697 s, 1.5 MB/s 2024-05-11T13:20:09.282 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T13:20:09.340 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-05-11T13:20:09.391 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T13:20:09.391 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:09.392 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-11T13:20:09.392 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:09.392 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:09.392 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-11 13:19:52.454396778 +0000 2024-05-11T13:20:09.392 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-11 13:13:07.040377560 +0000 2024-05-11T13:20:09.392 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-11 13:13:07.040377560 +0000 2024-05-11T13:20:09.392 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-05-11 13:13:07.040377560 +0000 2024-05-11T13:20:09.392 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T13:20:09.469 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-11T13:20:09.470 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-11T13:20:09.470 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000351056 s, 1.5 MB/s 2024-05-11T13:20:09.474 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T13:20:09.532 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-05-11T13:20:09.578 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T13:20:09.578 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:09.578 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-11T13:20:09.578 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:09.578 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:09.579 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-11 13:19:52.454396778 +0000 2024-05-11T13:20:09.579 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-11 13:13:07.632367399 +0000 2024-05-11T13:20:09.579 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-11 13:13:07.632367399 +0000 2024-05-11T13:20:09.579 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-05-11 13:13:07.632367399 +0000 2024-05-11T13:20:09.579 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T13:20:09.698 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-11T13:20:09.699 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-11T13:20:09.699 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000366122 s, 1.4 MB/s 2024-05-11T13:20:09.702 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T13:20:09.759 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-11T13:20:09.759 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T13:20:09.811 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T13:20:09.811 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-05-11T13:20:09.855 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T13:20:09.855 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:09.855 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-11T13:20:09.856 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:09.856 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:09.856 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-11 13:19:40.491968457 +0000 2024-05-11T13:20:09.856 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-11 13:13:04.957906685 +0000 2024-05-11T13:20:09.856 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-11 13:13:04.957906685 +0000 2024-05-11T13:20:09.856 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-11 13:13:04.957906685 +0000 2024-05-11T13:20:09.856 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T13:20:09.973 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-11T13:20:09.973 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-11T13:20:09.974 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000351883 s, 1.5 MB/s 2024-05-11T13:20:09.977 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T13:20:10.027 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-05-11T13:20:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T13:20:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:10.073 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-11T13:20:10.074 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:10.074 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:10.074 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-11 13:19:40.492968443 +0000 2024-05-11T13:20:10.074 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-11 13:13:05.548898011 +0000 2024-05-11T13:20:10.074 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-11 13:13:05.548898011 +0000 2024-05-11T13:20:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-11 13:13:05.548898011 +0000 2024-05-11T13:20:10.074 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T13:20:10.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-11T13:20:10.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-11T13:20:10.187 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000311354 s, 1.6 MB/s 2024-05-11T13:20:10.190 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T13:20:10.234 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-05-11T13:20:10.314 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-11 13:19:40.493968428 +0000 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-11 13:13:06.138889352 +0000 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-11 13:13:06.138889352 +0000 2024-05-11T13:20:10.315 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-11 13:13:06.138889352 +0000 2024-05-11T13:20:10.315 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T13:20:10.417 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-11T13:20:10.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-11T13:20:10.418 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000310528 s, 1.6 MB/s 2024-05-11T13:20:10.420 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T13:20:10.462 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-11 13:19:40.493968428 +0000 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-11 13:13:06.736880575 +0000 2024-05-11T13:20:10.547 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-11 13:13:06.736880575 +0000 2024-05-11T13:20:10.548 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-11 13:13:06.736880575 +0000 2024-05-11T13:20:10.548 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T13:20:10.660 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-11T13:20:10.660 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-11T13:20:10.660 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000346091 s, 1.5 MB/s 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T13:20:10.661 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:10.662 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:10.662 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:10 smithi165 ceph-mon[33235]: Deploying daemon mgr.z on smithi165 2024-05-11T13:20:10.663 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T13:20:10.715 INFO:tasks.cephadm:Deploying osd.0 on smithi063 with /dev/vg_nvme/lv_4... 2024-05-11T13:20:10.715 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:10.738 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T13:20:10.739 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:10.739 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:10.739 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:10 smithi063 ceph-mon[29809]: Deploying daemon mgr.z on smithi165 2024-05-11T13:20:10.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:10.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:10.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:10 smithi071 ceph-mon[33293]: Deploying daemon mgr.z on smithi165 2024-05-11T13:20:11.181 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:20:12.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:12 smithi063 ceph-mon[29809]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:12.722 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:12 smithi165 ceph-mon[33235]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:12.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:12 smithi071 ceph-mon[33293]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:13.020 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:12 smithi165 podman[35091]: 2024-05-11 13:20:12.970350941 +0000 UTC m=+0.376953588 container start 2186da7408dd913de20435c42f229dd7d9acad00197fe2530ed71a5c007b7ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-11T13:20:13.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:13 smithi165 ceph-mgr[35110]: -- 172.21.15.165:0/813455450 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c19c185a0 con 0x560c19bec800 2024-05-11T13:20:13.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:13 smithi165 bash[35091]: 2186da7408dd913de20435c42f229dd7d9acad00197fe2530ed71a5c007b7ac8 2024-05-11T13:20:13.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:13 smithi165 systemd[1]: Started Ceph mgr.z for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:20:13.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:13.207+0000 7f7017a46200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:20:13.816 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:13.458+0000 7f7017a46200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:20:13.816 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:13.589+0000 7f7017a46200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:20:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-11T13:20:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-11T13:20:13.871 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-11T13:20:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-11T13:20:13.871 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0373452 s, 281 MB/s 2024-05-11T13:20:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-11T13:20:14.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:13.992+0000 7f7017a46200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:20:14.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:14.104+0000 7f7017a46200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:20:14.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:14.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T13:20:14.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:14.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:14.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: Deploying daemon mgr.y on smithi071 2024-05-11T13:20:14.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:14 smithi165 ceph-mon[33235]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:14.494 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: Deploying daemon mgr.y on smithi071 2024-05-11T13:20:14.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:14 smithi071 ceph-mon[33293]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: Deploying daemon mgr.y on smithi071 2024-05-11T13:20:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:14 smithi063 ceph-mon[29809]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:14.740 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_4 2024-05-11T13:20:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:20:16.066 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:15.676+0000 7f7017a46200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:20:16.066 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:15.904+0000 7f7017a46200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:20:16.448 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:16.121+0000 7f7017a46200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:20:16.544 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:16 smithi063 ceph-mon[29809]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:16.684 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:16 smithi071 ceph-mon[33293]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:16.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:16 smithi165 ceph-mon[33235]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:16.816 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:16.500+0000 7f7017a46200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:20:16.816 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:16.622+0000 7f7017a46200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:20:16.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:16 smithi071 podman[34652]: 2024-05-11 13:20:16.93294402 +0000 UTC m=+0.378140043 container init bdd917e054d80db94f8c82d2b3bd1fc8859cee87ba7540f9418733b710e49a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-11T13:20:16.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:16 smithi071 podman[34652]: 2024-05-11 13:20:16.943867989 +0000 UTC m=+0.389063972 container start bdd917e054d80db94f8c82d2b3bd1fc8859cee87ba7540f9418733b710e49a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-11T13:20:17.133 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:16.876+0000 7f7017a46200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:20:17.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:16 smithi071 ceph-mgr[34670]: -- 172.21.15.71:0/3958421796 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f8e12c5a0 con 0x558f8e100800 2024-05-11T13:20:17.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:17 smithi071 bash[34652]: bdd917e054d80db94f8c82d2b3bd1fc8859cee87ba7540f9418733b710e49a3b 2024-05-11T13:20:17.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:17 smithi071 systemd[1]: Started Ceph mgr.y for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:20:17.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:17.179+0000 7f3572fb3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:20:17.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 -- 172.21.15.63:0/1822812691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 msgr2=0x7f6bdc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 --2- 172.21.15.63:0/1822812691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc10d930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400b0a0 tx=0x7f6bd400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 -- 172.21.15.63:0/1822812691 shutdown_connections 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 --2- 172.21.15.63:0/1822812691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc10d930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 --2- 172.21.15.63:0/1822812691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bdc077520 0x7f6bdc075980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 --2- 172.21.15.63:0/1822812691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bdc075f50 0x7f6bdc0763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 -- 172.21.15.63:0/1822812691 >> 172.21.15.63:0/1822812691 conn(0x7f6bdc0feb60 msgr2=0x7f6bdc100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 -- 172.21.15.63:0/1822812691 shutdown_connections 2024-05-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.370+0000 7f6be2400700 1 -- 172.21.15.63:0/1822812691 wait complete. 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.371+0000 7f6be2400700 1 Processor -- start 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6be2400700 1 -- start start 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6be2400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bdc075f50 0x7f6bdc19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6be2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bdc077520 0x7f6bdc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6be2400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc1a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6be2400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc111140 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6be2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc110fc0 con 0x7f6bdc077520 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6be2400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1112c0 con 0x7f6bdc075f50 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.372+0000 7f6bdbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc1a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc1a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41048/0 (socket says 172.21.15.63:41048) 2024-05-11T13:20:17.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdbe00700 1 -- 172.21.15.63:0/227152302 learned_addr learned my addr 172.21.15.63:0/227152302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:20:17.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdb400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bdc075f50 0x7f6bdc19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:17.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdaa00700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bdc077520 0x7f6bdc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:17.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdbe00700 1 -- 172.21.15.63:0/227152302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bdc075f50 msgr2=0x7f6bdc19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:17.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdbe00700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bdc075f50 0x7f6bdc19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:17.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdbe00700 1 -- 172.21.15.63:0/227152302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bdc077520 msgr2=0x7f6bdc19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:17.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdbe00700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bdc077520 0x7f6bdc19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:17.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.373+0000 7f6bdbe00700 1 -- 172.21.15.63:0/227152302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc1a3430 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.374+0000 7f6bdbe00700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc1a2d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400b070 tx=0x7f6bd4009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:17.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.374+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4021070 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.374+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bd4007b80 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.374+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc1a36c0 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.374+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd401b690 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.374+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a3b70 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.375+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6bd4024070 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.375+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bdc06a240 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.376+0000 7f6bd3e00700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6bc003e750 0x7f6bc0040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:17.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.376+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6bd400f020 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.376+0000 7f6bdb400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6bc003e750 0x7f6bc0040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:17.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.377+0000 7f6bdb400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6bc003e750 0x7f6bc0040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc007be0 tx=0x7f6bcc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:17.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.382+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6bd402b020 con 0x7f6bdc10b4a0 2024-05-11T13:20:17.565 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:17.132+0000 7f7017a46200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:20:17.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:17.656+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6bdc063a50 con 0x7f6bc003e750 2024-05-11T13:20:17.703 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:17.446+0000 7f3572fb3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:20:17.704 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:17.586+0000 7f3572fb3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:20:18.255 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:17.955+0000 7f7017a46200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:20:18.256 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:18.002+0000 7f3572fb3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:20:18.256 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:18.120+0000 7f3572fb3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:20:18.565 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:18.255+0000 7f7017a46200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:20:18.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:18.615 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.615 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.615 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:18.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:18.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:18.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:20:18.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:20:18.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:20:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:19.317 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:18.863+0000 7f7017a46200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:20:19.317 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:18.996+0000 7f7017a46200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:20:19.816 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:19.508+0000 7f7017a46200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:20:20.203 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:19.778+0000 7f3572fb3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:20:20.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:20.014+0000 7f3572fb3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:20:20.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:19.957+0000 7f7017a46200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:20:20.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:20.081+0000 7f7017a46200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:20:20.590 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:20.408+0000 7f7017a46200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:20:20.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:20 smithi165 ceph-mon[33235]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:20.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.591 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.638 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:20.233+0000 7f3572fb3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:20:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:20 smithi071 ceph-mon[33293]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.640 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.640 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:20 smithi063 ceph-mon[29809]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:20.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:20.881 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:20.588+0000 7f7017a46200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:20:20.882 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:20.717+0000 7f7017a46200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:20:20.953 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:20.636+0000 7f3572fb3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:20:20.954 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:20.761+0000 7f3572fb3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:20:21.378 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:21.016+0000 7f3572fb3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:20:21.378 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:21.263+0000 7f3572fb3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:20:21.544 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:21.146+0000 7f7017a46200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:20:21.544 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:21.274+0000 7f7017a46200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:20:21.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:21 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:21.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:21 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:21.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:21 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:21.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:21 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:21.545 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:21 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:21.545 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:21 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:21.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:21 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:21.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:21 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:21.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:21 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:21.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:21 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:21.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:21 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:21.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:21 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:21.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:21 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:21.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:21 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:21.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:21 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:21.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:21 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T13:20:21.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:21 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T13:20:21.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:21 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:22.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:22.064+0000 7f7017a46200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:20:22.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:22 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:22.055+0000 7f3572fb3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:20:22.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:22.587+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f6bd4005040 con 0x7f6bdc10b4a0 2024-05-11T13:20:22.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: Reconfiguring mgr.x (unknown last config time)... 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: Reconfiguring daemon mgr.x on smithi063 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/140496442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22f2805e-feb6-4126-b7f7-d4567818ce77"}]: dispatch 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/140496442' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22f2805e-feb6-4126-b7f7-d4567818ce77"}]': finished 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: osdmap e5: 1 total, 0 up, 1 in 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: Standby manager daemon z started 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T13:20:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:22 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:20:22.703 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:22 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:22.339+0000 7f3572fb3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: Reconfiguring mgr.x (unknown last config time)... 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: Reconfiguring daemon mgr.x on smithi063 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/140496442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22f2805e-feb6-4126-b7f7-d4567818ce77"}]: dispatch 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/140496442' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22f2805e-feb6-4126-b7f7-d4567818ce77"}]': finished 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: osdmap e5: 1 total, 0 up, 1 in 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: Standby manager daemon z started 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T13:20:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:22 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:20:22.816 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:20:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:20:22.356+0000 7f7017a46200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: Reconfiguring mgr.x (unknown last config time)... 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: Reconfiguring daemon mgr.x on smithi063 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/140496442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22f2805e-feb6-4126-b7f7-d4567818ce77"}]: dispatch 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/140496442' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22f2805e-feb6-4126-b7f7-d4567818ce77"}]': finished 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: osdmap e5: 1 total, 0 up, 1 in 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: Standby manager daemon z started 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T13:20:22.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:22 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/3458584978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:20:23.203 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:22 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:22.892+0000 7f3572fb3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:20:23.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:23 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:23.019+0000 7f3572fb3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:20:23.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:23 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1190798698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:20:23.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:23 smithi063 ceph-mon[29809]: mgrmap e14: x(active, since 103s), standbys: z 2024-05-11T13:20:23.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:23 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T13:20:23.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:23 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1190798698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:20:23.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:23 smithi071 ceph-mon[33293]: mgrmap e14: x(active, since 103s), standbys: z 2024-05-11T13:20:23.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:23 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T13:20:23.704 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:23 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:23.484+0000 7f3572fb3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:20:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:23 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1190798698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:20:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:23 smithi165 ceph-mon[33235]: mgrmap e14: x(active, since 103s), standbys: z 2024-05-11T13:20:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:23 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T13:20:24.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:23 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:23.935+0000 7f3572fb3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:20:24.204 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:24 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:24.058+0000 7f3572fb3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:20:24.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:24 smithi063 ceph-mon[29809]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:24.686 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:24 smithi071 ceph-mon[33293]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:24.686 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:24 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:24.379+0000 7f3572fb3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:20:24.686 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:24 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:24.559+0000 7f3572fb3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:20:24.686 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:24 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:24.684+0000 7f3572fb3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:20:24.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:24 smithi165 ceph-mon[33235]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:25.454 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:25 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:25.080+0000 7f3572fb3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:20:25.454 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:25 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:25.202+0000 7f3572fb3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:20:26.238 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:25 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:25.955+0000 7f3572fb3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:20:26.239 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:20:26 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:20:26.237+0000 7f3572fb3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:20:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:26.437+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f6bd402e350 con 0x7f6bdc10b4a0 2024-05-11T13:20:26.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:26 smithi063 ceph-mon[29809]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:26.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:26 smithi063 ceph-mon[29809]: Standby manager daemon y started 2024-05-11T13:20:26.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:26 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T13:20:26.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:26 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:20:26.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:26 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T13:20:26.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:26 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:20:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:26 smithi071 ceph-mon[33293]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:26 smithi071 ceph-mon[33293]: Standby manager daemon y started 2024-05-11T13:20:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:26 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T13:20:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:26 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:20:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:26 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T13:20:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:26 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:20:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:26 smithi165 ceph-mon[33235]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:26 smithi165 ceph-mon[33235]: Standby manager daemon y started 2024-05-11T13:20:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:26 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T13:20:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:26 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:20:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:26 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T13:20:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:26 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3185058042' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:20:27.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:27 smithi063 ceph-mon[29809]: mgrmap e15: x(active, since 107s), standbys: z, y 2024-05-11T13:20:27.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T13:20:27.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:27 smithi071 ceph-mon[33293]: mgrmap e15: x(active, since 107s), standbys: z, y 2024-05-11T13:20:27.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T13:20:27.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:27 smithi165 ceph-mon[33235]: mgrmap e15: x(active, since 107s), standbys: z, y 2024-05-11T13:20:27.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T13:20:28.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:28 smithi063 ceph-mon[29809]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:28.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:28.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:28.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:28.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:28 smithi071 ceph-mon[33293]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:28.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:28.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:28.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:28.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:28 smithi165 ceph-mon[33235]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:28.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:28.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:28.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:30 smithi063 ceph-mon[29809]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:30.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:30.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:30.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:30.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:30 smithi071 ceph-mon[33293]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:30.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:30.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:30 smithi165 ceph-mon[33235]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:32.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:32 smithi071 ceph-mon[33293]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:32.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:32 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-11T13:20:32.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:32 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:32.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:32 smithi071 ceph-mon[33293]: Deploying daemon osd.0 on smithi063 2024-05-11T13:20:32.745 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:32 smithi063 ceph-mon[29809]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:32.745 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-11T13:20:32.745 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:32.745 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:32 smithi063 ceph-mon[29809]: Deploying daemon osd.0 on smithi063 2024-05-11T13:20:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:32 smithi165 ceph-mon[33235]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:32 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-11T13:20:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:32 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:32 smithi165 ceph-mon[33235]: Deploying daemon osd.0 on smithi063 2024-05-11T13:20:34.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:34 smithi071 ceph-mon[33293]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:34.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:34 smithi063 ceph-mon[29809]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:34.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:34 smithi165 ceph-mon[33235]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:36.664 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:36 smithi063 ceph-mon[29809]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:36.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:36 smithi165 ceph-mon[33235]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:36.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:36 smithi071 ceph-mon[33293]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:38 smithi063 ceph-mon[29809]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:38.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:38 smithi165 ceph-mon[33235]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:38.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:38 smithi071 ceph-mon[33293]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:40.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:40 smithi071 ceph-mon[33293]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:40.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:40.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:40.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:40 smithi165 ceph-mon[33235]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:40.918 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:40 smithi063 ceph-mon[29809]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:40.918 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:40.918 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:40.918 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:42.519 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:42 smithi063 ceph-mon[29809]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:42.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:42 smithi165 ceph-mon[33235]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:42.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:42 smithi071 ceph-mon[33293]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:44 smithi063 ceph-mon[29809]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:44 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:44 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:44 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:44 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:44 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:44.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:44 smithi063 ceph-mon[29809]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-11T13:20:44.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.834+0000 7f6bd3e00700 1 -- 172.21.15.63:0/227152302 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6bdc063a50 con 0x7f6bc003e750 2024-05-11T13:20:44.835 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 0 on host 'smithi063' 2024-05-11T13:20:44.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.838+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6bc003e750 msgr2=0x7f6bc0040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:44.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.838+0000 7f6be2400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6bc003e750 0x7f6bc0040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc007be0 tx=0x7f6bcc0073d0 comp rx=0 tx=0).stop 2024-05-11T13:20:44.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.838+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 msgr2=0x7f6bdc1a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:44.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.838+0000 7f6be2400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc1a2d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400b070 tx=0x7f6bd4009c10 comp rx=0 tx=0).stop 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 shutdown_connections 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6bc003e750 0x7f6bc0040c10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bdc10b4a0 0x7f6bdc1a2d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bdc075f50 0x7f6bdc19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 --2- 172.21.15.63:0/227152302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bdc077520 0x7f6bdc19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 >> 172.21.15.63:0/227152302 conn(0x7f6bdc0feb60 msgr2=0x7f6bdc1073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 shutdown_connections 2024-05-11T13:20:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:20:44.839+0000 7f6be2400700 1 -- 172.21.15.63:0/227152302 wait complete. 2024-05-11T13:20:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:44 smithi071 ceph-mon[33293]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:44 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:44 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:44 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:44 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:44 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:44.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:44 smithi071 ceph-mon[33293]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-11T13:20:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:44 smithi165 ceph-mon[33235]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:44 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:44 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:44 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:44 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:44 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:44 smithi165 ceph-mon[33235]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-11T13:20:45.684 DEBUG:teuthology.orchestra.run.smithi063:osd.0> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.0.service 2024-05-11T13:20:45.688 INFO:tasks.cephadm:Deploying osd.1 on smithi071 with /dev/vg_nvme/lv_4... 2024-05-11T13:20:45.688 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-11T13:20:45.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:45 smithi063 ceph-mon[29809]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-11T13:20:45.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:45 smithi063 ceph-mon[29809]: osdmap e6: 1 total, 0 up, 1 in 2024-05-11T13:20:45.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:45 smithi063 ceph-mon[29809]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:20:45.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:45 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:45.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:45 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:45.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:45 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:45.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:45 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:45.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:45 smithi071 ceph-mon[33293]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-11T13:20:45.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:45 smithi071 ceph-mon[33293]: osdmap e6: 1 total, 0 up, 1 in 2024-05-11T13:20:45.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:45 smithi071 ceph-mon[33293]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:20:45.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:45 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:45.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:45 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:45.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:45 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:45.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:45 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:45 smithi165 ceph-mon[33235]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-11T13:20:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:45 smithi165 ceph-mon[33235]: osdmap e6: 1 total, 0 up, 1 in 2024-05-11T13:20:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:45 smithi165 ceph-mon[33235]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:20:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:45 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:45 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:45 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:45 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:20:46.129 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:20:46.619 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:20:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0[41949]: 2024-05-11T13:20:46.508+0000 7f72f8a00700 -1 osd.0 0 waiting for initial osdmap 2024-05-11T13:20:46.620 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:20:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0[41949]: 2024-05-11T13:20:46.527+0000 7f72ee800700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:20:46.873 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:46 smithi063 ceph-mon[29809]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:46.873 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:46 smithi063 ceph-mon[29809]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:20:46.873 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:46 smithi063 ceph-mon[29809]: osdmap e7: 1 total, 0 up, 1 in 2024-05-11T13:20:46.873 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:46.873 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:46.874 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:46 smithi063 ceph-mon[29809]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' 2024-05-11T13:20:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:46 smithi071 ceph-mon[33293]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:46 smithi071 ceph-mon[33293]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:20:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:46 smithi071 ceph-mon[33293]: osdmap e7: 1 total, 0 up, 1 in 2024-05-11T13:20:46.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:46.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:46.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:46 smithi071 ceph-mon[33293]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' 2024-05-11T13:20:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:46 smithi165 ceph-mon[33235]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:46 smithi165 ceph-mon[33235]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:20:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:46 smithi165 ceph-mon[33235]: osdmap e7: 1 total, 0 up, 1 in 2024-05-11T13:20:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:46 smithi165 ceph-mon[33235]: from='osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800]' entity='osd.0' 2024-05-11T13:20:47.721 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:47 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:20:47.721 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:47 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:20:47.721 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:47.721 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:47 smithi063 ceph-mon[29809]: osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] boot 2024-05-11T13:20:47.721 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:47 smithi063 ceph-mon[29809]: osdmap e8: 1 total, 1 up, 1 in 2024-05-11T13:20:47.721 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:47.779 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:47 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:20:47.779 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:47 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:20:47.779 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:47 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:47.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:47 smithi071 ceph-mon[33293]: osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] boot 2024-05-11T13:20:47.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:47 smithi071 ceph-mon[33293]: osdmap e8: 1 total, 1 up, 1 in 2024-05-11T13:20:47.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:47 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:48.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:47 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:20:48.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:47 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:20:48.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:48.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:47 smithi165 ceph-mon[33235]: osd.0 [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] boot 2024-05-11T13:20:48.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:47 smithi165 ceph-mon[33235]: osdmap e8: 1 total, 1 up, 1 in 2024-05-11T13:20:48.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:20:48.934 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:48 smithi071 ceph-mon[33293]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:48.934 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-11T13:20:48.934 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-11T13:20:48.935 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-11T13:20:48.935 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-11T13:20:48.935 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0356393 s, 294 MB/s 2024-05-11T13:20:48.935 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-11T13:20:48.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:48 smithi063 ceph-mon[29809]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:48 smithi165 ceph-mon[33235]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T13:20:49.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-05-11T13:20:49.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:49 smithi063 ceph-mon[29809]: osdmap e9: 1 total, 1 up, 1 in 2024-05-11T13:20:49.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:49 smithi071 ceph-mon[33293]: osdmap e9: 1 total, 1 up, 1 in 2024-05-11T13:20:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:49 smithi165 ceph-mon[33235]: osdmap e9: 1 total, 1 up, 1 in 2024-05-11T13:20:50.136 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:20:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:20:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:50.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:20:50.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:50.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:50.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:20:50.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:20:51.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:51 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi063 2024-05-11T13:20:51.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:51 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi063 to 17089M 2024-05-11T13:20:51.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:51 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi063 2024-05-11T13:20:51.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:51 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi063 to 17089M 2024-05-11T13:20:52.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:51 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi063 2024-05-11T13:20:52.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:51 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi063 to 17089M 2024-05-11T13:20:52.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 -- 172.21.15.71:0/3006180093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 msgr2=0x7feda0105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:52.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 --2- 172.21.15.71:0/3006180093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 0x7feda0105c10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed940099c0 tx=0x7fed94009cd0 comp rx=0 tx=0).stop 2024-05-11T13:20:52.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 -- 172.21.15.71:0/3006180093 shutdown_connections 2024-05-11T13:20:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 --2- 172.21.15.71:0/3006180093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 0x7feda010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 --2- 172.21.15.71:0/3006180093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 0x7feda0105c10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 --2- 172.21.15.71:0/3006180093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda01069f0 0x7feda0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 -- 172.21.15.71:0/3006180093 >> 172.21.15.71:0/3006180093 conn(0x7feda0100db0 msgr2=0x7feda01031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:20:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 -- 172.21.15.71:0/3006180093 shutdown_connections 2024-05-11T13:20:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.148+0000 7feda7c00700 1 -- 172.21.15.71:0/3006180093 wait complete. 2024-05-11T13:20:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.150+0000 7feda7c00700 1 Processor -- start 2024-05-11T13:20:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.150+0000 7feda7c00700 1 -- start start 2024-05-11T13:20:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.150+0000 7feda7c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 0x7feda019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.150+0000 7feda7c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda01069f0 0x7feda019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.150+0000 7feda7c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 0x7feda01a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7feda7c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feda0111130 con 0x7feda01069f0 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7feda7c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feda0110fb0 con 0x7feda01057f0 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7feda7c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feda01112b0 con 0x7feda0107390 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7feda5c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 0x7feda01a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7fed9fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda01069f0 0x7feda019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7feda5c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 0x7feda01a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.71:46614/0 (socket says 172.21.15.71:46614) 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7feda5c00700 1 -- 172.21.15.71:0/363108225 learned_addr learned my addr 172.21.15.71:0/363108225 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.151+0000 7feda5200700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 0x7feda019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda5200700 1 -- 172.21.15.71:0/363108225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 msgr2=0x7feda01a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda5200700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 0x7feda01a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda5200700 1 -- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda01069f0 msgr2=0x7feda019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda5200700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda01069f0 0x7feda019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:20:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda5200700 1 -- 172.21.15.71:0/363108225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda01a3490 con 0x7feda01057f0 2024-05-11T13:20:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda5c00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 0x7feda01a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:20:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda5200700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 0x7feda019b830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed940036f0 tx=0x7fed9400f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7fed9d600700 1 -- 172.21.15.71:0/363108225 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94009e60 con 0x7feda01057f0 2024-05-11T13:20:52.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7fed9d600700 1 -- 172.21.15.71:0/363108225 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed94005940 con 0x7feda01057f0 2024-05-11T13:20:52.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7fed9d600700 1 -- 172.21.15.71:0/363108225 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940053c0 con 0x7feda01057f0 2024-05-11T13:20:52.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.152+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda01a3720 con 0x7feda01057f0 2024-05-11T13:20:52.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.153+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda01a3c50 con 0x7feda01057f0 2024-05-11T13:20:52.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.154+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda006a240 con 0x7feda01057f0 2024-05-11T13:20:52.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.158+0000 7fed9d600700 1 -- 172.21.15.71:0/363108225 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fed94005560 con 0x7feda01057f0 2024-05-11T13:20:52.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.159+0000 7fed9d600700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fed880b2870 0x7fed880b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:20:52.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.159+0000 7fed9d600700 1 -- 172.21.15.71:0/363108225 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fed940e15b0 con 0x7feda01057f0 2024-05-11T13:20:52.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.160+0000 7fed9fe00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fed880b2870 0x7fed880b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:20:52.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.161+0000 7fed9fe00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fed880b2870 0x7fed880b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009c60 tx=0x7fed8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:20:52.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.163+0000 7fed9d600700 1 -- 172.21.15.71:0/363108225 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fed94013070 con 0x7feda01057f0 2024-05-11T13:20:52.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:20:52.403+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7feda0063990 con 0x7fed880b2870 2024-05-11T13:20:52.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:52 smithi165 ceph-mon[33235]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:52.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:20:52.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:20:52.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:52 smithi063 ceph-mon[29809]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:20:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:20:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:52 smithi071 ceph-mon[33293]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:20:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:20:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:20:53.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:53 smithi063 ceph-mon[29809]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:53.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:53 smithi071 ceph-mon[33293]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:54.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:53 smithi165 ceph-mon[33235]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:20:54.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:54 smithi063 ceph-mon[29809]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:54.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:54 smithi071 ceph-mon[33293]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:55.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:54 smithi165 ceph-mon[33235]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:56 smithi071 ceph-mon[33293]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:56 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/2029737127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]: dispatch 2024-05-11T13:20:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:56 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]: dispatch 2024-05-11T13:20:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:56 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]': finished 2024-05-11T13:20:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:56 smithi071 ceph-mon[33293]: osdmap e10: 2 total, 1 up, 2 in 2024-05-11T13:20:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:20:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:56 smithi165 ceph-mon[33235]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:56 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/2029737127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]: dispatch 2024-05-11T13:20:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:56 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]: dispatch 2024-05-11T13:20:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:56 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]': finished 2024-05-11T13:20:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:56 smithi165 ceph-mon[33235]: osdmap e10: 2 total, 1 up, 2 in 2024-05-11T13:20:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:20:57.186 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:56 smithi063 ceph-mon[29809]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:57.186 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:56 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/2029737127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]: dispatch 2024-05-11T13:20:57.186 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:56 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]: dispatch 2024-05-11T13:20:57.186 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:56 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29a3b52e-3ed5-403c-b9c4-0a69b8d1b989"}]': finished 2024-05-11T13:20:57.187 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:56 smithi063 ceph-mon[29809]: osdmap e10: 2 total, 1 up, 2 in 2024-05-11T13:20:57.187 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:20:57.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:57 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/540104974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:20:58.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:57 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/540104974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:20:58.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:57 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/540104974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:20:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:20:58 smithi165 ceph-mon[33235]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:59.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:20:58 smithi063 ceph-mon[29809]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:20:59.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:20:58 smithi071 ceph-mon[33293]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:00 smithi165 ceph-mon[33235]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:01.163 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:00 smithi071 ceph-mon[33293]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:01.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:00 smithi063 ceph-mon[29809]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:02 smithi165 ceph-mon[33235]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:03.161 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:02 smithi071 ceph-mon[33293]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:03.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:02 smithi063 ceph-mon[29809]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:05.017 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:04 smithi071 ceph-mon[33293]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:05.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:04 smithi165 ceph-mon[33235]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:05.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:04 smithi063 ceph-mon[29809]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:06.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:05 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-11T13:21:06.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:05 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:06.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:05 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-11T13:21:06.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:05 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:06.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:05 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-11T13:21:06.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:05 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:07.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:06 smithi165 ceph-mon[33235]: Deploying daemon osd.1 on smithi071 2024-05-11T13:21:07.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:06 smithi165 ceph-mon[33235]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:07.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:06 smithi071 ceph-mon[33293]: Deploying daemon osd.1 on smithi071 2024-05-11T13:21:07.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:06 smithi071 ceph-mon[33293]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:07.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:06 smithi063 ceph-mon[29809]: Deploying daemon osd.1 on smithi071 2024-05-11T13:21:07.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:06 smithi063 ceph-mon[29809]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:08.923 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:08 smithi071 ceph-mon[33293]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:09.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:08 smithi165 ceph-mon[33235]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:09.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:08 smithi063 ceph-mon[29809]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:10.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:10 smithi071 ceph-mon[33293]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:11.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:10 smithi165 ceph-mon[33235]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:11.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:10 smithi063 ceph-mon[29809]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:13.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:12 smithi165 ceph-mon[33235]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:13.133 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:12 smithi071 ceph-mon[33293]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:13.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:12 smithi063 ceph-mon[29809]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:14.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:14.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:14.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:14 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:14.576 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:14 smithi071 ceph-mon[33293]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:14 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:14.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:14 smithi063 ceph-mon[29809]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:14 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:14 smithi165 ceph-mon[33235]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:16.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:16 smithi071 ceph-mon[33293]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:16.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:16 smithi165 ceph-mon[33235]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:16.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:16 smithi063 ceph-mon[29809]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:17.631 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.631 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.631 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:17.631 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:17.631 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.632 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:17 smithi071 ceph-mon[33293]: from='osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T13:21:17.632 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:17 smithi071 ceph-mon[33293]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T13:21:17.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:17.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:17.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:17 smithi165 ceph-mon[33235]: from='osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T13:21:17.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:17 smithi165 ceph-mon[33235]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T13:21:17.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:17 smithi063 ceph-mon[29809]: from='osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T13:21:17.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:17 smithi063 ceph-mon[29809]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T13:21:18.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.512+0000 7fed9d600700 1 -- 172.21.15.71:0/363108225 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feda0063990 con 0x7fed880b2870 2024-05-11T13:21:18.513 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 1 on host 'smithi071' 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fed880b2870 msgr2=0x7fed880b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fed880b2870 0x7fed880b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009c60 tx=0x7fed8c009260 comp rx=0 tx=0).stop 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 msgr2=0x7feda019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 0x7feda019b830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed940036f0 tx=0x7fed9400f710 comp rx=0 tx=0).stop 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 shutdown_connections 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fed880b2870 0x7fed880b4d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda0107390 0x7feda01a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda01057f0 0x7feda019b830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 --2- 172.21.15.71:0/363108225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda01069f0 0x7feda019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 >> 172.21.15.71:0/363108225 conn(0x7feda0100db0 msgr2=0x7feda0109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 shutdown_connections 2024-05-11T13:21:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:21:18.515+0000 7feda7c00700 1 -- 172.21.15.71:0/363108225 wait complete. 2024-05-11T13:21:18.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:18.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-11T13:21:18.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: osdmap e11: 2 total, 1 up, 2 in 2024-05-11T13:21:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: from='osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:21:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:21:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:18.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:18 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:18.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:18.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-11T13:21:18.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: osdmap e11: 2 total, 1 up, 2 in 2024-05-11T13:21:18.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:18.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: from='osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:21:18.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:21:18.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:18.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:18.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:18 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-11T13:21:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: osdmap e11: 2 total, 1 up, 2 in 2024-05-11T13:21:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: from='osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:21:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:21:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:19.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:19.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:18 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:19.357 DEBUG:teuthology.orchestra.run.smithi071:osd.1> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.1.service 2024-05-11T13:21:19.362 INFO:tasks.cephadm:Deploying osd.2 on smithi165 with /dev/vg_nvme/lv_4... 2024-05-11T13:21:19.362 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-11T13:21:19.603 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1[37852]: 2024-05-11T13:21:19.495+0000 7f37b1e00700 -1 osd.1 0 waiting for initial osdmap 2024-05-11T13:21:19.604 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1[37852]: 2024-05-11T13:21:19.514+0000 7f37a7c00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:21:19.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:19 smithi165 ceph-mon[33235]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:21:19.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:19 smithi165 ceph-mon[33235]: osdmap e12: 2 total, 1 up, 2 in 2024-05-11T13:21:19.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:19 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:19 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:19 smithi165 ceph-mon[33235]: from='osd.1 ' entity='osd.1' 2024-05-11T13:21:19.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:19 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.817 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:21:19.929 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:19 smithi071 ceph-mon[33293]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:21:19.929 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:19 smithi071 ceph-mon[33293]: osdmap e12: 2 total, 1 up, 2 in 2024-05-11T13:21:19.929 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:19 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.929 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:19 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.929 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:19 smithi071 ceph-mon[33293]: from='osd.1 ' entity='osd.1' 2024-05-11T13:21:19.929 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:19 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:19 smithi063 ceph-mon[29809]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:21:19.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:19 smithi063 ceph-mon[29809]: osdmap e12: 2 total, 1 up, 2 in 2024-05-11T13:21:19.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:19 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:19 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:19.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:19 smithi063 ceph-mon[29809]: from='osd.1 ' entity='osd.1' 2024-05-11T13:21:19.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:19 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:20 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:21:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:20 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:21:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:20 smithi071 ceph-mon[33293]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:20.639 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:20 smithi071 ceph-mon[33293]: osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] boot 2024-05-11T13:21:20.640 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:20 smithi071 ceph-mon[33293]: osdmap e13: 2 total, 2 up, 2 in 2024-05-11T13:21:20.640 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:20 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:21:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:20 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:21:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:20 smithi063 ceph-mon[29809]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:20.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:20 smithi063 ceph-mon[29809]: osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] boot 2024-05-11T13:21:20.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:20 smithi063 ceph-mon[29809]: osdmap e13: 2 total, 2 up, 2 in 2024-05-11T13:21:20.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:20.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:20 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:21:20.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:20 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:21:20.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:20 smithi165 ceph-mon[33235]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-11T13:21:20.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:20 smithi165 ceph-mon[33235]: osd.1 [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] boot 2024-05-11T13:21:20.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:20 smithi165 ceph-mon[33235]: osdmap e13: 2 total, 2 up, 2 in 2024-05-11T13:21:20.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:21:22.436 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-11T13:21:22.436 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-11T13:21:22.436 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-11T13:21:22.436 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-11T13:21:22.437 INFO:teuthology.orchestra.run.smithi165.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0330443 s, 317 MB/s 2024-05-11T13:21:22.437 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-11T13:21:22.811 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:22 smithi165 ceph-mon[33235]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:22.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:22 smithi165 ceph-mon[33235]: osdmap e14: 2 total, 2 up, 2 in 2024-05-11T13:21:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:22 smithi063 ceph-mon[29809]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:22 smithi063 ceph-mon[29809]: osdmap e14: 2 total, 2 up, 2 in 2024-05-11T13:21:22.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:22 smithi071 ceph-mon[33293]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:22.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:22 smithi071 ceph-mon[33293]: osdmap e14: 2 total, 2 up, 2 in 2024-05-11T13:21:23.224 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-05-11T13:21:23.638 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:21:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:21:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:24.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:24.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:24.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:24.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:25.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 -- 172.21.15.165:0/3580401052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c1047b0 msgr2=0x7f476c104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:25.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 --2- 172.21.15.165:0/3580401052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c1047b0 0x7f476c104c10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4758009990 tx=0x7f4758009ca0 comp rx=0 tx=0).stop 2024-05-11T13:21:25.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 -- 172.21.15.165:0/3580401052 shutdown_connections 2024-05-11T13:21:25.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 --2- 172.21.15.165:0/3580401052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f476c105150 0x7f476c10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:25.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 --2- 172.21.15.165:0/3580401052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f476c1035b0 0x7f476c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:25.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 --2- 172.21.15.165:0/3580401052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c1047b0 0x7f476c104c10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:25.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 -- 172.21.15.165:0/3580401052 >> 172.21.15.165:0/3580401052 conn(0x7f476c0feb50 msgr2=0x7f476c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:21:25.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 -- 172.21.15.165:0/3580401052 shutdown_connections 2024-05-11T13:21:25.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.456+0000 7f4772a00700 1 -- 172.21.15.165:0/3580401052 wait complete. 2024-05-11T13:21:25.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.457+0000 7f4772a00700 1 Processor -- start 2024-05-11T13:21:25.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.457+0000 7f4772a00700 1 -- start start 2024-05-11T13:21:25.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4772a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f476c1035b0 0x7f476c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:25.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4772a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f476c1047b0 0x7f476c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:25.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4772a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c105150 0x7f476c1a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:25.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4772a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10ef40 con 0x7f476c1035b0 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4772a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10edc0 con 0x7f476c1047b0 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4772a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10f0c0 con 0x7f476c105150 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4770a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c105150 0x7f476c1a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4770a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c105150 0x7f476c1a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54568/0 (socket says 172.21.15.165:54568) 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f4770a00700 1 -- 172.21.15.165:0/369876941 learned_addr learned my addr 172.21.15.165:0/369876941 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f476be00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f476c1035b0 0x7f476c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.458+0000 7f476b400700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f476c1047b0 0x7f476c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4770a00700 1 -- 172.21.15.165:0/369876941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f476c1047b0 msgr2=0x7f476c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4770a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f476c1047b0 0x7f476c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4770a00700 1 -- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f476c1035b0 msgr2=0x7f476c19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4770a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f476c1035b0 0x7f476c19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:25.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4770a00700 1 -- 172.21.15.165:0/369876941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1a3450 con 0x7f476c105150 2024-05-11T13:21:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f476be00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f476c1035b0 0x7f476c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:21:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4770a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c105150 0x7f476c1a2db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4764007d40 tx=0x7f476400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:21:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4768c00700 1 -- 172.21.15.165:0/369876941 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764016020 con 0x7f476c105150 2024-05-11T13:21:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4768c00700 1 -- 172.21.15.165:0/369876941 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f476400ad60 con 0x7f476c105150 2024-05-11T13:21:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4768c00700 1 -- 172.21.15.165:0/369876941 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640115e0 con 0x7f476c105150 2024-05-11T13:21:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.459+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1a3740 con 0x7f476c105150 2024-05-11T13:21:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.460+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a3c30 con 0x7f476c105150 2024-05-11T13:21:25.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.461+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c06a240 con 0x7f476c105150 2024-05-11T13:21:25.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.463+0000 7f4768c00700 1 -- 172.21.15.165:0/369876941 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f4764019020 con 0x7f476c105150 2024-05-11T13:21:25.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.465+0000 7f4768c00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f47500b2870 0x7f47500b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:25.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.465+0000 7f4768c00700 1 -- 172.21.15.165:0/369876941 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f47640e0e50 con 0x7f476c105150 2024-05-11T13:21:25.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.465+0000 7f476be00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f47500b2870 0x7f47500b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:25.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.466+0000 7f476be00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f47500b2870 0x7f47500b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f475c005950 tx=0x7f475c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:21:25.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.468+0000 7f4768c00700 1 -- 172.21.15.165:0/369876941 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4764020020 con 0x7f476c105150 2024-05-11T13:21:25.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:25.713+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f476c076ea0 con 0x7f47500b2870 2024-05-11T13:21:25.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:25 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi071 2024-05-11T13:21:25.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:25 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi071 to 17089M 2024-05-11T13:21:25.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:25 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi071 2024-05-11T13:21:25.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:25 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi071 to 17089M 2024-05-11T13:21:26.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:25 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi071 2024-05-11T13:21:26.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:25 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi071 to 17089M 2024-05-11T13:21:26.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:26 smithi063 ceph-mon[29809]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:26.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:26 smithi063 ceph-mon[29809]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:26 smithi165 ceph-mon[33235]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:26 smithi165 ceph-mon[33235]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:21:26.935 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:26.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:26 smithi071 ceph-mon[33293]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:26.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:26 smithi071 ceph-mon[33293]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:21:26.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:21:26.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:21:26.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:28 smithi063 ceph-mon[29809]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:28.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:28 smithi071 ceph-mon[33293]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:29.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:28 smithi165 ceph-mon[33235]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:29.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:29 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/3913854101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]: dispatch 2024-05-11T13:21:29.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:29 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]: dispatch 2024-05-11T13:21:29.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:29 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]': finished 2024-05-11T13:21:29.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:29 smithi063 ceph-mon[29809]: osdmap e15: 3 total, 2 up, 3 in 2024-05-11T13:21:29.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:29 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:29.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:29 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/3913854101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]: dispatch 2024-05-11T13:21:29.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:29 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]: dispatch 2024-05-11T13:21:29.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:29 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]': finished 2024-05-11T13:21:29.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:29 smithi071 ceph-mon[33293]: osdmap e15: 3 total, 2 up, 3 in 2024-05-11T13:21:29.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:29 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:29.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:29 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/3913854101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]: dispatch 2024-05-11T13:21:29.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:29 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]: dispatch 2024-05-11T13:21:29.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:29 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f94b33d-6308-46a0-9f5c-63bd1bcaa477"}]': finished 2024-05-11T13:21:29.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:29 smithi165 ceph-mon[33235]: osdmap e15: 3 total, 2 up, 3 in 2024-05-11T13:21:29.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:29 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:30.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:30 smithi063 ceph-mon[29809]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:30.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:30 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/1035896228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:21:30.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:30 smithi071 ceph-mon[33293]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:30 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/1035896228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:21:31.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:30 smithi165 ceph-mon[33235]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:31.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:30 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/1035896228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:21:32.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:32 smithi063 ceph-mon[29809]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:32.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:32 smithi071 ceph-mon[33293]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:33.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:32 smithi165 ceph-mon[33235]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:34.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:34 smithi063 ceph-mon[29809]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:34.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:34 smithi071 ceph-mon[33293]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:35.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:34 smithi165 ceph-mon[33235]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:36.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:36 smithi165 ceph-mon[33235]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:36 smithi063 ceph-mon[29809]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:36.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:36 smithi071 ceph-mon[33293]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:38.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:38 smithi071 ceph-mon[33293]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:38.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-11T13:21:38.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:38 smithi165 ceph-mon[33235]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-11T13:21:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:39.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:38 smithi063 ceph-mon[29809]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:39.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-11T13:21:39.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:39.782 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:39 smithi165 ceph-mon[33235]: Deploying daemon osd.2 on smithi165 2024-05-11T13:21:39.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:39 smithi071 ceph-mon[33293]: Deploying daemon osd.2 on smithi165 2024-05-11T13:21:40.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:39 smithi063 ceph-mon[29809]: Deploying daemon osd.2 on smithi165 2024-05-11T13:21:41.001 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:40 smithi165 ceph-mon[33235]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:41.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:40 smithi063 ceph-mon[29809]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:41.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:40 smithi071 ceph-mon[33293]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:42.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:42 smithi165 ceph-mon[33235]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:43.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:42 smithi063 ceph-mon[29809]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:43.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:42 smithi071 ceph-mon[33293]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:44 smithi165 ceph-mon[33235]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:45.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:44 smithi063 ceph-mon[29809]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:45.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:44 smithi071 ceph-mon[33293]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:46 smithi165 ceph-mon[33235]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:47.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:46 smithi063 ceph-mon[29809]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:47.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:47.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:47.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:47.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:46 smithi071 ceph-mon[33293]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:47.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:48 smithi165 ceph-mon[33235]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:48 smithi063 ceph-mon[29809]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:49.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:48 smithi071 ceph-mon[33293]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:50.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.925+0000 7f4768c00700 1 -- 172.21.15.165:0/369876941 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f476c076ea0 con 0x7f47500b2870 2024-05-11T13:21:50.926 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 2 on host 'smithi165' 2024-05-11T13:21:50.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.928+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f47500b2870 msgr2=0x7f47500b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:50.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.928+0000 7f4772a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f47500b2870 0x7f47500b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f475c005950 tx=0x7f475c0058e0 comp rx=0 tx=0).stop 2024-05-11T13:21:50.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c105150 msgr2=0x7f476c1a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:50.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c105150 0x7f476c1a2db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4764007d40 tx=0x7f476400a340 comp rx=0 tx=0).stop 2024-05-11T13:21:50.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 shutdown_connections 2024-05-11T13:21:50.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f47500b2870 0x7f47500b4d30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:50.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476c105150 0x7f476c1a2db0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:50.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f476c1035b0 0x7f476c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:50.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 --2- 172.21.15.165:0/369876941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f476c1047b0 0x7f476c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:50.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 >> 172.21.15.165:0/369876941 conn(0x7f476c0feb50 msgr2=0x7f476c075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:21:50.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 shutdown_connections 2024-05-11T13:21:50.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:21:50.929+0000 7f4772a00700 1 -- 172.21.15.165:0/369876941 wait complete. 2024-05-11T13:21:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:50 smithi165 ceph-mon[33235]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:50 smithi165 ceph-mon[33235]: from='osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T13:21:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:50 smithi165 ceph-mon[33235]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T13:21:51.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:50 smithi063 ceph-mon[29809]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:51.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:51.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:51.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:50 smithi063 ceph-mon[29809]: from='osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T13:21:51.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:50 smithi063 ceph-mon[29809]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T13:21:51.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:50 smithi071 ceph-mon[33293]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:50 smithi071 ceph-mon[33293]: from='osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T13:21:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:50 smithi071 ceph-mon[33293]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T13:21:51.632 DEBUG:teuthology.orchestra.run.smithi165:osd.2> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.2.service 2024-05-11T13:21:51.639 INFO:tasks.cephadm:Deploying osd.3 on smithi063 with /dev/vg_nvme/lv_3... 2024-05-11T13:21:51.639 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-11T13:21:52.095 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:21:52.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:51 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:52.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:51 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:52.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:51 smithi063 ceph-mon[29809]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-11T13:21:52.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:51 smithi063 ceph-mon[29809]: from='osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:21:52.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:51 smithi063 ceph-mon[29809]: osdmap e16: 3 total, 2 up, 3 in 2024-05-11T13:21:52.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:51 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:52.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:51 smithi063 ceph-mon[29809]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:21:52.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:51 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:52.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:51 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:52.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:51 smithi071 ceph-mon[33293]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-11T13:21:52.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:51 smithi071 ceph-mon[33293]: from='osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:21:52.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:51 smithi071 ceph-mon[33293]: osdmap e16: 3 total, 2 up, 3 in 2024-05-11T13:21:52.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:51 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:52.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:51 smithi071 ceph-mon[33293]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:21:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:51 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:51 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:51 smithi165 ceph-mon[33235]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-11T13:21:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:51 smithi165 ceph-mon[33235]: from='osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:21:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:51 smithi165 ceph-mon[33235]: osdmap e16: 3 total, 2 up, 3 in 2024-05-11T13:21:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:51 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:51 smithi165 ceph-mon[33235]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:21:52.965 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:52 smithi165 ceph-mon[33235]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:52.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:52 smithi165 ceph-mon[33235]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:21:52.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:52 smithi165 ceph-mon[33235]: osdmap e17: 3 total, 2 up, 3 in 2024-05-11T13:21:52.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:52.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:52.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:52 smithi165 ceph-mon[33235]: from='osd.2 ' entity='osd.2' 2024-05-11T13:21:52.966 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2[38393]: 2024-05-11T13:21:52.701+0000 7f75f7a00700 -1 osd.2 0 waiting for initial osdmap 2024-05-11T13:21:52.967 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2[38393]: 2024-05-11T13:21:52.725+0000 7f75ed800700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:21:53.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:52 smithi063 ceph-mon[29809]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:52 smithi063 ceph-mon[29809]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:21:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:52 smithi063 ceph-mon[29809]: osdmap e17: 3 total, 2 up, 3 in 2024-05-11T13:21:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:52 smithi063 ceph-mon[29809]: from='osd.2 ' entity='osd.2' 2024-05-11T13:21:53.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:52 smithi071 ceph-mon[33293]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:53.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:52 smithi071 ceph-mon[33293]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:21:53.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:52 smithi071 ceph-mon[33293]: osdmap e17: 3 total, 2 up, 3 in 2024-05-11T13:21:53.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:53.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:53.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:52 smithi071 ceph-mon[33293]: from='osd.2 ' entity='osd.2' 2024-05-11T13:21:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:53 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:21:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:53 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:21:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:53 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:53 smithi165 ceph-mon[33235]: osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] boot 2024-05-11T13:21:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:53 smithi165 ceph-mon[33235]: osdmap e18: 3 total, 3 up, 3 in 2024-05-11T13:21:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:53 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:54.135 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:53 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:21:54.135 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:53 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:21:54.135 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:53 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:54.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:53 smithi063 ceph-mon[29809]: osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] boot 2024-05-11T13:21:54.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:53 smithi063 ceph-mon[29809]: osdmap e18: 3 total, 3 up, 3 in 2024-05-11T13:21:54.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:53 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:53 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:21:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:53 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:21:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:53 smithi071 ceph-mon[33293]: osd.2 [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] boot 2024-05-11T13:21:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:53 smithi071 ceph-mon[33293]: osdmap e18: 3 total, 3 up, 3 in 2024-05-11T13:21:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:21:55.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:54 smithi063 ceph-mon[29809]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:55.154 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-11T13:21:55.154 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-11T13:21:55.154 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-11T13:21:55.155 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-11T13:21:55.155 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0384512 s, 273 MB/s 2024-05-11T13:21:55.155 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-11T13:21:55.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:54 smithi071 ceph-mon[33293]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:55.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:54 smithi165 ceph-mon[33235]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-11T13:21:55.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_3 2024-05-11T13:21:56.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:55 smithi165 ceph-mon[33235]: osdmap e19: 3 total, 3 up, 3 in 2024-05-11T13:21:56.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:55 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:21:56.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:55 smithi063 ceph-mon[29809]: osdmap e19: 3 total, 3 up, 3 in 2024-05-11T13:21:56.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:55 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:21:56.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:55 smithi071 ceph-mon[33293]: osdmap e19: 3 total, 3 up, 3 in 2024-05-11T13:21:56.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:55 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:21:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:21:57.131 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: pgmap v107: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: osdmap e20: 3 total, 3 up, 3 in 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:57.132 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.133 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46081]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T13:21:57.133 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.133 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46081]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.133 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46081]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: pgmap v107: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:21:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-11T13:21:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: osdmap e20: 3 total, 3 up, 3 in 2024-05-11T13:21:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:21:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:21:57.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:57.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:57.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: pgmap v107: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:21:57.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: osdmap e20: 3 total, 3 up, 3 in 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:57.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:57.434 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T13:21:57.434 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.435 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46084]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.435 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46084]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:57.704 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41586]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T13:21:57.704 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41586]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.704 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41586]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.704 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41586]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:57.705 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T13:21:57.705 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.705 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41605]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.705 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41605]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:57.719 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42064]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T13:21:57.719 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42064]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.719 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42064]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.719 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42064]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:57.719 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42089]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T13:21:57.719 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42089]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.720 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42089]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.720 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42089]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46123]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46123]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46123]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46123]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46126]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46126]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46126]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:57.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 sudo[46126]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:58.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42094]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T13:21:58.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42094]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:58.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42094]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42094]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42097]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 sudo[42097]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi165 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi165 to 17089M 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: osdmap e21: 3 total, 3 up, 3 in 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:58.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.070 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.070 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.070 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.070 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.070 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:57 smithi165 ceph-mon[33235]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi165 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi165 to 17089M 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: osdmap e21: 3 total, 3 up, 3 in 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:58.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:57 smithi063 ceph-mon[29809]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41624]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T13:21:58.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41624]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:58.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41624]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:58.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41624]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:58.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41627]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41627]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41627]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 sudo[41627]: pam_unix(sudo:session): session closed for user root 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi165 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi165 to 17089M 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: osdmap e21: 3 total, 3 up, 3 in 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:21:58.208 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:57 smithi071 ceph-mon[33293]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T13:21:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.600+0000 7fb318a00700 1 -- 172.21.15.63:0/3588000327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 msgr2=0x7fb3141039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.600+0000 7fb318a00700 1 --2- 172.21.15.63:0/3588000327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3141039d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3040099c0 tx=0x7fb304009cd0 comp rx=0 tx=0).stop 2024-05-11T13:21:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.601+0000 7fb318a00700 1 -- 172.21.15.63:0/3588000327 shutdown_connections 2024-05-11T13:21:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.601+0000 7fb318a00700 1 --2- 172.21.15.63:0/3588000327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb314105150 0x7fb31410ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.601+0000 7fb318a00700 1 --2- 172.21.15.63:0/3588000327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3141039d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.601+0000 7fb318a00700 1 --2- 172.21.15.63:0/3588000327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3141047b0 0x7fb314104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.601+0000 7fb318a00700 1 -- 172.21.15.63:0/3588000327 >> 172.21.15.63:0/3588000327 conn(0x7fb3140feb50 msgr2=0x7fb314100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:21:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.601+0000 7fb318a00700 1 -- 172.21.15.63:0/3588000327 shutdown_connections 2024-05-11T13:21:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.601+0000 7fb318a00700 1 -- 172.21.15.63:0/3588000327 wait complete. 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 Processor -- start 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 -- start start 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3140772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3141047b0 0x7fb314075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb314105150 0x7fb314075e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb31410ef20 con 0x7fb3141035b0 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb31410eda0 con 0x7fb3141047b0 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.602+0000 7fb318a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb31410f0a0 con 0x7fb314105150 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3140772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3140772a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37504/0 (socket says 172.21.15.63:37504) 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 -- 172.21.15.63:0/158649650 learned_addr learned my addr 172.21.15.63:0/158649650 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311400700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3141047b0 0x7fb314075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb312800700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb314105150 0x7fb314075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 -- 172.21.15.63:0/158649650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb314105150 msgr2=0x7fb314075e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb314105150 0x7fb314075e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 -- 172.21.15.63:0/158649650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3141047b0 msgr2=0x7fb314075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3141047b0 0x7fb314075940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 -- 172.21.15.63:0/158649650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb314076750 con 0x7fb3141035b0 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.603+0000 7fb311e00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3140772a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb3040098e0 tx=0x7fb304009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.604+0000 7fb30aa00700 1 -- 172.21.15.63:0/158649650 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304027070 con 0x7fb3141035b0 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.604+0000 7fb30aa00700 1 -- 172.21.15.63:0/158649650 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb30400bd30 con 0x7fb3141035b0 2024-05-11T13:21:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.604+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3141a37d0 con 0x7fb3141035b0 2024-05-11T13:21:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.604+0000 7fb30aa00700 1 -- 172.21.15.63:0/158649650 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3040037f0 con 0x7fb3141035b0 2024-05-11T13:21:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.604+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3141a3cd0 con 0x7fb3141035b0 2024-05-11T13:21:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.605+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31406a240 con 0x7fb3141035b0 2024-05-11T13:21:58.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.607+0000 7fb30aa00700 1 -- 172.21.15.63:0/158649650 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb30402a070 con 0x7fb3141035b0 2024-05-11T13:21:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.610+0000 7fb30aa00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2f80b2900 0x7fb2f80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:21:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.610+0000 7fb30aa00700 1 -- 172.21.15.63:0/158649650 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7fb3040e1dd0 con 0x7fb3141035b0 2024-05-11T13:21:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.610+0000 7fb311400700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2f80b2900 0x7fb2f80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:21:58.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.611+0000 7fb311400700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2f80b2900 0x7fb2f80b4dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb300005fd0 tx=0x7fb300005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:21:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.615+0000 7fb30aa00700 1 -- 172.21.15.63:0/158649650 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb30405fdf0 con 0x7fb3141035b0 2024-05-11T13:21:58.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:21:58.862+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb314063a50 con 0x7fb2f80b2900 2024-05-11T13:21:59.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:58 smithi063 ceph-mon[29809]: pgmap v110: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:21:59.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:58 smithi063 ceph-mon[29809]: osdmap e22: 3 total, 3 up, 3 in 2024-05-11T13:21:59.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:58 smithi063 ceph-mon[29809]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-11T13:21:59.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:58 smithi063 ceph-mon[29809]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:21:59.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:21:59.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:21:59.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:58 smithi165 ceph-mon[33235]: pgmap v110: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:21:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:58 smithi165 ceph-mon[33235]: osdmap e22: 3 total, 3 up, 3 in 2024-05-11T13:21:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:58 smithi165 ceph-mon[33235]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-11T13:21:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:58 smithi165 ceph-mon[33235]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:21:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:21:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:21:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:21:59.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:58 smithi071 ceph-mon[33293]: pgmap v110: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:21:59.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:58 smithi071 ceph-mon[33293]: osdmap e22: 3 total, 3 up, 3 in 2024-05-11T13:21:59.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:58 smithi071 ceph-mon[33293]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-11T13:21:59.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:58 smithi071 ceph-mon[33293]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:21:59.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:21:59.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:21:59.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:00.174 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:21:59 smithi063 ceph-mon[29809]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:00.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:21:59 smithi165 ceph-mon[33235]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:00.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:21:59 smithi071 ceph-mon[33293]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:02.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:02 smithi165 ceph-mon[33235]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:02.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:02 smithi063 ceph-mon[29809]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:02.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:02 smithi071 ceph-mon[33293]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:03.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:03 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1385337229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d387295c-4396-466f-92e3-400391f0e674"}]: dispatch 2024-05-11T13:22:03.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:03 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1385337229' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d387295c-4396-466f-92e3-400391f0e674"}]': finished 2024-05-11T13:22:03.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:03 smithi165 ceph-mon[33235]: osdmap e23: 4 total, 3 up, 4 in 2024-05-11T13:22:03.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:03.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:03 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1385337229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d387295c-4396-466f-92e3-400391f0e674"}]: dispatch 2024-05-11T13:22:03.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:03 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1385337229' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d387295c-4396-466f-92e3-400391f0e674"}]': finished 2024-05-11T13:22:03.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:03 smithi063 ceph-mon[29809]: osdmap e23: 4 total, 3 up, 4 in 2024-05-11T13:22:03.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:03.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:03 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1385337229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d387295c-4396-466f-92e3-400391f0e674"}]: dispatch 2024-05-11T13:22:03.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:03 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1385337229' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d387295c-4396-466f-92e3-400391f0e674"}]': finished 2024-05-11T13:22:03.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:03 smithi071 ceph-mon[33293]: osdmap e23: 4 total, 3 up, 4 in 2024-05-11T13:22:03.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:04.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:04 smithi165 ceph-mon[33235]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:04.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:04 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1483411898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:22:04.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:04 smithi063 ceph-mon[29809]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:04.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:04 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1483411898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:22:04.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:04 smithi071 ceph-mon[33293]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:04.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:04 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1483411898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:22:06.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:06 smithi071 ceph-mon[33293]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:06.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:06 smithi165 ceph-mon[33235]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:06 smithi063 ceph-mon[29809]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:08.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:08 smithi063 ceph-mon[29809]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:08.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:08 smithi165 ceph-mon[33235]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:08.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:08 smithi071 ceph-mon[33293]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:10.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:10 smithi165 ceph-mon[33235]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:10.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:10 smithi063 ceph-mon[29809]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:10.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:10 smithi071 ceph-mon[33293]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:12.779 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:12 smithi063 ceph-mon[29809]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:12 smithi165 ceph-mon[33235]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:12 smithi071 ceph-mon[33293]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:13 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-11T13:22:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:13 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:13 smithi165 ceph-mon[33235]: Deploying daemon osd.3 on smithi063 2024-05-11T13:22:13.859 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:13 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-11T13:22:13.859 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:13 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:13.859 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:13 smithi063 ceph-mon[29809]: Deploying daemon osd.3 on smithi063 2024-05-11T13:22:13.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:13 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-11T13:22:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:13 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:13 smithi071 ceph-mon[33293]: Deploying daemon osd.3 on smithi063 2024-05-11T13:22:14.737 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:14 smithi063 ceph-mon[29809]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:14 smithi165 ceph-mon[33235]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:14.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:14 smithi071 ceph-mon[33293]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:16.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:16 smithi165 ceph-mon[33235]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:16.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:16 smithi063 ceph-mon[29809]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:16.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:16 smithi071 ceph-mon[33293]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:18.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:18 smithi063 ceph-mon[29809]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:18.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:18 smithi165 ceph-mon[33235]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:18.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:18 smithi071 ceph-mon[33293]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:20.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:20 smithi165 ceph-mon[33235]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:20.915 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:20 smithi063 ceph-mon[29809]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:20.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:20 smithi071 ceph-mon[33293]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:22.532 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:22.532 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:22.532 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:22 smithi063 ceph-mon[29809]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:22.532 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:22 smithi165 ceph-mon[33235]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:22 smithi071 ceph-mon[33293]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:24.802 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:24 smithi063 ceph-mon[29809]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:24.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:24 smithi165 ceph-mon[33235]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:24.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:24 smithi071 ceph-mon[33293]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:26.308 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:26.308 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:26.308 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:26 smithi063 ceph-mon[29809]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:26.309 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:26 smithi063 ceph-mon[29809]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-11T13:22:26.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:26.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:26.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:26 smithi165 ceph-mon[33235]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:26.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:26 smithi165 ceph-mon[33235]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-11T13:22:26.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.408+0000 7fb30aa00700 1 -- 172.21.15.63:0/158649650 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb314063a50 con 0x7fb2f80b2900 2024-05-11T13:22:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 3 on host 'smithi063' 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.412+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2f80b2900 msgr2=0x7fb2f80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.412+0000 7fb318a00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2f80b2900 0x7fb2f80b4dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb300005fd0 tx=0x7fb300005f60 comp rx=0 tx=0).stop 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.412+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 msgr2=0x7fb3140772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.412+0000 7fb318a00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3140772a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb3040098e0 tx=0x7fb304009580 comp rx=0 tx=0).stop 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 shutdown_connections 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2f80b2900 0x7fb2f80b4dc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb314105150 0x7fb314075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3141035b0 0x7fb3140772a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 --2- 172.21.15.63:0/158649650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3141047b0 0x7fb314075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 >> 172.21.15.63:0/158649650 conn(0x7fb3140feb50 msgr2=0x7fb314107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:22:26.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 shutdown_connections 2024-05-11T13:22:26.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:22:26.413+0000 7fb318a00700 1 -- 172.21.15.63:0/158649650 wait complete. 2024-05-11T13:22:26.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:26.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:26.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:26 smithi071 ceph-mon[33293]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:26.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:26 smithi071 ceph-mon[33293]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-11T13:22:27.070 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-11T13:22:27.070 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: osdmap e24: 4 total, 3 up, 4 in 2024-05-11T13:22:27.070 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:22:27.070 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:27.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:27.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:22:27.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:27.315 DEBUG:teuthology.orchestra.run.smithi063:osd.3> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.3.service 2024-05-11T13:22:27.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: osdmap e24: 4 total, 3 up, 4 in 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:27.323 INFO:tasks.cephadm:Deploying osd.4 on smithi071 with /dev/vg_nvme/lv_3... 2024-05-11T13:22:27.323 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: osdmap e24: 4 total, 3 up, 4 in 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:27.400 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:27.763 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:22:28.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:28 smithi071 ceph-mon[33293]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:22:28.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:28 smithi071 ceph-mon[33293]: osdmap e25: 4 total, 3 up, 4 in 2024-05-11T13:22:28.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:28.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:28.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:28 smithi071 ceph-mon[33293]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:28.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:28 smithi165 ceph-mon[33235]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:22:28.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:28 smithi165 ceph-mon[33235]: osdmap e25: 4 total, 3 up, 4 in 2024-05-11T13:22:28.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:28.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:28.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:28 smithi165 ceph-mon[33235]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:28.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:28 smithi063 ceph-mon[29809]: from='osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:22:28.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:28 smithi063 ceph-mon[29809]: osdmap e25: 4 total, 3 up, 4 in 2024-05-11T13:22:28.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:28.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:28.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:28 smithi063 ceph-mon[29809]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-11T13:22:29.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:29 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:22:29.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:29 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:22:29.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:29 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:29.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:29 smithi071 ceph-mon[33293]: osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] boot 2024-05-11T13:22:29.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:29 smithi071 ceph-mon[33293]: osdmap e26: 4 total, 4 up, 4 in 2024-05-11T13:22:29.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:29 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:29 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:22:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:29 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:22:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:29 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:29 smithi165 ceph-mon[33235]: osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] boot 2024-05-11T13:22:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:29 smithi165 ceph-mon[33235]: osdmap e26: 4 total, 4 up, 4 in 2024-05-11T13:22:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:29 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:29.377 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:29 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:22:29.377 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:29 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:22:29.377 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:29 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:29.377 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:29 smithi063 ceph-mon[29809]: osd.3 [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] boot 2024-05-11T13:22:29.377 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:29 smithi063 ceph-mon[29809]: osdmap e26: 4 total, 4 up, 4 in 2024-05-11T13:22:29.377 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:29 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:22:30.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:30 smithi071 ceph-mon[33293]: osdmap e27: 4 total, 4 up, 4 in 2024-05-11T13:22:30.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:30 smithi071 ceph-mon[33293]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:30.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:30 smithi071 ceph-mon[33293]: osdmap e28: 4 total, 4 up, 4 in 2024-05-11T13:22:30.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:30 smithi063 ceph-mon[29809]: osdmap e27: 4 total, 4 up, 4 in 2024-05-11T13:22:30.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:30 smithi063 ceph-mon[29809]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:30.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:30 smithi063 ceph-mon[29809]: osdmap e28: 4 total, 4 up, 4 in 2024-05-11T13:22:30.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:30 smithi165 ceph-mon[33235]: osdmap e27: 4 total, 4 up, 4 in 2024-05-11T13:22:30.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:30 smithi165 ceph-mon[33235]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:30.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:30 smithi165 ceph-mon[33235]: osdmap e28: 4 total, 4 up, 4 in 2024-05-11T13:22:30.853 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-11T13:22:30.853 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-11T13:22:30.853 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-11T13:22:30.853 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-11T13:22:30.853 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0328457 s, 319 MB/s 2024-05-11T13:22:30.854 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-11T13:22:31.720 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-05-11T13:22:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:22:32.649 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:32 smithi071 ceph-mon[33293]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-11T13:22:32.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:32 smithi063 ceph-mon[29809]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-11T13:22:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:32 smithi165 ceph-mon[33235]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-11T13:22:33.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi063 2024-05-11T13:22:33.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:22:33.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:22:33.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi063 to 8544M 2024-05-11T13:22:33.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:33.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:22:33.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.853 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi063 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi063 to 8544M 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:22:33.854 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi063 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi063 to 8544M 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:22:33.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 -- 172.21.15.71:0/1689247887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc106a60 msgr2=0x7f86bc108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 --2- 172.21.15.71:0/1689247887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc106a60 0x7f86bc108e50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009990 tx=0x7f86a8009ca0 comp rx=0 tx=0).stop 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 -- 172.21.15.71:0/1689247887 shutdown_connections 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 --2- 172.21.15.71:0/1689247887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86bc109390 0x7f86bc10b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 --2- 172.21.15.71:0/1689247887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86bc104130 0x7f86bc106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 --2- 172.21.15.71:0/1689247887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc106a60 0x7f86bc108e50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 -- 172.21.15.71:0/1689247887 >> 172.21.15.71:0/1689247887 conn(0x7f86bc0fdaa0 msgr2=0x7f86bc0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:22:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.375+0000 7f86c1800700 1 -- 172.21.15.71:0/1689247887 shutdown_connections 2024-05-11T13:22:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.376+0000 7f86c1800700 1 -- 172.21.15.71:0/1689247887 wait complete. 2024-05-11T13:22:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.377+0000 7f86c1800700 1 Processor -- start 2024-05-11T13:22:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86c1800700 1 -- start start 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86c1800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86bc104130 0x7f86bc199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86c1800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86bc106a60 0x7f86bc199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86c1800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc109390 0x7f86bc1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86c1800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc10ef90 con 0x7f86bc104130 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86c1800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc10ee10 con 0x7f86bc109390 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86c1800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc10f110 con 0x7f86bc106a60 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86bb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc109390 0x7f86bc1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:22:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.378+0000 7f86bb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc109390 0x7f86bc1a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55292/0 (socket says 172.21.15.71:55292) 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86bb400700 1 -- 172.21.15.71:0/3558591110 learned_addr learned my addr 172.21.15.71:0/3558591110 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86ba000700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86bc106a60 0x7f86bc199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86baa00700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86bc104130 0x7f86bc199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86bb400700 1 -- 172.21.15.71:0/3558591110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86bc106a60 msgr2=0x7f86bc199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86bb400700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86bc106a60 0x7f86bc199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86bb400700 1 -- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86bc104130 msgr2=0x7f86bc199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86bb400700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86bc104130 0x7f86bc199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:22:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86bb400700 1 -- 172.21.15.71:0/3558591110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc1a1180 con 0x7f86bc109390 2024-05-11T13:22:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.379+0000 7f86bb400700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc109390 0x7f86bc1a0ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f86b4007d70 tx=0x7f86b400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:22:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.380+0000 7f86b3400700 1 -- 172.21.15.71:0/3558591110 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4016020 con 0x7f86bc109390 2024-05-11T13:22:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.380+0000 7f86b3400700 1 -- 172.21.15.71:0/3558591110 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86b400ad40 con 0x7f86bc109390 2024-05-11T13:22:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.380+0000 7f86b3400700 1 -- 172.21.15.71:0/3558591110 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b40115e0 con 0x7f86bc109390 2024-05-11T13:22:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.380+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc1a1470 con 0x7f86bc109390 2024-05-11T13:22:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.380+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a1960 con 0x7f86bc109390 2024-05-11T13:22:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.381+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc06a240 con 0x7f86bc109390 2024-05-11T13:22:34.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.383+0000 7f86b3400700 1 -- 172.21.15.71:0/3558591110 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f86b4019020 con 0x7f86bc109390 2024-05-11T13:22:34.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.385+0000 7f86b3400700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f86a00b2900 0x7f86a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:22:34.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.385+0000 7f86b3400700 1 -- 172.21.15.71:0/3558591110 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f86b40e15d0 con 0x7f86bc109390 2024-05-11T13:22:34.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.385+0000 7f86baa00700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f86a00b2900 0x7f86a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:22:34.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.386+0000 7f86baa00700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f86a00b2900 0x7f86a00b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f86ac005ea0 tx=0x7f86ac005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:22:34.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.388+0000 7f86b3400700 1 -- 172.21.15.71:0/3558591110 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86b4020020 con 0x7f86bc109390 2024-05-11T13:22:34.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:22:34.633+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f86bc063990 con 0x7f86a00b2900 2024-05-11T13:22:34.635 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:34 smithi071 ceph-mon[33293]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:22:34.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:34 smithi165 ceph-mon[33235]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:22:34.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:34 smithi063 ceph-mon[29809]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:22:35.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:35 smithi165 ceph-mon[33235]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:22:35.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:22:35.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:22:35.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:35 smithi071 ceph-mon[33293]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:22:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:22:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:22:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:35.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:35 smithi063 ceph-mon[29809]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:22:35.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:22:35.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:22:35.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:36 smithi071 ceph-mon[33293]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-11T13:22:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:36 smithi063 ceph-mon[29809]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-11T13:22:37.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:36 smithi165 ceph-mon[33235]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-11T13:22:38.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:38 smithi063 ceph-mon[29809]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-11T13:22:38.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:38 smithi071 ceph-mon[33293]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-11T13:22:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:38 smithi165 ceph-mon[33235]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-11T13:22:39.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:39 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/2286988623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]: dispatch 2024-05-11T13:22:39.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:39 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]: dispatch 2024-05-11T13:22:39.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:39 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]': finished 2024-05-11T13:22:39.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:39 smithi063 ceph-mon[29809]: osdmap e29: 5 total, 4 up, 5 in 2024-05-11T13:22:39.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:22:39.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:39 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/2286988623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]: dispatch 2024-05-11T13:22:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:39 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]: dispatch 2024-05-11T13:22:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:39 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]': finished 2024-05-11T13:22:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:39 smithi071 ceph-mon[33293]: osdmap e29: 5 total, 4 up, 5 in 2024-05-11T13:22:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:22:40.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:39 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/2286988623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]: dispatch 2024-05-11T13:22:40.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:39 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]: dispatch 2024-05-11T13:22:40.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:39 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d209500e-da38-45b9-9ae7-eddaafae3127"}]': finished 2024-05-11T13:22:40.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:39 smithi165 ceph-mon[33235]: osdmap e29: 5 total, 4 up, 5 in 2024-05-11T13:22:40.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:22:40.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:40 smithi063 ceph-mon[29809]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T13:22:40.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:40 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/1067144500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:22:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:40 smithi071 ceph-mon[33293]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T13:22:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:40 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/1067144500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:22:41.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:40 smithi165 ceph-mon[33235]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T13:22:41.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:40 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/1067144500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:22:42.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:42 smithi071 ceph-mon[33293]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:42.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:42 smithi063 ceph-mon[29809]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:43.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:42 smithi165 ceph-mon[33235]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:44.927 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:44 smithi071 ceph-mon[33293]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:44.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:44 smithi063 ceph-mon[29809]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:44 smithi165 ceph-mon[33235]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:46.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:46 smithi063 ceph-mon[29809]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:46 smithi071 ceph-mon[33293]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:46 smithi165 ceph-mon[33235]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:48.902 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:48 smithi071 ceph-mon[33293]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:48.902 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-11T13:22:48.902 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:48.902 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:48 smithi071 ceph-mon[33293]: Deploying daemon osd.4 on smithi071 2024-05-11T13:22:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:48 smithi063 ceph-mon[29809]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-11T13:22:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:48 smithi063 ceph-mon[29809]: Deploying daemon osd.4 on smithi071 2024-05-11T13:22:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:48 smithi165 ceph-mon[33235]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-11T13:22:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:22:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:48 smithi165 ceph-mon[33235]: Deploying daemon osd.4 on smithi071 2024-05-11T13:22:50.834 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:50 smithi071 ceph-mon[33293]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:50.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:50 smithi165 ceph-mon[33235]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:50 smithi063 ceph-mon[29809]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:52 smithi063 ceph-mon[29809]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:52.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:52 smithi071 ceph-mon[33293]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:52 smithi165 ceph-mon[33235]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:54.906 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:54 smithi071 ceph-mon[33293]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:54.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:54 smithi063 ceph-mon[29809]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:55.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:54 smithi165 ceph-mon[33235]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:56 smithi063 ceph-mon[29809]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:56.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:56.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:56 smithi071 ceph-mon[33293]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:56.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:56.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:56.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:56 smithi165 ceph-mon[33235]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:22:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:22:58.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:22:58 smithi071 ceph-mon[33293]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:58.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:22:58 smithi063 ceph-mon[29809]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:22:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:22:58 smithi165 ceph-mon[33235]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:00.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:00 smithi071 ceph-mon[33293]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:00.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:00 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:00.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:00 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:00.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:00 smithi071 ceph-mon[33293]: from='osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T13:23:00.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:00 smithi071 ceph-mon[33293]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T13:23:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:00 smithi063 ceph-mon[29809]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:00 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:00 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:00.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:00 smithi063 ceph-mon[29809]: from='osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T13:23:00.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:00 smithi063 ceph-mon[29809]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T13:23:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:00 smithi165 ceph-mon[33235]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:00 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:00 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:00 smithi165 ceph-mon[33235]: from='osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T13:23:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:00 smithi165 ceph-mon[33235]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T13:23:01.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.308+0000 7f86b3400700 1 -- 172.21.15.71:0/3558591110 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f86bc063990 con 0x7f86a00b2900 2024-05-11T13:23:01.310 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 4 on host 'smithi071' 2024-05-11T13:23:01.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f86a00b2900 msgr2=0x7f86a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f86a00b2900 0x7f86a00b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f86ac005ea0 tx=0x7f86ac005e30 comp rx=0 tx=0).stop 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc109390 msgr2=0x7f86bc1a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc109390 0x7f86bc1a0ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f86b4007d70 tx=0x7f86b400a340 comp rx=0 tx=0).stop 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 shutdown_connections 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f86a00b2900 0x7f86a00b4dc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc109390 0x7f86bc1a0ae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86bc104130 0x7f86bc199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 --2- 172.21.15.71:0/3558591110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86bc106a60 0x7f86bc199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 >> 172.21.15.71:0/3558591110 conn(0x7f86bc0fdaa0 msgr2=0x7f86bc104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 shutdown_connections 2024-05-11T13:23:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:23:01.313+0000 7f86c1800700 1 -- 172.21.15.71:0/3558591110 wait complete. 2024-05-11T13:23:01.986 DEBUG:teuthology.orchestra.run.smithi071:osd.4> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.4.service 2024-05-11T13:23:01.991 INFO:tasks.cephadm:Deploying osd.5 on smithi165 with /dev/vg_nvme/lv_3... 2024-05-11T13:23:01.991 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-11T13:23:02.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: osdmap e30: 5 total, 4 up, 5 in 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: osdmap e30: 5 total, 4 up, 5 in 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:02.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:02.430 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:23:02.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-11T13:23:02.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: osdmap e30: 5 total, 4 up, 5 in 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:02.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:03.034 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 13:23:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T13:23:02.890+0000 7f3ab4c00700 -1 osd.4 0 waiting for initial osdmap 2024-05-11T13:23:03.034 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 13:23:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T13:23:02.912+0000 7f3aaaa00700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:23:03.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:03 smithi165 ceph-mon[33235]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:23:03.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:03 smithi165 ceph-mon[33235]: osdmap e31: 5 total, 4 up, 5 in 2024-05-11T13:23:03.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:03.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:03 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:03.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:03 smithi165 ceph-mon[33235]: from='osd.4 ' entity='osd.4' 2024-05-11T13:23:03.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:03 smithi063 ceph-mon[29809]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:23:03.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:03 smithi063 ceph-mon[29809]: osdmap e31: 5 total, 4 up, 5 in 2024-05-11T13:23:03.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:03.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:03 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:03.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:03 smithi063 ceph-mon[29809]: from='osd.4 ' entity='osd.4' 2024-05-11T13:23:03.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:03 smithi071 ceph-mon[33293]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:23:03.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:03 smithi071 ceph-mon[33293]: osdmap e31: 5 total, 4 up, 5 in 2024-05-11T13:23:03.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:03.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:03 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:03.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:03 smithi071 ceph-mon[33293]: from='osd.4 ' entity='osd.4' 2024-05-11T13:23:04.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:04 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:23:04.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:04 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:23:04.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:04.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:04 smithi071 ceph-mon[33293]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:04.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:04 smithi071 ceph-mon[33293]: osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] boot 2024-05-11T13:23:04.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:04 smithi071 ceph-mon[33293]: osdmap e32: 5 total, 5 up, 5 in 2024-05-11T13:23:04.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:04.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:04 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:23:04.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:04 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:23:04.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:04.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:04 smithi165 ceph-mon[33235]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:04.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:04 smithi165 ceph-mon[33235]: osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] boot 2024-05-11T13:23:04.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:04 smithi165 ceph-mon[33235]: osdmap e32: 5 total, 5 up, 5 in 2024-05-11T13:23:04.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:04.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:04 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:23:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:04 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:23:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:04 smithi063 ceph-mon[29809]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-11T13:23:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:04 smithi063 ceph-mon[29809]: osd.4 [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] boot 2024-05-11T13:23:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:04 smithi063 ceph-mon[29809]: osdmap e32: 5 total, 5 up, 5 in 2024-05-11T13:23:04.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:23:05.201 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-11T13:23:05.201 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-11T13:23:05.201 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-11T13:23:05.201 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-11T13:23:05.202 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0541236 s, 194 MB/s 2024-05-11T13:23:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-11T13:23:06.026 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-05-11T13:23:06.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:06 smithi165 ceph-mon[33235]: osdmap e33: 5 total, 5 up, 5 in 2024-05-11T13:23:06.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:06 smithi165 ceph-mon[33235]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:06.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:06 smithi063 ceph-mon[29809]: osdmap e33: 5 total, 5 up, 5 in 2024-05-11T13:23:06.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:06 smithi063 ceph-mon[29809]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:06.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:06 smithi071 ceph-mon[33293]: osdmap e33: 5 total, 5 up, 5 in 2024-05-11T13:23:06.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:06 smithi071 ceph-mon[33293]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:06.462 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.451+0000 7ff444200700 1 -- 172.21.15.165:0/3381317830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c1057f0 msgr2=0x7ff43c105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.451+0000 7ff444200700 1 --2- 172.21.15.165:0/3381317830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c1057f0 0x7ff43c105c10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff430009990 tx=0x7ff430009ca0 comp rx=0 tx=0).stop 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.451+0000 7ff444200700 1 -- 172.21.15.165:0/3381317830 shutdown_connections 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.451+0000 7ff444200700 1 --2- 172.21.15.165:0/3381317830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c107390 0x7ff43c10dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.451+0000 7ff444200700 1 --2- 172.21.15.165:0/3381317830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c1057f0 0x7ff43c105c10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.451+0000 7ff444200700 1 --2- 172.21.15.165:0/3381317830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43c1069f0 0x7ff43c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.452+0000 7ff444200700 1 -- 172.21.15.165:0/3381317830 >> 172.21.15.165:0/3381317830 conn(0x7ff43c100db0 msgr2=0x7ff43c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.452+0000 7ff444200700 1 -- 172.21.15.165:0/3381317830 shutdown_connections 2024-05-11T13:23:08.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.452+0000 7ff444200700 1 -- 172.21.15.165:0/3381317830 wait complete. 2024-05-11T13:23:08.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 Processor -- start 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 -- start start 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43c1057f0 0x7ff43c19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c1069f0 0x7ff43c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c107390 0x7ff43c1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c111180 con 0x7ff43c1057f0 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c111000 con 0x7ff43c1069f0 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff444200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c111300 con 0x7ff43c107390 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.453+0000 7ff442200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c107390 0x7ff43c1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c107390 0x7ff43c1a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51736/0 (socket says 172.21.15.165:51736) 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 -- 172.21.15.165:0/769957289 learned_addr learned my addr 172.21.15.165:0/769957289 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:23:08.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff441800700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43c1057f0 0x7ff43c19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff440e00700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c1069f0 0x7ff43c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 -- 172.21.15.165:0/769957289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c1069f0 msgr2=0x7ff43c19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c1069f0 0x7ff43c19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 -- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43c1057f0 msgr2=0x7ff43c19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43c1057f0 0x7ff43c19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 -- 172.21.15.165:0/769957289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c1a34c0 con 0x7ff43c107390 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff441800700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43c1057f0 0x7ff43c19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.454+0000 7ff442200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c107390 0x7ff43c1a2e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff43400add0 tx=0x7ff43400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.455+0000 7ff43a000700 1 -- 172.21.15.165:0/769957289 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434017020 con 0x7ff43c107390 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.455+0000 7ff43a000700 1 -- 172.21.15.165:0/769957289 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff43400cdd0 con 0x7ff43c107390 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.455+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c1a37b0 con 0x7ff43c107390 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.455+0000 7ff43a000700 1 -- 172.21.15.165:0/769957289 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434005330 con 0x7ff43c107390 2024-05-11T13:23:08.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.455+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a3cf0 con 0x7ff43c107390 2024-05-11T13:23:08.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.456+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c06a240 con 0x7ff43c107390 2024-05-11T13:23:08.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.458+0000 7ff43a000700 1 -- 172.21.15.165:0/769957289 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff434004ad0 con 0x7ff43c107390 2024-05-11T13:23:08.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.460+0000 7ff43a000700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff4240b2880 0x7ff4240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:08.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.460+0000 7ff43a000700 1 -- 172.21.15.165:0/769957289 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7ff4340e0f90 con 0x7ff43c107390 2024-05-11T13:23:08.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.461+0000 7ff441800700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff4240b2880 0x7ff4240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:08.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.462+0000 7ff441800700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff4240b2880 0x7ff4240b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b5c0 tx=0x7ff430005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:23:08.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.463+0000 7ff43a000700 1 -- 172.21.15.165:0/769957289 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff434021020 con 0x7ff43c107390 2024-05-11T13:23:08.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi071 2024-05-11T13:23:08.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:08.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:08.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi071 to 8544M 2024-05-11T13:23:08.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:08.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:08.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:08.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi071 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi071 to 8544M 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:08.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi071 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi071 to 8544M 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:08.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:08.709+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff43c063a50 con 0x7ff4240b2880 2024-05-11T13:23:09.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:09 smithi071 ceph-mon[33293]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:23:09.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:23:09.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:23:09.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:09.763 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:09 smithi165 ceph-mon[33235]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:23:09.764 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:23:09.764 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:23:09.764 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:09.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:09 smithi063 ceph-mon[29809]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:23:09.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:23:09.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:23:09.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:10.642 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:10 smithi165 ceph-mon[33235]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:10.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:10 smithi063 ceph-mon[29809]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:10.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:10 smithi071 ceph-mon[33293]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:12 smithi165 ceph-mon[33235]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:12.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:12 smithi063 ceph-mon[29809]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:12.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:12 smithi071 ceph-mon[33293]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:13 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/3185589096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]: dispatch 2024-05-11T13:23:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:13 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]: dispatch 2024-05-11T13:23:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:13 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]': finished 2024-05-11T13:23:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:13 smithi165 ceph-mon[33235]: osdmap e34: 6 total, 5 up, 6 in 2024-05-11T13:23:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:13 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:13 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/4142919653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:23:13.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:13 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/3185589096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]: dispatch 2024-05-11T13:23:13.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:13 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]: dispatch 2024-05-11T13:23:13.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:13 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]': finished 2024-05-11T13:23:13.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:13 smithi063 ceph-mon[29809]: osdmap e34: 6 total, 5 up, 6 in 2024-05-11T13:23:13.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:13 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:13.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:13 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/4142919653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:23:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:13 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/3185589096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]: dispatch 2024-05-11T13:23:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:13 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]: dispatch 2024-05-11T13:23:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:13 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dd5fca3-e67c-4824-af4e-2e6efa55db80"}]': finished 2024-05-11T13:23:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:13 smithi071 ceph-mon[33293]: osdmap e34: 6 total, 5 up, 6 in 2024-05-11T13:23:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:13 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:13.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:13 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/4142919653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:23:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:14 smithi165 ceph-mon[33235]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:14 smithi063 ceph-mon[29809]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:14.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:14 smithi071 ceph-mon[33293]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:16.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:16 smithi165 ceph-mon[33235]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:16.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:16 smithi063 ceph-mon[29809]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:16.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:16 smithi071 ceph-mon[33293]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:18.904 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:18 smithi165 ceph-mon[33235]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:18.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:18 smithi063 ceph-mon[29809]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:18.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:18 smithi071 ceph-mon[33293]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:20.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:20 smithi165 ceph-mon[33235]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:20 smithi063 ceph-mon[29809]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:20.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:20 smithi071 ceph-mon[33293]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:22.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:22 smithi165 ceph-mon[33235]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:22.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-11T13:23:22.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:22.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:22 smithi165 ceph-mon[33235]: Deploying daemon osd.5 on smithi165 2024-05-11T13:23:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:22 smithi063 ceph-mon[29809]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:22.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-11T13:23:22.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:22.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:22 smithi063 ceph-mon[29809]: Deploying daemon osd.5 on smithi165 2024-05-11T13:23:22.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:22 smithi071 ceph-mon[33293]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:22.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-11T13:23:22.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:22.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:22 smithi071 ceph-mon[33293]: Deploying daemon osd.5 on smithi165 2024-05-11T13:23:24.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:24 smithi165 ceph-mon[33235]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:24.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:24 smithi063 ceph-mon[29809]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:24.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:24 smithi071 ceph-mon[33293]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:26.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:26 smithi165 ceph-mon[33235]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:26.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:26 smithi063 ceph-mon[29809]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:26.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:26 smithi071 ceph-mon[33293]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:28 smithi063 ceph-mon[29809]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:28.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:28 smithi071 ceph-mon[33293]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:29.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:28 smithi165 ceph-mon[33235]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:31.197 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:30 smithi165 ceph-mon[33235]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:31.198 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:31.198 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:31.198 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:31.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:30 smithi071 ceph-mon[33293]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:31.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:31.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:31.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:31.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:30 smithi063 ceph-mon[29809]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:31.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:31.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:31.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:33.058 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:32 smithi165 ceph-mon[33235]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:33.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:32 smithi071 ceph-mon[33293]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:33.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:32 smithi063 ceph-mon[29809]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: from='osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T13:23:34.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:34 smithi165 ceph-mon[33235]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T13:23:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.667+0000 7ff43a000700 1 -- 172.21.15.165:0/769957289 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff43c063a50 con 0x7ff4240b2880 2024-05-11T13:23:34.669 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 5 on host 'smithi165' 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.670+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff4240b2880 msgr2=0x7ff4240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.670+0000 7ff444200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff4240b2880 0x7ff4240b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b5c0 tx=0x7ff430005fd0 comp rx=0 tx=0).stop 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.670+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c107390 msgr2=0x7ff43c1a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.670+0000 7ff444200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c107390 0x7ff43c1a2e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff43400add0 tx=0x7ff43400c330 comp rx=0 tx=0).stop 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 shutdown_connections 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff4240b2880 0x7ff4240b4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c107390 0x7ff43c1a2e20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43c1057f0 0x7ff43c19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 --2- 172.21.15.165:0/769957289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c1069f0 0x7ff43c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 >> 172.21.15.165:0/769957289 conn(0x7ff43c100db0 msgr2=0x7ff43c109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:23:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 shutdown_connections 2024-05-11T13:23:34.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:23:34.671+0000 7ff444200700 1 -- 172.21.15.165:0/769957289 wait complete. 2024-05-11T13:23:34.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:34.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:34.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:34.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: from='osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T13:23:34.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:34 smithi063 ceph-mon[29809]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T13:23:34.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:34.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:34.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:34.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:34.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: from='osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T13:23:34.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:34 smithi071 ceph-mon[33293]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T13:23:35.360 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-11T13:23:35.360 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: osdmap e35: 6 total, 5 up, 6 in 2024-05-11T13:23:35.360 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:35.361 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: from='osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:23:35.361 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:23:35.361 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:35.361 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:35.361 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:35.498 DEBUG:teuthology.orchestra.run.smithi165:osd.5> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.5.service 2024-05-11T13:23:35.502 INFO:tasks.cephadm:Deploying osd.6 on smithi063 with /dev/vg_nvme/lv_2... 2024-05-11T13:23:35.502 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: osdmap e35: 6 total, 5 up, 6 in 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: from='osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:35.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: osdmap e35: 6 total, 5 up, 6 in 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: from='osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:23:35.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:35.958 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:23:36.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:36 smithi063 ceph-mon[29809]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:23:36.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:36 smithi063 ceph-mon[29809]: osdmap e36: 6 total, 5 up, 6 in 2024-05-11T13:23:36.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:36 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:36.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:36 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:36.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:36 smithi063 ceph-mon[29809]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:36 smithi071 ceph-mon[33293]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:23:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:36 smithi071 ceph-mon[33293]: osdmap e36: 6 total, 5 up, 6 in 2024-05-11T13:23:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:36 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:36 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:36 smithi071 ceph-mon[33293]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:36.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:36 smithi165 ceph-mon[33235]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:23:36.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:36 smithi165 ceph-mon[33235]: osdmap e36: 6 total, 5 up, 6 in 2024-05-11T13:23:36.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:36 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:36.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:36 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:36.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:36 smithi165 ceph-mon[33235]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-11T13:23:37.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:37 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:23:37.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:37 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:23:37.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:37 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:37.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:37 smithi063 ceph-mon[29809]: osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] boot 2024-05-11T13:23:37.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:37 smithi063 ceph-mon[29809]: osdmap e37: 6 total, 6 up, 6 in 2024-05-11T13:23:37.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:37 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:37.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:37 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:23:37.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:37 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:23:37.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:37 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:37.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:37 smithi071 ceph-mon[33293]: osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] boot 2024-05-11T13:23:37.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:37 smithi071 ceph-mon[33293]: osdmap e37: 6 total, 6 up, 6 in 2024-05-11T13:23:37.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:37 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:37.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:37 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:23:37.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:37 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:23:37.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:37 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:37.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:37 smithi165 ceph-mon[33235]: osd.5 [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] boot 2024-05-11T13:23:37.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:37 smithi165 ceph-mon[33235]: osdmap e37: 6 total, 6 up, 6 in 2024-05-11T13:23:37.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:37 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:23:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:38 smithi063 ceph-mon[29809]: osdmap e38: 6 total, 6 up, 6 in 2024-05-11T13:23:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:38 smithi063 ceph-mon[29809]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:38.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:38 smithi071 ceph-mon[33293]: osdmap e38: 6 total, 6 up, 6 in 2024-05-11T13:23:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:38 smithi071 ceph-mon[33293]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:38.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:38 smithi165 ceph-mon[33235]: osdmap e38: 6 total, 6 up, 6 in 2024-05-11T13:23:38.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:38 smithi165 ceph-mon[33235]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:39.033 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-11T13:23:39.034 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-11T13:23:39.034 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-11T13:23:39.034 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-11T13:23:39.034 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0379961 s, 276 MB/s 2024-05-11T13:23:39.034 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-11T13:23:39.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:39 smithi063 ceph-mon[29809]: osdmap e39: 6 total, 6 up, 6 in 2024-05-11T13:23:39.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:39 smithi071 ceph-mon[33293]: osdmap e39: 6 total, 6 up, 6 in 2024-05-11T13:23:39.720 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:39 smithi165 ceph-mon[33235]: osdmap e39: 6 total, 6 up, 6 in 2024-05-11T13:23:39.869 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_2 2024-05-11T13:23:40.303 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi165 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi165 to 8544M 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:41.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:41.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi165 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi165 to 8544M 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:41.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi165 2024-05-11T13:23:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:41.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:23:41.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi165 to 8544M 2024-05-11T13:23:41.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:41.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:41.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:23:41.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:23:42.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 -- 172.21.15.63:0/1772319996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 msgr2=0x7fb440106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 --2- 172.21.15.63:0/1772319996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb440106430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb430009990 tx=0x7fb430009ca0 comp rx=0 tx=0).stop 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 -- 172.21.15.63:0/1772319996 shutdown_connections 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 --2- 172.21.15.63:0/1772319996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4401092a0 0x7fb44010b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 --2- 172.21.15.63:0/1772319996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb440106430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 --2- 172.21.15.63:0/1772319996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb440106970 0x7fb440108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 -- 172.21.15.63:0/1772319996 >> 172.21.15.63:0/1772319996 conn(0x7fb4400fda50 msgr2=0x7fb4400ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 -- 172.21.15.63:0/1772319996 shutdown_connections 2024-05-11T13:23:42.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.614+0000 7fb445e00700 1 -- 172.21.15.63:0/1772319996 wait complete. 2024-05-11T13:23:42.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 Processor -- start 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 -- start start 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb4401039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb440106970 0x7fb440101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4401092a0 0x7fb440102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb44010ef70 con 0x7fb440104040 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb44010edf0 con 0x7fb4401092a0 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.615+0000 7fb445e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb44010f0f0 con 0x7fb440106970 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb4401039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb4401039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45912/0 (socket says 172.21.15.63:45912) 2024-05-11T13:23:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 -- 172.21.15.63:0/68301778 learned_addr learned my addr 172.21.15.63:0/68301778 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43e000700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb440106970 0x7fb440101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43f400700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4401092a0 0x7fb440102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 -- 172.21.15.63:0/68301778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb440106970 msgr2=0x7fb440101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb440106970 0x7fb440101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 -- 172.21.15.63:0/68301778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4401092a0 msgr2=0x7fb440102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4401092a0 0x7fb440102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43ea00700 1 -- 172.21.15.63:0/68301778 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb440102de0 con 0x7fb440104040 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43f400700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4401092a0 0x7fb440102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.616+0000 7fb43e000700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb440106970 0x7fb440101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.617+0000 7fb43ea00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb4401039a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb430009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.617+0000 7fb444a00700 1 -- 172.21.15.63:0/68301778 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430027070 con 0x7fb440104040 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.617+0000 7fb444a00700 1 -- 172.21.15.63:0/68301778 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4300040a0 con 0x7fb440104040 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.617+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4401a3810 con 0x7fb440104040 2024-05-11T13:23:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.617+0000 7fb444a00700 1 -- 172.21.15.63:0/68301778 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430022d50 con 0x7fb440104040 2024-05-11T13:23:42.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.618+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4401a3c60 con 0x7fb440104040 2024-05-11T13:23:42.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.619+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44006a240 con 0x7fb440104040 2024-05-11T13:23:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.621+0000 7fb444a00700 1 -- 172.21.15.63:0/68301778 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb43002a070 con 0x7fb440104040 2024-05-11T13:23:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.623+0000 7fb444a00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4240b2900 0x7fb4240b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:23:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.623+0000 7fb444a00700 1 -- 172.21.15.63:0/68301778 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7fb4300e2d00 con 0x7fb440104040 2024-05-11T13:23:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.623+0000 7fb43e000700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4240b2900 0x7fb4240b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:23:42.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.624+0000 7fb43e000700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4240b2900 0x7fb4240b4dc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00bcf0 tx=0x7fb42c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:23:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.626+0000 7fb444a00700 1 -- 172.21.15.63:0/68301778 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb430013070 con 0x7fb440104040 2024-05-11T13:23:42.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:23:42.879+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb440063a50 con 0x7fb4240b2900 2024-05-11T13:23:43.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:42 smithi063 ceph-mon[29809]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:23:43.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:42 smithi071 ceph-mon[33293]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:23:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:42 smithi165 ceph-mon[33235]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:23:44.105 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:43 smithi063 ceph-mon[29809]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:23:44.106 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:23:44.106 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:23:44.106 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:44.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:43 smithi071 ceph-mon[33293]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:23:44.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:23:44.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:23:44.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:43 smithi165 ceph-mon[33235]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:23:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:23:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:23:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:45.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:44 smithi063 ceph-mon[29809]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-11T13:23:45.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:44 smithi071 ceph-mon[33293]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-11T13:23:45.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:44 smithi165 ceph-mon[33235]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-11T13:23:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:46 smithi063 ceph-mon[29809]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-11T13:23:47.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:46 smithi071 ceph-mon[33293]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-11T13:23:47.225 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:46 smithi165 ceph-mon[33235]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-11T13:23:48.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:47 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1558516499' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18fec429-9a88-4389-b373-438e9cad3dde"}]: dispatch 2024-05-11T13:23:48.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:47 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1558516499' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18fec429-9a88-4389-b373-438e9cad3dde"}]': finished 2024-05-11T13:23:48.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:47 smithi063 ceph-mon[29809]: osdmap e40: 7 total, 6 up, 7 in 2024-05-11T13:23:48.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:23:48.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:47 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1558516499' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18fec429-9a88-4389-b373-438e9cad3dde"}]: dispatch 2024-05-11T13:23:48.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:47 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1558516499' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18fec429-9a88-4389-b373-438e9cad3dde"}]': finished 2024-05-11T13:23:48.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:47 smithi071 ceph-mon[33293]: osdmap e40: 7 total, 6 up, 7 in 2024-05-11T13:23:48.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:47 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:23:48.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:47 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1558516499' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18fec429-9a88-4389-b373-438e9cad3dde"}]: dispatch 2024-05-11T13:23:48.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:47 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1558516499' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18fec429-9a88-4389-b373-438e9cad3dde"}]': finished 2024-05-11T13:23:48.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:47 smithi165 ceph-mon[33235]: osdmap e40: 7 total, 6 up, 7 in 2024-05-11T13:23:48.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:23:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:48 smithi063 ceph-mon[29809]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-11T13:23:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:48 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2878787518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:23:49.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:48 smithi071 ceph-mon[33293]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-11T13:23:49.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:48 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2878787518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:23:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:48 smithi165 ceph-mon[33235]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-11T13:23:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:48 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2878787518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:23:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:50 smithi165 ceph-mon[33235]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:23:51.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:50 smithi063 ceph-mon[29809]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:23:51.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:50 smithi071 ceph-mon[33293]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:23:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:52 smithi165 ceph-mon[33235]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:53.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:52 smithi063 ceph-mon[29809]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:53.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:52 smithi071 ceph-mon[33293]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:54.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:54 smithi063 ceph-mon[29809]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:55.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:54 smithi071 ceph-mon[33293]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:55.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:54 smithi165 ceph-mon[33235]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:57.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:56 smithi063 ceph-mon[29809]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:57.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-11T13:23:57.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:56 smithi071 ceph-mon[33293]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-11T13:23:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:57.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:56 smithi165 ceph-mon[33235]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:57.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-11T13:23:57.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:23:58.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:57 smithi063 ceph-mon[29809]: Deploying daemon osd.6 on smithi063 2024-05-11T13:23:58.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:57 smithi071 ceph-mon[33293]: Deploying daemon osd.6 on smithi063 2024-05-11T13:23:58.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:57 smithi165 ceph-mon[33235]: Deploying daemon osd.6 on smithi063 2024-05-11T13:23:59.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:23:58 smithi063 ceph-mon[29809]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:59.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:23:58 smithi071 ceph-mon[33293]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:23:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:23:58 smithi165 ceph-mon[33235]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:01.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:00 smithi063 ceph-mon[29809]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:01.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:00 smithi071 ceph-mon[33293]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:01.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:00 smithi165 ceph-mon[33235]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:03.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:02 smithi063 ceph-mon[29809]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:03.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:02 smithi071 ceph-mon[33293]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:03.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:02 smithi165 ceph-mon[33235]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:05.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:04 smithi063 ceph-mon[29809]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:05.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:04 smithi071 ceph-mon[33293]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:04 smithi165 ceph-mon[33235]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:06.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:06 smithi165 ceph-mon[33235]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:06.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:06.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:06.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:06.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:06 smithi063 ceph-mon[29809]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:06.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:06.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:06.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:06.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:06 smithi071 ceph-mon[33293]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:06.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:06.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:06.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:08.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:08 smithi165 ceph-mon[33235]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:08.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:08 smithi063 ceph-mon[29809]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:08.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:08 smithi071 ceph-mon[33293]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:10.449 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.449 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.449 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:10.449 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:10.450 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.450 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:10 smithi063 ceph-mon[29809]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:10.450 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:10 smithi063 ceph-mon[29809]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-11T13:24:10.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:10.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:10.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:10 smithi071 ceph-mon[33293]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:10.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:10 smithi071 ceph-mon[33293]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-11T13:24:10.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:10.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:10.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:10.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:10 smithi165 ceph-mon[33235]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:10.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:10 smithi165 ceph-mon[33235]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-11T13:24:10.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.932+0000 7fb444a00700 1 -- 172.21.15.63:0/68301778 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb440063a50 con 0x7fb4240b2900 2024-05-11T13:24:10.933 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 6 on host 'smithi063' 2024-05-11T13:24:10.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4240b2900 msgr2=0x7fb4240b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:10.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4240b2900 0x7fb4240b4dc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00bcf0 tx=0x7fb42c00b2c0 comp rx=0 tx=0).stop 2024-05-11T13:24:10.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 msgr2=0x7fb4401039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:10.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb4401039a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb430009580 comp rx=0 tx=0).stop 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 shutdown_connections 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4240b2900 0x7fb4240b4dc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4401092a0 0x7fb440102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb440104040 0x7fb4401039a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 --2- 172.21.15.63:0/68301778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb440106970 0x7fb440101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 >> 172.21.15.63:0/68301778 conn(0x7fb4400fda50 msgr2=0x7fb440105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 shutdown_connections 2024-05-11T13:24:10.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:24:10.936+0000 7fb445e00700 1 -- 172.21.15.63:0/68301778 wait complete. 2024-05-11T13:24:11.662 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:11 smithi063 ceph-mon[29809]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-11T13:24:11.662 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:11 smithi063 ceph-mon[29809]: osdmap e41: 7 total, 6 up, 7 in 2024-05-11T13:24:11.663 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:11 smithi063 ceph-mon[29809]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:24:11.663 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:11 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:11.663 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:11 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:11.663 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:11 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:11.663 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:11 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:11.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:11 smithi071 ceph-mon[33293]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-11T13:24:11.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:11 smithi071 ceph-mon[33293]: osdmap e41: 7 total, 6 up, 7 in 2024-05-11T13:24:11.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:11 smithi071 ceph-mon[33293]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:24:11.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:11 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:11.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:11 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:11.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:11 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:11.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:11 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:11.732 DEBUG:teuthology.orchestra.run.smithi063:osd.6> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.6.service 2024-05-11T13:24:11.737 INFO:tasks.cephadm:Deploying osd.7 on smithi071 with /dev/vg_nvme/lv_2... 2024-05-11T13:24:11.737 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-11T13:24:11.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:11 smithi165 ceph-mon[33235]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-11T13:24:11.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:11 smithi165 ceph-mon[33235]: osdmap e41: 7 total, 6 up, 7 in 2024-05-11T13:24:11.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:11 smithi165 ceph-mon[33235]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:24:11.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:11 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:11.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:11 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:11.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:11 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:11.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:11 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:12.179 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:24:12.681 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:12 smithi071 ceph-mon[33293]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:24:12.682 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:12 smithi071 ceph-mon[33293]: osdmap e42: 7 total, 6 up, 7 in 2024-05-11T13:24:12.682 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:12 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:12.682 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:12 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:12.682 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:12 smithi071 ceph-mon[33293]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:12.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:12 smithi063 ceph-mon[29809]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:24:12.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:12 smithi063 ceph-mon[29809]: osdmap e42: 7 total, 6 up, 7 in 2024-05-11T13:24:12.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:12 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:12.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:12 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:12 smithi063 ceph-mon[29809]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:12 smithi165 ceph-mon[33235]: from='osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:24:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:12 smithi165 ceph-mon[33235]: osdmap e42: 7 total, 6 up, 7 in 2024-05-11T13:24:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:12 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:12 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:12 smithi165 ceph-mon[33235]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-11T13:24:13.593 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:13 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:24:13.593 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:13 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:24:13.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:13 smithi063 ceph-mon[29809]: osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] boot 2024-05-11T13:24:13.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:13 smithi063 ceph-mon[29809]: osdmap e43: 7 total, 7 up, 7 in 2024-05-11T13:24:13.594 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:13 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:13.684 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:13 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:24:13.685 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:13 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:24:13.685 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:13 smithi071 ceph-mon[33293]: osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] boot 2024-05-11T13:24:13.685 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:13 smithi071 ceph-mon[33293]: osdmap e43: 7 total, 7 up, 7 in 2024-05-11T13:24:13.685 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:13 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:13 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:24:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:13 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:24:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:13 smithi165 ceph-mon[33235]: osd.6 [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] boot 2024-05-11T13:24:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:13 smithi165 ceph-mon[33235]: osdmap e43: 7 total, 7 up, 7 in 2024-05-11T13:24:13.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:13 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:24:14.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:14 smithi071 ceph-mon[33293]: osdmap e44: 7 total, 7 up, 7 in 2024-05-11T13:24:14.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:14 smithi071 ceph-mon[33293]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:14 smithi165 ceph-mon[33235]: osdmap e44: 7 total, 7 up, 7 in 2024-05-11T13:24:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:14 smithi165 ceph-mon[33235]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:14 smithi063 ceph-mon[29809]: osdmap e44: 7 total, 7 up, 7 in 2024-05-11T13:24:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:14 smithi063 ceph-mon[29809]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:15.257 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-11T13:24:15.257 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-11T13:24:15.257 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-11T13:24:15.257 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-11T13:24:15.258 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0597045 s, 176 MB/s 2024-05-11T13:24:15.258 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-11T13:24:16.239 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-05-11T13:24:16.599 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:16 smithi063 ceph-mon[29809]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:16.657 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:24:16.808 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:16 smithi071 ceph-mon[33293]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:16.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:16 smithi165 ceph-mon[33235]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi063 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi063 to 5696M 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:18.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.150 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi063 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi063 to 5696M 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:18.151 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi063 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi063 to 5696M 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:18.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:19.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 -- 172.21.15.71:0/3671644975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 msgr2=0x7fc9b0112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:19.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3671644975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b0112040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b0d0 tx=0x7fc9a800b3e0 comp rx=0 tx=0).stop 2024-05-11T13:24:19.054 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:18 smithi071 ceph-mon[33293]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:19.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 -- 172.21.15.71:0/3671644975 shutdown_connections 2024-05-11T13:24:19.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3671644975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b0112040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:19.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3671644975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0109bf0 0x7fc9b010a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:19.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3671644975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9b010adf0 0x7fc9b010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 -- 172.21.15.71:0/3671644975 >> 172.21.15.71:0/3671644975 conn(0x7fc9b0078bc0 msgr2=0x7fc9b007b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 -- 172.21.15.71:0/3671644975 shutdown_connections 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.051+0000 7fc9b6400700 1 -- 172.21.15.71:0/3671644975 wait complete. 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 Processor -- start 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 -- start start 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0109bf0 0x7fc9b019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9b010adf0 0x7fc9b01a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b01a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b0115540 con 0x7fc9b0109bf0 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b01153c0 con 0x7fc9b010b790 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.053+0000 7fc9b6400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b01156c0 con 0x7fc9b010adf0 2024-05-11T13:24:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b01a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b01a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34730/0 (socket says 172.21.15.71:34730) 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 -- 172.21.15.71:0/3837591474 learned_addr learned my addr 172.21.15.71:0/3837591474 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9a7e00700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9b010adf0 0x7fc9b01a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9af400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0109bf0 0x7fc9b019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 -- 172.21.15.71:0/3837591474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9b010adf0 msgr2=0x7fc9b01a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9b010adf0 0x7fc9b01a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 -- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0109bf0 msgr2=0x7fc9b019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0109bf0 0x7fc9b019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 -- 172.21.15.71:0/3837591474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b01a7810 con 0x7fc9b010b790 2024-05-11T13:24:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.054+0000 7fc9afe00700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b01a7170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8004fd0 tx=0x7fc9a8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:24:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.055+0000 7fc9acc00700 1 -- 172.21.15.71:0/3837591474 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a801f070 con 0x7fc9b010b790 2024-05-11T13:24:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.055+0000 7fc9acc00700 1 -- 172.21.15.71:0/3837591474 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a80054d0 con 0x7fc9b010b790 2024-05-11T13:24:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.055+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b01a7aa0 con 0x7fc9b010b790 2024-05-11T13:24:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.055+0000 7fc9acc00700 1 -- 172.21.15.71:0/3837591474 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a801b850 con 0x7fc9b010b790 2024-05-11T13:24:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.055+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b01a7f50 con 0x7fc9b010b790 2024-05-11T13:24:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.056+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b006a240 con 0x7fc9b010b790 2024-05-11T13:24:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.058+0000 7fc9acc00700 1 -- 172.21.15.71:0/3837591474 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc9a8022070 con 0x7fc9b010b790 2024-05-11T13:24:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.060+0000 7fc9acc00700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc9940b2980 0x7fc9940b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.060+0000 7fc9acc00700 1 -- 172.21.15.71:0/3837591474 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fc9a800f020 con 0x7fc9b010b790 2024-05-11T13:24:19.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.061+0000 7fc9af400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc9940b2980 0x7fc9940b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:24:19.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.062+0000 7fc9af400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc9940b2980 0x7fc9940b4e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc9b010ac50 tx=0x7fc9a000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:24:19.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.064+0000 7fc9acc00700 1 -- 172.21.15.71:0/3837591474 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9a8029020 con 0x7fc9b010b790 2024-05-11T13:24:19.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:18 smithi165 ceph-mon[33235]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:19.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:18 smithi063 ceph-mon[29809]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:19.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:19.304+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc9b0063990 con 0x7fc9940b2980 2024-05-11T13:24:20.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:19 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:24:20.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:19 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:24:20.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:19 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:20.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:19 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:24:20.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:19 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:24:20.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:19 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:20.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:19 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:24:20.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:19 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:24:20.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:19 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:20.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:20 smithi071 ceph-mon[33293]: from='client.34259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:24:20.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:20 smithi071 ceph-mon[33293]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:21.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:20 smithi165 ceph-mon[33235]: from='client.34259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:24:21.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:20 smithi165 ceph-mon[33235]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:21.099 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:20 smithi063 ceph-mon[29809]: from='client.34259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:24:21.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:20 smithi063 ceph-mon[29809]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:23.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:22 smithi165 ceph-mon[33235]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:23.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:22 smithi063 ceph-mon[29809]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:23.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:22 smithi071 ceph-mon[33293]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:23 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/3724296030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]: dispatch 2024-05-11T13:24:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:23 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]: dispatch 2024-05-11T13:24:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:23 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]': finished 2024-05-11T13:24:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:23 smithi165 ceph-mon[33235]: osdmap e45: 8 total, 7 up, 8 in 2024-05-11T13:24:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:23 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:24.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:23 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/3724296030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]: dispatch 2024-05-11T13:24:24.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:23 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]: dispatch 2024-05-11T13:24:24.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:23 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]': finished 2024-05-11T13:24:24.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:23 smithi063 ceph-mon[29809]: osdmap e45: 8 total, 7 up, 8 in 2024-05-11T13:24:24.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:23 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:24.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:23 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/3724296030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]: dispatch 2024-05-11T13:24:24.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:23 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]: dispatch 2024-05-11T13:24:24.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:23 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad08b6e9-8303-4d9b-8a92-f837c203f300"}]': finished 2024-05-11T13:24:24.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:23 smithi071 ceph-mon[33293]: osdmap e45: 8 total, 7 up, 8 in 2024-05-11T13:24:24.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:23 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:25.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:24 smithi165 ceph-mon[33235]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:25.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:24 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/1859554281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:24:25.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:24 smithi063 ceph-mon[29809]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:25.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:24 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/1859554281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:24:25.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:24 smithi071 ceph-mon[33293]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:25.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:24 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/1859554281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:24:27.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:26 smithi165 ceph-mon[33235]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:27.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:26 smithi063 ceph-mon[29809]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:27.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:26 smithi071 ceph-mon[33293]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:29.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:28 smithi063 ceph-mon[29809]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:29.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:28 smithi071 ceph-mon[33293]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:28 smithi165 ceph-mon[33235]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:30.949 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:30 smithi071 ceph-mon[33293]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:31.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:30 smithi063 ceph-mon[29809]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:31.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:30 smithi165 ceph-mon[33235]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:33.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:32 smithi063 ceph-mon[29809]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:33.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-11T13:24:33.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:32 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:33.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:32 smithi071 ceph-mon[33293]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:33.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:32 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-11T13:24:33.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:32 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:33.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:32 smithi165 ceph-mon[33235]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:33.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:32 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-11T13:24:33.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:32 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:33.931 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:33 smithi071 ceph-mon[33293]: Deploying daemon osd.7 on smithi071 2024-05-11T13:24:34.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:33 smithi063 ceph-mon[29809]: Deploying daemon osd.7 on smithi071 2024-05-11T13:24:34.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:33 smithi165 ceph-mon[33235]: Deploying daemon osd.7 on smithi071 2024-05-11T13:24:35.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:34 smithi063 ceph-mon[29809]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:35.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:34 smithi071 ceph-mon[33293]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:34 smithi165 ceph-mon[33235]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:37.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:36 smithi063 ceph-mon[29809]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:37.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:36 smithi071 ceph-mon[33293]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:37.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:36 smithi165 ceph-mon[33235]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:39.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:38 smithi063 ceph-mon[29809]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:39.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:38 smithi071 ceph-mon[33293]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:39.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:38 smithi165 ceph-mon[33235]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:41.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:40 smithi071 ceph-mon[33293]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:41.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:40 smithi063 ceph-mon[29809]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:40 smithi165 ceph-mon[33235]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:42.557 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:42 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:42.557 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:42 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:42.557 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:42 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:42.557 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:42 smithi071 ceph-mon[33293]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:42.650 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:42.650 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:42.651 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:42.651 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:42 smithi063 ceph-mon[29809]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:42.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:42.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:42 smithi165 ceph-mon[33235]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:44.633 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:44 smithi071 ceph-mon[33293]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:44.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:44 smithi165 ceph-mon[33235]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:44.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:44 smithi063 ceph-mon[29809]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:46.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.324+0000 7fc9acc00700 1 -- 172.21.15.71:0/3837591474 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc9b0063990 con 0x7fc9940b2980 2024-05-11T13:24:46.324 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 7 on host 'smithi071' 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.327+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc9940b2980 msgr2=0x7fc9940b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.327+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc9940b2980 0x7fc9940b4e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc9b010ac50 tx=0x7fc9a000b2c0 comp rx=0 tx=0).stop 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.327+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 msgr2=0x7fc9b01a7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.327+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b01a7170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8004fd0 tx=0x7fc9a8009c30 comp rx=0 tx=0).stop 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 shutdown_connections 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc9940b2980 0x7fc9940b4e40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9b010b790 0x7fc9b01a7170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0109bf0 0x7fc9b019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 --2- 172.21.15.71:0/3837591474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9b010adf0 0x7fc9b01a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:46.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 >> 172.21.15.71:0/3837591474 conn(0x7fc9b0078bc0 msgr2=0x7fc9b010e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:24:46.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 shutdown_connections 2024-05-11T13:24:46.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:24:46.328+0000 7fc9b6400700 1 -- 172.21.15.71:0/3837591474 wait complete. 2024-05-11T13:24:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T13:24:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T13:24:46.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:46.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:46.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:46.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:46.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:46.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T13:24:46.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T13:24:46.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:46.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:46.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:24:46.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:47.133 DEBUG:teuthology.orchestra.run.smithi071:osd.7> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.7.service 2024-05-11T13:24:47.138 INFO:tasks.cephadm:Deploying osd.8 on smithi165 with /dev/vg_nvme/lv_2... 2024-05-11T13:24:47.138 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-11T13:24:47.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:47 smithi165 ceph-mon[33235]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-11T13:24:47.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:47 smithi165 ceph-mon[33235]: from='osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:24:47.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:47 smithi165 ceph-mon[33235]: osdmap e46: 8 total, 7 up, 8 in 2024-05-11T13:24:47.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:47.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:47 smithi165 ceph-mon[33235]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:24:47.587 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:24:47.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:47 smithi063 ceph-mon[29809]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-11T13:24:47.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:47 smithi063 ceph-mon[29809]: from='osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:24:47.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:47 smithi063 ceph-mon[29809]: osdmap e46: 8 total, 7 up, 8 in 2024-05-11T13:24:47.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:47.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:47 smithi063 ceph-mon[29809]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:24:47.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:47 smithi071 ceph-mon[33293]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-11T13:24:47.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:47 smithi071 ceph-mon[33293]: from='osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:24:47.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:47 smithi071 ceph-mon[33293]: osdmap e46: 8 total, 7 up, 8 in 2024-05-11T13:24:47.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:47 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:47.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:47 smithi071 ceph-mon[33293]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:24:47.691 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 13:24:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T13:24:47.463+0000 7f465ac00700 -1 osd.7 0 waiting for initial osdmap 2024-05-11T13:24:47.691 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 13:24:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T13:24:47.497+0000 7f4655a00700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:24:48.602 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:48 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:24:48.602 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:48 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:24:48.602 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:48 smithi071 ceph-mon[33293]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:24:48.603 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:48 smithi071 ceph-mon[33293]: osdmap e47: 8 total, 7 up, 8 in 2024-05-11T13:24:48.603 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:48.603 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:48.603 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:48 smithi071 ceph-mon[33293]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:48.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:48 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:24:48.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:48 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:24:48.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:48 smithi165 ceph-mon[33235]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:24:48.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:48 smithi165 ceph-mon[33235]: osdmap e47: 8 total, 7 up, 8 in 2024-05-11T13:24:48.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:48.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:48.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:48 smithi165 ceph-mon[33235]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:48 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:24:48.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:48 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:24:48.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:48 smithi063 ceph-mon[29809]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:24:48.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:48 smithi063 ceph-mon[29809]: osdmap e47: 8 total, 7 up, 8 in 2024-05-11T13:24:48.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:48.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:48.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:48 smithi063 ceph-mon[29809]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-11T13:24:49.539 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:49 smithi165 ceph-mon[33235]: osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] boot 2024-05-11T13:24:49.540 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:49 smithi165 ceph-mon[33235]: osdmap e48: 8 total, 8 up, 8 in 2024-05-11T13:24:49.540 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:49 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:49.540 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:49 smithi165 ceph-mon[33235]: osdmap e49: 8 total, 8 up, 8 in 2024-05-11T13:24:49.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:49 smithi063 ceph-mon[29809]: osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] boot 2024-05-11T13:24:49.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:49 smithi063 ceph-mon[29809]: osdmap e48: 8 total, 8 up, 8 in 2024-05-11T13:24:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:49 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:49 smithi063 ceph-mon[29809]: osdmap e49: 8 total, 8 up, 8 in 2024-05-11T13:24:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:49 smithi071 ceph-mon[33293]: osd.7 [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] boot 2024-05-11T13:24:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:49 smithi071 ceph-mon[33293]: osdmap e48: 8 total, 8 up, 8 in 2024-05-11T13:24:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:49 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:24:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:49 smithi071 ceph-mon[33293]: osdmap e49: 8 total, 8 up, 8 in 2024-05-11T13:24:50.555 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-11T13:24:50.555 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-11T13:24:50.555 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-11T13:24:50.555 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-11T13:24:50.555 INFO:teuthology.orchestra.run.smithi165.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350777 s, 299 MB/s 2024-05-11T13:24:50.555 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-11T13:24:50.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:50 smithi165 ceph-mon[33235]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:24:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:50 smithi063 ceph-mon[29809]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:24:50.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:50 smithi071 ceph-mon[33293]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:24:51.466 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-05-11T13:24:51.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:51 smithi071 ceph-mon[33293]: osdmap e50: 8 total, 8 up, 8 in 2024-05-11T13:24:51.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:51 smithi165 ceph-mon[33235]: osdmap e50: 8 total, 8 up, 8 in 2024-05-11T13:24:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:24:51.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:51 smithi063 ceph-mon[29809]: osdmap e50: 8 total, 8 up, 8 in 2024-05-11T13:24:53.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi071 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:53.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:52 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi071 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:53.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:53.318 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:52 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:24:53.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi071 2024-05-11T13:24:53.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:24:53.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:53.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:53.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:24:53.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:52 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 -- 172.21.15.165:0/980225906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001069f0 msgr2=0x7f0c00106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 --2- 172.21.15.165:0/980225906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001069f0 0x7f0c00106e50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009990 tx=0x7f0bec009ca0 comp rx=0 tx=0).stop 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 -- 172.21.15.165:0/980225906 shutdown_connections 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 --2- 172.21.15.165:0/980225906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c00107390 0x7f0c0010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 --2- 172.21.15.165:0/980225906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c001057f0 0x7f0c00105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 --2- 172.21.15.165:0/980225906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001069f0 0x7f0c00106e50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 -- 172.21.15.165:0/980225906 >> 172.21.15.165:0/980225906 conn(0x7f0c00100db0 msgr2=0x7f0c001031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 -- 172.21.15.165:0/980225906 shutdown_connections 2024-05-11T13:24:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.188+0000 7f0c05e00700 1 -- 172.21.15.165:0/980225906 wait complete. 2024-05-11T13:24:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 Processor -- start 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 -- start start 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001057f0 0x7f0c0019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c001069f0 0x7f0c0019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c00107390 0x7f0c001a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00111190 con 0x7f0c001069f0 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00111010 con 0x7f0c00107390 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.190+0000 7f0c05e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00111310 con 0x7f0c001057f0 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001057f0 0x7f0c0019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001057f0 0x7f0c0019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59482/0 (socket says 172.21.15.165:59482) 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 -- 172.21.15.165:0/462546108 learned_addr learned my addr 172.21.15.165:0/462546108 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 -- 172.21.15.165:0/462546108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c00107390 msgr2=0x7f0c001a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfe000700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c001069f0 0x7f0c0019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:24:54.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c00107390 0x7f0c001a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:54.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 -- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c001069f0 msgr2=0x7f0c0019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:24:54.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c001069f0 0x7f0c0019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:24:54.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 -- 172.21.15.165:0/462546108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001a3430 con 0x7f0c001057f0 2024-05-11T13:24:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.191+0000 7f0bfea00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001057f0 0x7f0c0019b7d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c9c0 tx=0x7f0bf000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:24:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.192+0000 7f0c04a00700 1 -- 172.21.15.165:0/462546108 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0007ac0 con 0x7f0c001057f0 2024-05-11T13:24:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.192+0000 7f0c04a00700 1 -- 172.21.15.165:0/462546108 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bf0007c60 con 0x7f0c001057f0 2024-05-11T13:24:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.192+0000 7f0c04a00700 1 -- 172.21.15.165:0/462546108 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0018660 con 0x7f0c001057f0 2024-05-11T13:24:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.192+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001a3720 con 0x7f0c001057f0 2024-05-11T13:24:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.192+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001a3c10 con 0x7f0c001057f0 2024-05-11T13:24:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.193+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0006a240 con 0x7f0c001057f0 2024-05-11T13:24:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.195+0000 7f0c04a00700 1 -- 172.21.15.165:0/462546108 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0bf0004020 con 0x7f0c001057f0 2024-05-11T13:24:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.197+0000 7f0c04a00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0be40b2900 0x7f0be40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:24:54.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.197+0000 7f0c04a00700 1 -- 172.21.15.165:0/462546108 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f0bf00e1870 con 0x7f0c001057f0 2024-05-11T13:24:54.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.197+0000 7f0bfe000700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0be40b2900 0x7f0be40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:24:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.199+0000 7f0bfe000700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0be40b2900 0x7f0be40b4dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006010 tx=0x7f0bec009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:24:54.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.200+0000 7f0c04a00700 1 -- 172.21.15.165:0/462546108 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0bf0053df0 con 0x7f0c001057f0 2024-05-11T13:24:54.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:53 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi071 to 5696M 2024-05-11T13:24:54.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:53 smithi165 ceph-mon[33235]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:24:54.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:24:54.426+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0c00063990 con 0x7f0be40b2900 2024-05-11T13:24:54.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:53 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi071 to 5696M 2024-05-11T13:24:54.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:53 smithi063 ceph-mon[29809]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:24:54.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:53 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi071 to 5696M 2024-05-11T13:24:54.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:53 smithi071 ceph-mon[33293]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:24:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:55 smithi165 ceph-mon[33235]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:24:55.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:55 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:24:55.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:55 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:24:55.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:55 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:55.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:54 smithi063 ceph-mon[29809]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:24:55.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:24:55.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:24:55.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:55.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:55 smithi071 ceph-mon[33293]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:24:55.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:55 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:24:55.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:55 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:24:55.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:55 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:24:56.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:56 smithi165 ceph-mon[33235]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-11T13:24:56.336 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:56 smithi071 ceph-mon[33293]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-11T13:24:56.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:56 smithi063 ceph-mon[29809]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-11T13:24:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:58 smithi165 ceph-mon[33235]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T13:24:58.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:58 smithi063 ceph-mon[29809]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T13:24:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:58 smithi071 ceph-mon[33293]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T13:24:59.641 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:59 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/2238369191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]: dispatch 2024-05-11T13:24:59.641 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:59 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]: dispatch 2024-05-11T13:24:59.642 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:59 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]': finished 2024-05-11T13:24:59.642 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:59 smithi071 ceph-mon[33293]: osdmap e51: 9 total, 8 up, 9 in 2024-05-11T13:24:59.642 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:24:59.642 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:24:59 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/2071988766' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:24:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:59 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/2238369191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]: dispatch 2024-05-11T13:24:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:59 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]: dispatch 2024-05-11T13:24:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:59 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]': finished 2024-05-11T13:24:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:59 smithi165 ceph-mon[33235]: osdmap e51: 9 total, 8 up, 9 in 2024-05-11T13:24:59.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:24:59.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:24:59 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/2071988766' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:24:59.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:59 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/2238369191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]: dispatch 2024-05-11T13:24:59.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:59 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]: dispatch 2024-05-11T13:24:59.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:59 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5ddd20e-3176-42ba-853e-47ca81ab5ef0"}]': finished 2024-05-11T13:24:59.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:59 smithi063 ceph-mon[29809]: osdmap e51: 9 total, 8 up, 9 in 2024-05-11T13:24:59.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:24:59.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:24:59 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/2071988766' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:25:00.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:00 smithi165 ceph-mon[33235]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-11T13:25:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:00 smithi063 ceph-mon[29809]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-11T13:25:00.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:00 smithi071 ceph-mon[33293]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-11T13:25:02.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:02 smithi165 ceph-mon[33235]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:02.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:02 smithi063 ceph-mon[29809]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:02.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:02 smithi071 ceph-mon[33293]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:04.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:04 smithi165 ceph-mon[33235]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:04.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:04 smithi063 ceph-mon[29809]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:04.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:04 smithi071 ceph-mon[33293]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:06 smithi165 ceph-mon[33235]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:06 smithi063 ceph-mon[29809]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:06.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:06 smithi071 ceph-mon[33293]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:08 smithi063 ceph-mon[29809]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-11T13:25:08.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:08 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:08.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:08 smithi071 ceph-mon[33293]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:08.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-11T13:25:08.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:08 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:09.037 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:08 smithi165 ceph-mon[33235]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:09.037 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-11T13:25:09.037 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:08 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:09.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:09 smithi165 ceph-mon[33235]: Deploying daemon osd.8 on smithi165 2024-05-11T13:25:09.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:09 smithi063 ceph-mon[29809]: Deploying daemon osd.8 on smithi165 2024-05-11T13:25:09.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:09 smithi071 ceph-mon[33293]: Deploying daemon osd.8 on smithi165 2024-05-11T13:25:10.894 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:10 smithi165 ceph-mon[33235]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:10.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:10 smithi063 ceph-mon[29809]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:10.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:10 smithi071 ceph-mon[33293]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:12.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:12 smithi063 ceph-mon[29809]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:12.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:12 smithi071 ceph-mon[33293]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:12 smithi165 ceph-mon[33235]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:14 smithi063 ceph-mon[29809]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:14.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:14 smithi071 ceph-mon[33293]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:15.020 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:14 smithi165 ceph-mon[33235]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:16.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:16 smithi165 ceph-mon[33235]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:16.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:16 smithi063 ceph-mon[29809]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:16.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:16 smithi071 ceph-mon[33293]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:17.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:17.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:17.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:17 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:17.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:17.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:17.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:17 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:17.978 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:17.978 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:17.978 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:17 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:18.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:18 smithi071 ceph-mon[33293]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:19.052 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:18 smithi165 ceph-mon[33235]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:19.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:18 smithi063 ceph-mon[29809]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:20.918 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:20 smithi165 ceph-mon[33235]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:20.918 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:20.918 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:20.919 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:20 smithi165 ceph-mon[33235]: from='osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T13:25:20.919 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:20 smithi165 ceph-mon[33235]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T13:25:20.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:20 smithi071 ceph-mon[33293]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:20.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:20.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:20.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:20 smithi071 ceph-mon[33293]: from='osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T13:25:20.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:20 smithi071 ceph-mon[33293]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T13:25:21.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:20 smithi063 ceph-mon[29809]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:21.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:21.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:21.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:20 smithi063 ceph-mon[29809]: from='osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T13:25:21.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:20 smithi063 ceph-mon[29809]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T13:25:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.343+0000 7f0c04a00700 1 -- 172.21.15.165:0/462546108 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c00063990 con 0x7f0be40b2900 2024-05-11T13:25:21.344 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 8 on host 'smithi165' 2024-05-11T13:25:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0be40b2900 msgr2=0x7f0be40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:25:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0be40b2900 0x7f0be40b4dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006010 tx=0x7f0bec009650 comp rx=0 tx=0).stop 2024-05-11T13:25:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001057f0 msgr2=0x7f0c0019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:25:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001057f0 0x7f0c0019b7d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c9c0 tx=0x7f0bf000cd80 comp rx=0 tx=0).stop 2024-05-11T13:25:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 shutdown_connections 2024-05-11T13:25:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0be40b2900 0x7f0be40b4dc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c00107390 0x7f0c001a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c001057f0 0x7f0c0019b7d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 --2- 172.21.15.165:0/462546108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c001069f0 0x7f0c0019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 >> 172.21.15.165:0/462546108 conn(0x7f0c00100db0 msgr2=0x7f0c00109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:25:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 shutdown_connections 2024-05-11T13:25:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:25:21.346+0000 7f0c05e00700 1 -- 172.21.15.165:0/462546108 wait complete. 2024-05-11T13:25:22.184 DEBUG:teuthology.orchestra.run.smithi165:osd.8> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.8.service 2024-05-11T13:25:22.190 INFO:tasks.cephadm:Deploying osd.9 on smithi063 with /dev/vg_nvme/lv_1... 2024-05-11T13:25:22.190 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-11T13:25:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-11T13:25:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: from='osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:25:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: osdmap e52: 9 total, 8 up, 9 in 2024-05-11T13:25:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:22.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:25:22.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:22 smithi165 ceph-mon[33235]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:22.674 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:25:22.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:22.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:22.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-11T13:25:22.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: from='osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:25:22.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: osdmap e52: 9 total, 8 up, 9 in 2024-05-11T13:25:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:25:22.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:22 smithi063 ceph-mon[29809]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: from='osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: osdmap e52: 9 total, 8 up, 9 in 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:25:22.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:22 smithi071 ceph-mon[33293]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-11T13:25:23.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:23 smithi165 ceph-mon[33235]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:25:23.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:23 smithi165 ceph-mon[33235]: osdmap e53: 9 total, 8 up, 9 in 2024-05-11T13:25:23.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:23 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:23.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:23 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:23.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:23 smithi165 ceph-mon[33235]: from='osd.8 ' entity='osd.8' 2024-05-11T13:25:23.567 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 13:25:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T13:25:23.165+0000 7f1ee3a00700 -1 osd.8 0 waiting for initial osdmap 2024-05-11T13:25:23.567 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 13:25:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T13:25:23.190+0000 7f1ed9800700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:25:23.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:23 smithi063 ceph-mon[29809]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:25:23.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:23 smithi063 ceph-mon[29809]: osdmap e53: 9 total, 8 up, 9 in 2024-05-11T13:25:23.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:23 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:23.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:23 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:23.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:23 smithi063 ceph-mon[29809]: from='osd.8 ' entity='osd.8' 2024-05-11T13:25:23.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:23 smithi071 ceph-mon[33293]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:25:23.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:23 smithi071 ceph-mon[33293]: osdmap e53: 9 total, 8 up, 9 in 2024-05-11T13:25:23.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:23 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:23.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:23 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:23.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:23 smithi071 ceph-mon[33293]: from='osd.8 ' entity='osd.8' 2024-05-11T13:25:24.540 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:24 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:25:24.540 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:24 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:25:24.540 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:24 smithi063 ceph-mon[29809]: osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] boot 2024-05-11T13:25:24.540 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:24 smithi063 ceph-mon[29809]: osdmap e54: 9 total, 9 up, 9 in 2024-05-11T13:25:24.540 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:24 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:24.540 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:24 smithi063 ceph-mon[29809]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 659 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:24.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:24 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:25:24.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:24 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:25:24.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:24 smithi165 ceph-mon[33235]: osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] boot 2024-05-11T13:25:24.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:24 smithi165 ceph-mon[33235]: osdmap e54: 9 total, 9 up, 9 in 2024-05-11T13:25:24.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:24 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:24.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:24 smithi165 ceph-mon[33235]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 659 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:24.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:24 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:25:24.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:24 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:25:24.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:24 smithi071 ceph-mon[33293]: osd.8 [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] boot 2024-05-11T13:25:24.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:24 smithi071 ceph-mon[33293]: osdmap e54: 9 total, 9 up, 9 in 2024-05-11T13:25:24.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:24 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:25:24.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:24 smithi071 ceph-mon[33293]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 659 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:25.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:25 smithi063 ceph-mon[29809]: osdmap e55: 9 total, 9 up, 9 in 2024-05-11T13:25:25.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:25 smithi071 ceph-mon[33293]: osdmap e55: 9 total, 9 up, 9 in 2024-05-11T13:25:25.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:25 smithi165 ceph-mon[33235]: osdmap e55: 9 total, 9 up, 9 in 2024-05-11T13:25:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-11T13:25:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-11T13:25:25.832 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-11T13:25:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-11T13:25:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0339999 s, 308 MB/s 2024-05-11T13:25:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-11T13:25:26.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:26 smithi063 ceph-mon[29809]: osdmap e56: 9 total, 9 up, 9 in 2024-05-11T13:25:26.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:26 smithi063 ceph-mon[29809]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 659 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:26 smithi071 ceph-mon[33293]: osdmap e56: 9 total, 9 up, 9 in 2024-05-11T13:25:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:26 smithi071 ceph-mon[33293]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 659 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:26.762 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_1 2024-05-11T13:25:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:26 smithi165 ceph-mon[33235]: osdmap e56: 9 total, 9 up, 9 in 2024-05-11T13:25:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:26 smithi165 ceph-mon[33235]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 659 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:27.207 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:25:28.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi165 2024-05-11T13:25:28.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi165 to 5696M 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:25:28.686 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi165 2024-05-11T13:25:28.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi165 to 5696M 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:25:28.706 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi165 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi165 to 5696M 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:28.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:28.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:28.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:28.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:25:28.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 -- 172.21.15.63:0/264138647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 msgr2=0x7fdddc105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:25:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 --2- 172.21.15.63:0/264138647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc105c20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fddd00099c0 tx=0x7fddd0009cd0 comp rx=0 tx=0).stop 2024-05-11T13:25:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 -- 172.21.15.63:0/264138647 shutdown_connections 2024-05-11T13:25:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 --2- 172.21.15.63:0/264138647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdddc1073a0 0x7fdddc10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 --2- 172.21.15.63:0/264138647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc105c20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 --2- 172.21.15.63:0/264138647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc106a00 0x7fdddc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 -- 172.21.15.63:0/264138647 >> 172.21.15.63:0/264138647 conn(0x7fdddc100da0 msgr2=0x7fdddc1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:25:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 -- 172.21.15.63:0/264138647 shutdown_connections 2024-05-11T13:25:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.597+0000 7fdde3600700 1 -- 172.21.15.63:0/264138647 wait complete. 2024-05-11T13:25:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.598+0000 7fdde3600700 1 Processor -- start 2024-05-11T13:25:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde3600700 1 -- start start 2024-05-11T13:25:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde3600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:25:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde3600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdddc106a00 0x7fdddc19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:25:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde3600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc1073a0 0x7fdddc1a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:25:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde3600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc111170 con 0x7fdddc105800 2024-05-11T13:25:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde3600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc110ff0 con 0x7fdddc1073a0 2024-05-11T13:25:29.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde3600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1112f0 con 0x7fdddc106a00 2024-05-11T13:25:29.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:25:29.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.599+0000 7fdde0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53140/0 (socket says 172.21.15.63:53140) 2024-05-11T13:25:29.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde0c00700 1 -- 172.21.15.63:0/4207746192 learned_addr learned my addr 172.21.15.63:0/4207746192 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:25:29.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdddbe00700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdddc106a00 0x7fdddc19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:25:29.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde1600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc1073a0 0x7fdddc1a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:25:29.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde0c00700 1 -- 172.21.15.63:0/4207746192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdddc106a00 msgr2=0x7fdddc19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:25:29.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde0c00700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdddc106a00 0x7fdddc19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:29.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde0c00700 1 -- 172.21.15.63:0/4207746192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc1073a0 msgr2=0x7fdddc1a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:25:29.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde0c00700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc1073a0 0x7fdddc1a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:29.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde0c00700 1 -- 172.21.15.63:0/4207746192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc1a3470 con 0x7fdddc105800 2024-05-11T13:25:29.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.600+0000 7fdde0c00700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc19b810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fddd000b580 tx=0x7fddd000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:25:29.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.601+0000 7fddd9600700 1 -- 172.21.15.63:0/4207746192 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0027070 con 0x7fdddc105800 2024-05-11T13:25:29.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.601+0000 7fddd9600700 1 -- 172.21.15.63:0/4207746192 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddd000fc00 con 0x7fdddc105800 2024-05-11T13:25:29.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.601+0000 7fddd9600700 1 -- 172.21.15.63:0/4207746192 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd00037f0 con 0x7fdddc105800 2024-05-11T13:25:29.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.601+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc1a3700 con 0x7fdddc105800 2024-05-11T13:25:29.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.601+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc1a3b00 con 0x7fdddc105800 2024-05-11T13:25:29.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.602+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc06a240 con 0x7fdddc105800 2024-05-11T13:25:29.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.606+0000 7fddd9600700 1 -- 172.21.15.63:0/4207746192 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fddd002a070 con 0x7fdddc105800 2024-05-11T13:25:29.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.608+0000 7fddd9600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fddc40b2900 0x7fddc40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:25:29.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.608+0000 7fddd9600700 1 -- 172.21.15.63:0/4207746192 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7fddd00e2b00 con 0x7fdddc105800 2024-05-11T13:25:29.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.608+0000 7fdddbe00700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fddc40b2900 0x7fddc40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:25:29.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.609+0000 7fdddbe00700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fddc40b2900 0x7fddc40b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005ea0 tx=0x7fddc8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:25:29.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.611+0000 7fddd9600700 1 -- 172.21.15.63:0/4207746192 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fddd0031080 con 0x7fdddc105800 2024-05-11T13:25:29.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:29.858+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdddc063990 con 0x7fddc40b2900 2024-05-11T13:25:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:30 smithi165 ceph-mon[33235]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-11T13:25:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:30 smithi165 ceph-mon[33235]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:25:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:25:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:25:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:30.856 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:30 smithi063 ceph-mon[29809]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-11T13:25:30.856 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:30 smithi063 ceph-mon[29809]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:25:30.857 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:25:30.857 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:25:30.857 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:30 smithi071 ceph-mon[33293]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-11T13:25:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:30 smithi071 ceph-mon[33293]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:25:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:25:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:25:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:32.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:32 smithi063 ceph-mon[29809]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T13:25:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:32 smithi165 ceph-mon[33235]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T13:25:32.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:32 smithi071 ceph-mon[33293]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T13:25:34.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:34 smithi165 ceph-mon[33235]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-11T13:25:34.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:34 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2183772545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d377b63-0bda-4801-8f95-5e2e9b6bf9ee"}]: dispatch 2024-05-11T13:25:34.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:34 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2183772545' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d377b63-0bda-4801-8f95-5e2e9b6bf9ee"}]': finished 2024-05-11T13:25:34.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:34 smithi165 ceph-mon[33235]: osdmap e57: 10 total, 9 up, 10 in 2024-05-11T13:25:34.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:34.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:34 smithi063 ceph-mon[29809]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-11T13:25:34.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:34 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2183772545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d377b63-0bda-4801-8f95-5e2e9b6bf9ee"}]: dispatch 2024-05-11T13:25:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:34 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2183772545' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d377b63-0bda-4801-8f95-5e2e9b6bf9ee"}]': finished 2024-05-11T13:25:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:34 smithi063 ceph-mon[29809]: osdmap e57: 10 total, 9 up, 10 in 2024-05-11T13:25:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:34 smithi071 ceph-mon[33293]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-11T13:25:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:34 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2183772545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d377b63-0bda-4801-8f95-5e2e9b6bf9ee"}]: dispatch 2024-05-11T13:25:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:34 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2183772545' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d377b63-0bda-4801-8f95-5e2e9b6bf9ee"}]': finished 2024-05-11T13:25:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:34 smithi071 ceph-mon[33293]: osdmap e57: 10 total, 9 up, 10 in 2024-05-11T13:25:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:35.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:35 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1396644628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:25:35.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:35 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1396644628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:25:35.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:35 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1396644628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:25:36.902 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:36 smithi063 ceph-mon[29809]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:36.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:36 smithi071 ceph-mon[33293]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:37.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:36 smithi165 ceph-mon[33235]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:38.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:38 smithi063 ceph-mon[29809]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:38.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:38 smithi071 ceph-mon[33293]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:38 smithi165 ceph-mon[33235]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:25:40.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:40 smithi063 ceph-mon[29809]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:40.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:40 smithi071 ceph-mon[33293]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:41.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:40 smithi165 ceph-mon[33235]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:42.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:42 smithi063 ceph-mon[29809]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:42.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:42 smithi071 ceph-mon[33293]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:43.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:42 smithi165 ceph-mon[33235]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:43.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-11T13:25:43.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:43.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-11T13:25:43.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:44.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-11T13:25:44.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:25:44.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:44 smithi063 ceph-mon[29809]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:44.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:44 smithi063 ceph-mon[29809]: Deploying daemon osd.9 on smithi063 2024-05-11T13:25:44.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:44 smithi071 ceph-mon[33293]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:44 smithi071 ceph-mon[33293]: Deploying daemon osd.9 on smithi063 2024-05-11T13:25:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:44 smithi165 ceph-mon[33235]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:44 smithi165 ceph-mon[33235]: Deploying daemon osd.9 on smithi063 2024-05-11T13:25:46.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:46 smithi063 ceph-mon[29809]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:46 smithi071 ceph-mon[33293]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:46 smithi165 ceph-mon[33235]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:48.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:48 smithi063 ceph-mon[29809]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:48.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:48 smithi071 ceph-mon[33293]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:48 smithi165 ceph-mon[33235]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:50 smithi063 ceph-mon[29809]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:50.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:50 smithi071 ceph-mon[33293]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:50 smithi165 ceph-mon[33235]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:52.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:52 smithi063 ceph-mon[29809]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:52.903 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:52 smithi165 ceph-mon[33235]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:52.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:52 smithi071 ceph-mon[33293]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:53.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:53 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:53.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:53 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:53.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:53 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:53.918 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:53 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:53.919 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:53 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:53.919 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:53 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:53.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:25:53.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:53.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:53 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:54.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:54 smithi165 ceph-mon[33235]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:54.820 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:54 smithi063 ceph-mon[29809]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:54.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:54 smithi071 ceph-mon[33293]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:56 smithi063 ceph-mon[29809]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:56 smithi063 ceph-mon[29809]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-11T13:25:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:56 smithi071 ceph-mon[33293]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:56.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:56 smithi071 ceph-mon[33293]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-11T13:25:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:56 smithi165 ceph-mon[33235]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:56 smithi165 ceph-mon[33235]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-11T13:25:57.597 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 9 on host 'smithi063' 2024-05-11T13:25:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.595+0000 7fddd9600700 1 -- 172.21.15.63:0/4207746192 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdddc063990 con 0x7fddc40b2900 2024-05-11T13:25:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fddc40b2900 msgr2=0x7fddc40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:25:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fddc40b2900 0x7fddc40b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005ea0 tx=0x7fddc8005e30 comp rx=0 tx=0).stop 2024-05-11T13:25:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 msgr2=0x7fdddc19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:25:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc19b810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fddd000b580 tx=0x7fddd000f720 comp rx=0 tx=0).stop 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 shutdown_connections 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fddc40b2900 0x7fddc40b4dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc1073a0 0x7fdddc1a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdddc105800 0x7fdddc19b810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.599+0000 7fdde3600700 1 --2- 172.21.15.63:0/4207746192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdddc106a00 0x7fdddc19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.600+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 >> 172.21.15.63:0/4207746192 conn(0x7fdddc100da0 msgr2=0x7fdddc109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.600+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 shutdown_connections 2024-05-11T13:25:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:25:57.600+0000 7fdde3600700 1 -- 172.21.15.63:0/4207746192 wait complete. 2024-05-11T13:25:58.453 DEBUG:teuthology.orchestra.run.smithi063:osd.9> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.9.service 2024-05-11T13:25:58.458 INFO:tasks.cephadm:Deploying osd.10 on smithi071 with /dev/vg_nvme/lv_1... 2024-05-11T13:25:58.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-11T13:25:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:58 smithi063 ceph-mon[29809]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:58 smithi063 ceph-mon[29809]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-11T13:25:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:58 smithi063 ceph-mon[29809]: osdmap e58: 10 total, 9 up, 10 in 2024-05-11T13:25:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:58 smithi063 ceph-mon[29809]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:25:58.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:58.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:58.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:58 smithi165 ceph-mon[33235]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:58 smithi165 ceph-mon[33235]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-11T13:25:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:58 smithi165 ceph-mon[33235]: osdmap e58: 10 total, 9 up, 10 in 2024-05-11T13:25:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:58 smithi165 ceph-mon[33235]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:25:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:58.889 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:25:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:58 smithi071 ceph-mon[33293]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:25:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:58 smithi071 ceph-mon[33293]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-11T13:25:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:58 smithi071 ceph-mon[33293]: osdmap e58: 10 total, 9 up, 10 in 2024-05-11T13:25:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:58 smithi071 ceph-mon[33293]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-11T13:25:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:25:59.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:59 smithi063 ceph-mon[29809]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:25:59.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:59 smithi063 ceph-mon[29809]: osdmap e59: 10 total, 9 up, 10 in 2024-05-11T13:25:59.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:59.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:59 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:59.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:25:59 smithi063 ceph-mon[29809]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' 2024-05-11T13:25:59.685 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 13:25:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T13:25:59.350+0000 7fbe76c00700 -1 osd.9 0 waiting for initial osdmap 2024-05-11T13:25:59.686 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 13:25:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T13:25:59.393+0000 7fbe6ca00700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:25:59.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:59 smithi071 ceph-mon[33293]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:25:59.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:59 smithi071 ceph-mon[33293]: osdmap e59: 10 total, 9 up, 10 in 2024-05-11T13:25:59.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:59.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:59 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:25:59.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:25:59 smithi071 ceph-mon[33293]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' 2024-05-11T13:26:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:59 smithi165 ceph-mon[33235]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-11T13:26:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:59 smithi165 ceph-mon[33235]: osdmap e59: 10 total, 9 up, 10 in 2024-05-11T13:26:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:59 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:25:59 smithi165 ceph-mon[33235]: from='osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888]' entity='osd.9' 2024-05-11T13:26:00.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:00 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:26:00.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:00 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:26:00.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:00 smithi071 ceph-mon[33293]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:26:00.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:00 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:00.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:00 smithi071 ceph-mon[33293]: osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] boot 2024-05-11T13:26:00.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:00 smithi071 ceph-mon[33293]: osdmap e60: 10 total, 10 up, 10 in 2024-05-11T13:26:00.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:00 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:00 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:26:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:00 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:26:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:00 smithi063 ceph-mon[29809]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:26:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:00 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:00 smithi063 ceph-mon[29809]: osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] boot 2024-05-11T13:26:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:00 smithi063 ceph-mon[29809]: osdmap e60: 10 total, 10 up, 10 in 2024-05-11T13:26:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:00 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:00 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:26:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:00 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:26:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:00 smithi165 ceph-mon[33235]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-11T13:26:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:00 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:00 smithi165 ceph-mon[33235]: osd.9 [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] boot 2024-05-11T13:26:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:00 smithi165 ceph-mon[33235]: osdmap e60: 10 total, 10 up, 10 in 2024-05-11T13:26:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:00 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:26:01.999 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-11T13:26:01.999 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-11T13:26:01.999 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-11T13:26:01.999 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-11T13:26:01.999 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0602891 s, 174 MB/s 2024-05-11T13:26:01.999 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-11T13:26:02.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:02 smithi071 ceph-mon[33293]: osdmap e61: 10 total, 10 up, 10 in 2024-05-11T13:26:02.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:02 smithi071 ceph-mon[33293]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:02.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:02 smithi063 ceph-mon[29809]: osdmap e61: 10 total, 10 up, 10 in 2024-05-11T13:26:02.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:02 smithi063 ceph-mon[29809]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:02.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:02 smithi165 ceph-mon[33235]: osdmap e61: 10 total, 10 up, 10 in 2024-05-11T13:26:02.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:02 smithi165 ceph-mon[33235]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:02.978 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-05-11T13:26:03.391 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:26:04.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi063 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi063 to 4272M 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:04.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:04.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:04.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:04 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi063 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi063 to 4272M 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:04.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:04.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:04.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:04.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:04.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi063 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi063 to 4272M 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:04.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:04 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:05.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.836+0000 7f970c400700 1 -- 172.21.15.71:0/540319661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041040b0 msgr2=0x7f97041064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:05.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.836+0000 7f970c400700 1 --2- 172.21.15.71:0/540319661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041040b0 0x7f97041064a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009990 tx=0x7f96f8009ca0 comp rx=0 tx=0).stop 2024-05-11T13:26:05.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.837+0000 7f970c400700 1 -- 172.21.15.71:0/540319661 shutdown_connections 2024-05-11T13:26:05.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.837+0000 7f970c400700 1 --2- 172.21.15.71:0/540319661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9704109310 0x7f970410b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:05.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.837+0000 7f970c400700 1 --2- 172.21.15.71:0/540319661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041040b0 0x7f97041064a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:05.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.837+0000 7f970c400700 1 --2- 172.21.15.71:0/540319661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97041069e0 0x7f9704108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:05.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.837+0000 7f970c400700 1 -- 172.21.15.71:0/540319661 >> 172.21.15.71:0/540319661 conn(0x7f97040fda40 msgr2=0x7f97040ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:26:05.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.837+0000 7f970c400700 1 -- 172.21.15.71:0/540319661 shutdown_connections 2024-05-11T13:26:05.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.837+0000 7f970c400700 1 -- 172.21.15.71:0/540319661 wait complete. 2024-05-11T13:26:05.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.838+0000 7f970c400700 1 Processor -- start 2024-05-11T13:26:05.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.838+0000 7f970c400700 1 -- start start 2024-05-11T13:26:05.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.838+0000 7f970c400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97041040b0 0x7f9704103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:05.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f970c400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041069e0 0x7f9704102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:05.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f970c400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9704109310 0x7f97041025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f970c400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f970410eed0 con 0x7f97041040b0 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f970c400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f970410ed50 con 0x7f97041069e0 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f970c400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f970410f050 con 0x7f9704109310 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041069e0 0x7f9704102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041069e0 0x7f9704102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36162/0 (socket says 172.21.15.71:36162) 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 -- 172.21.15.71:0/57993847 learned_addr learned my addr 172.21.15.71:0/57993847 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709a00700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97041040b0 0x7f9704103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f970a400700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9704109310 0x7f97041025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 -- 172.21.15.71:0/57993847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9704109310 msgr2=0x7f97041025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9704109310 0x7f97041025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 -- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97041040b0 msgr2=0x7f9704103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97041040b0 0x7f9704103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.839+0000 7f9709000700 1 -- 172.21.15.71:0/57993847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9704102e50 con 0x7f97041069e0 2024-05-11T13:26:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.840+0000 7f9709000700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041069e0 0x7f9704102060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f96f000d950 tx=0x7f96f000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:26:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.840+0000 7f9702000700 1 -- 172.21.15.71:0/57993847 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0016070 con 0x7f97041069e0 2024-05-11T13:26:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.840+0000 7f9702000700 1 -- 172.21.15.71:0/57993847 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f0010460 con 0x7f97041069e0 2024-05-11T13:26:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.840+0000 7f9702000700 1 -- 172.21.15.71:0/57993847 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000f620 con 0x7f97041069e0 2024-05-11T13:26:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.840+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97041a3700 con 0x7f97041069e0 2024-05-11T13:26:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.840+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97041a3bc0 con 0x7f97041069e0 2024-05-11T13:26:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.841+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970406a240 con 0x7f97041069e0 2024-05-11T13:26:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.844+0000 7f9702000700 1 -- 172.21.15.71:0/57993847 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f96f0019070 con 0x7f97041069e0 2024-05-11T13:26:05.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.845+0000 7f9702000700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ec0b2900 0x7f96ec0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:05.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.845+0000 7f9702000700 1 -- 172.21.15.71:0/57993847 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f96f00e1670 con 0x7f97041069e0 2024-05-11T13:26:05.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.846+0000 7f9709a00700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ec0b2900 0x7f96ec0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:05.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.847+0000 7f9709a00700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ec0b2900 0x7f96ec0b4dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96f800b5c0 tx=0x7f96f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:26:05.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:05.848+0000 7f9702000700 1 -- 172.21.15.71:0/57993847 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96f00a9af0 con 0x7f97041069e0 2024-05-11T13:26:06.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:06.089+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9704063990 con 0x7f96ec0b2900 2024-05-11T13:26:06.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:06 smithi165 ceph-mon[33235]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:06.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:26:06.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:26:06.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:06 smithi063 ceph-mon[29809]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:26:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:26:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:06.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:06 smithi071 ceph-mon[33293]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:06.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:26:06.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:26:06.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:07.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:07 smithi071 ceph-mon[33293]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:26:07.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:07 smithi165 ceph-mon[33235]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:26:07.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:07 smithi063 ceph-mon[29809]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:26:08.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:08 smithi071 ceph-mon[33293]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:08.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:08 smithi165 ceph-mon[33235]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:08 smithi063 ceph-mon[29809]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:10.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:10 smithi063 ceph-mon[29809]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:10.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:10 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/1880268692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]: dispatch 2024-05-11T13:26:10.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:10 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]: dispatch 2024-05-11T13:26:10.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:10 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]': finished 2024-05-11T13:26:10.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:10 smithi063 ceph-mon[29809]: osdmap e62: 11 total, 10 up, 11 in 2024-05-11T13:26:10.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:10.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:10 smithi071 ceph-mon[33293]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:10.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:10 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/1880268692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]: dispatch 2024-05-11T13:26:10.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:10 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]: dispatch 2024-05-11T13:26:10.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:10 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]': finished 2024-05-11T13:26:10.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:10 smithi071 ceph-mon[33293]: osdmap e62: 11 total, 10 up, 11 in 2024-05-11T13:26:10.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:11.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:10 smithi165 ceph-mon[33235]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:11.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:10 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/1880268692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]: dispatch 2024-05-11T13:26:11.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:10 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]: dispatch 2024-05-11T13:26:11.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:10 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb80b66c-5c56-420b-8b5d-a0128ec05742"}]': finished 2024-05-11T13:26:11.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:10 smithi165 ceph-mon[33235]: osdmap e62: 11 total, 10 up, 11 in 2024-05-11T13:26:11.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:11.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:11 smithi063 ceph-mon[29809]: from='client.? 172.21.15.71:0/4124889848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:26:11.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:11 smithi071 ceph-mon[33293]: from='client.? 172.21.15.71:0/4124889848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:26:12.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:11 smithi165 ceph-mon[33235]: from='client.? 172.21.15.71:0/4124889848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:26:12.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:12 smithi063 ceph-mon[29809]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:12 smithi071 ceph-mon[33293]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:13.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:12 smithi165 ceph-mon[33235]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:14 smithi063 ceph-mon[29809]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:14.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:14 smithi071 ceph-mon[33293]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:15.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:14 smithi165 ceph-mon[33235]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:16.925 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:16 smithi071 ceph-mon[33293]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:16.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:16 smithi063 ceph-mon[29809]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:17.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:16 smithi165 ceph-mon[33235]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:18.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:18 smithi063 ceph-mon[29809]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:18.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:18 smithi071 ceph-mon[33293]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:18 smithi165 ceph-mon[33235]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:20 smithi063 ceph-mon[29809]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-11T13:26:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:20.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:20 smithi071 ceph-mon[33293]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:20.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-11T13:26:20.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:21.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:20 smithi165 ceph-mon[33235]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:21.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-11T13:26:21.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:21.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:21 smithi071 ceph-mon[33293]: Deploying daemon osd.10 on smithi071 2024-05-11T13:26:21.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:21 smithi063 ceph-mon[29809]: Deploying daemon osd.10 on smithi071 2024-05-11T13:26:22.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:21 smithi165 ceph-mon[33235]: Deploying daemon osd.10 on smithi071 2024-05-11T13:26:22.892 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:22 smithi071 ceph-mon[33293]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:22 smithi063 ceph-mon[29809]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:22.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:22 smithi165 ceph-mon[33235]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:24.928 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:24 smithi071 ceph-mon[33293]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:24.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:24 smithi063 ceph-mon[29809]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:25.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:24 smithi165 ceph-mon[33235]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:26.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:26 smithi063 ceph-mon[29809]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:26.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:26 smithi071 ceph-mon[33293]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:27.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:26 smithi165 ceph-mon[33235]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:28.797 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:28 smithi071 ceph-mon[33293]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:28 smithi063 ceph-mon[29809]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:29.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:28 smithi165 ceph-mon[33235]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:30.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:30.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:30.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:30.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:30 smithi071 ceph-mon[33293]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:30.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:30.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:30.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:30.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:30 smithi063 ceph-mon[29809]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:30.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:30.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:30.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:30.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:30 smithi165 ceph-mon[33235]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:32.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:32 smithi071 ceph-mon[33293]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:32 smithi165 ceph-mon[33235]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:32.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:32 smithi063 ceph-mon[29809]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:33.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:33 smithi165 ceph-mon[33235]: from='osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T13:26:33.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:33 smithi165 ceph-mon[33235]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T13:26:33.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:33.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:33.917 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:33 smithi071 ceph-mon[33293]: from='osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T13:26:33.917 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:33 smithi071 ceph-mon[33293]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T13:26:33.918 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:33.918 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:33.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:33 smithi063 ceph-mon[29809]: from='osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T13:26:33.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:33 smithi063 ceph-mon[29809]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T13:26:33.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:33.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.377+0000 7f9702000700 1 -- 172.21.15.71:0/57993847 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9704063990 con 0x7f96ec0b2900 2024-05-11T13:26:34.378 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 10 on host 'smithi071' 2024-05-11T13:26:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ec0b2900 msgr2=0x7f96ec0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ec0b2900 0x7f96ec0b4dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96f800b5c0 tx=0x7f96f8005fb0 comp rx=0 tx=0).stop 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041069e0 msgr2=0x7f9704102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041069e0 0x7f9704102060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f96f000d950 tx=0x7f96f000dd10 comp rx=0 tx=0).stop 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 shutdown_connections 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ec0b2900 0x7f96ec0b4dc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9704109310 0x7f97041025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97041040b0 0x7f9704103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 --2- 172.21.15.71:0/57993847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97041069e0 0x7f9704102060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 >> 172.21.15.71:0/57993847 conn(0x7f97040fda40 msgr2=0x7f97041049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.381+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 shutdown_connections 2024-05-11T13:26:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-11T13:26:34.382+0000 7f970c400700 1 -- 172.21.15.71:0/57993847 wait complete. 2024-05-11T13:26:34.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-11T13:26:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: from='osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:26:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: osdmap e63: 11 total, 10 up, 11 in 2024-05-11T13:26:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:26:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:34.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: from='osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: osdmap e63: 11 total, 10 up, 11 in 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:34.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:35.006 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:35.006 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-11T13:26:35.006 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: from='osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:26:35.006 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: osdmap e63: 11 total, 10 up, 11 in 2024-05-11T13:26:35.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:35.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-11T13:26:35.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:35.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:35.229 DEBUG:teuthology.orchestra.run.smithi071:osd.10> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.10.service 2024-05-11T13:26:35.234 INFO:tasks.cephadm:Deploying osd.11 on smithi165 with /dev/vg_nvme/lv_1... 2024-05-11T13:26:35.234 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-11T13:26:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:26:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:35 smithi071 ceph-mon[33293]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:26:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:35 smithi071 ceph-mon[33293]: osdmap e64: 11 total, 10 up, 11 in 2024-05-11T13:26:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:35 smithi071 ceph-mon[33293]: from='osd.10 ' entity='osd.10' 2024-05-11T13:26:35.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:35 smithi071 ceph-mon[33293]: osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] boot 2024-05-11T13:26:35.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:35 smithi071 ceph-mon[33293]: osdmap e65: 11 total, 11 up, 11 in 2024-05-11T13:26:35.858 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 13:26:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T13:26:35.507+0000 7f76e5600700 -1 osd.10 0 waiting for initial osdmap 2024-05-11T13:26:35.858 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 13:26:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T13:26:35.545+0000 7f76db400700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:26:35.860 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:35 smithi165 ceph-mon[33235]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:26:35.860 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:35 smithi165 ceph-mon[33235]: osdmap e64: 11 total, 10 up, 11 in 2024-05-11T13:26:35.860 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:35.860 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:35.860 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:35 smithi165 ceph-mon[33235]: from='osd.10 ' entity='osd.10' 2024-05-11T13:26:35.860 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:35 smithi165 ceph-mon[33235]: osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] boot 2024-05-11T13:26:35.860 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:35 smithi165 ceph-mon[33235]: osdmap e65: 11 total, 11 up, 11 in 2024-05-11T13:26:35.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:35 smithi063 ceph-mon[29809]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-11T13:26:35.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:35 smithi063 ceph-mon[29809]: osdmap e64: 11 total, 10 up, 11 in 2024-05-11T13:26:35.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:35.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:35.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:35 smithi063 ceph-mon[29809]: from='osd.10 ' entity='osd.10' 2024-05-11T13:26:35.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:35 smithi063 ceph-mon[29809]: osd.10 [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] boot 2024-05-11T13:26:35.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:35 smithi063 ceph-mon[29809]: osdmap e65: 11 total, 11 up, 11 in 2024-05-11T13:26:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:36 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:26:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:36 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:26:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:36 smithi071 ceph-mon[33293]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:36 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:36.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:36 smithi071 ceph-mon[33293]: osdmap e66: 11 total, 11 up, 11 in 2024-05-11T13:26:36.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:36 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:26:36.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:36 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:26:36.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:36 smithi165 ceph-mon[33235]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:36.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:36 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:36.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:36 smithi165 ceph-mon[33235]: osdmap e66: 11 total, 11 up, 11 in 2024-05-11T13:26:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:36 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:26:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:36 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:26:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:36 smithi063 ceph-mon[29809]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-11T13:26:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:36 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:26:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:36 smithi063 ceph-mon[29809]: osdmap e66: 11 total, 11 up, 11 in 2024-05-11T13:26:38.654 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-11T13:26:38.654 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-11T13:26:38.655 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-11T13:26:38.655 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-11T13:26:38.655 INFO:teuthology.orchestra.run.smithi165.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.033301 s, 315 MB/s 2024-05-11T13:26:38.655 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-11T13:26:38.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:38 smithi063 ceph-mon[29809]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 531 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:38.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:38 smithi063 ceph-mon[29809]: osdmap e67: 11 total, 11 up, 11 in 2024-05-11T13:26:38.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:38 smithi071 ceph-mon[33293]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 531 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:38.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:38 smithi071 ceph-mon[33293]: osdmap e67: 11 total, 11 up, 11 in 2024-05-11T13:26:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:38 smithi165 ceph-mon[33235]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 531 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:38 smithi165 ceph-mon[33235]: osdmap e67: 11 total, 11 up, 11 in 2024-05-11T13:26:39.609 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-05-11T13:26:40.044 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:26:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 531 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-11T13:26:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 531 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-11T13:26:40.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:40.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 531 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-11T13:26:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:41.787 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:41 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi071 2024-05-11T13:26:41.787 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:41 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi071 to 4272M 2024-05-11T13:26:41.787 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:41.788 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:41.788 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:41.788 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:41.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:41 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi071 2024-05-11T13:26:41.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:41 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi071 to 4272M 2024-05-11T13:26:41.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:41.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:41.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:41.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:41 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi071 2024-05-11T13:26:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:41 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi071 to 4272M 2024-05-11T13:26:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:26:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:26:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.304+0000 7f96caa00700 1 -- 172.21.15.165:0/3008864218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c41068d0 msgr2=0x7f96c4106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.304+0000 7f96caa00700 1 --2- 172.21.15.165:0/3008864218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c41068d0 0x7f96c4106d30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009990 tx=0x7f96b0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.304+0000 7f96caa00700 1 -- 172.21.15.165:0/3008864218 shutdown_connections 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.304+0000 7f96caa00700 1 --2- 172.21.15.165:0/3008864218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96c4107270 0x7f96c410db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.304+0000 7f96caa00700 1 --2- 172.21.15.165:0/3008864218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96c41056d0 0x7f96c4105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.304+0000 7f96caa00700 1 --2- 172.21.15.165:0/3008864218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c41068d0 0x7f96c4106d30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.304+0000 7f96caa00700 1 -- 172.21.15.165:0/3008864218 >> 172.21.15.165:0/3008864218 conn(0x7f96c4100c70 msgr2=0x7f96c41030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.305+0000 7f96caa00700 1 -- 172.21.15.165:0/3008864218 shutdown_connections 2024-05-11T13:26:42.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.305+0000 7f96caa00700 1 -- 172.21.15.165:0/3008864218 wait complete. 2024-05-11T13:26:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.306+0000 7f96caa00700 1 Processor -- start 2024-05-11T13:26:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.306+0000 7f96caa00700 1 -- start start 2024-05-11T13:26:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96caa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96c41056d0 0x7f96c419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96c41068d0 0x7f96c419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96caa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c4107270 0x7f96c41a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96caa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4110ff0 con 0x7f96c41068d0 2024-05-11T13:26:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96caa00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4110e70 con 0x7f96c41056d0 2024-05-11T13:26:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96caa00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4111170 con 0x7f96c4107270 2024-05-11T13:26:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96ca000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c4107270 0x7f96c41a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96ca000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c4107270 0x7f96c41a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35882/0 (socket says 172.21.15.165:35882) 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96ca000700 1 -- 172.21.15.165:0/3374422985 learned_addr learned my addr 172.21.15.165:0/3374422985 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96c8c00700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96c41068d0 0x7f96c419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96c9600700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96c41056d0 0x7f96c419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96ca000700 1 -- 172.21.15.165:0/3374422985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96c41056d0 msgr2=0x7f96c419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.307+0000 7f96ca000700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96c41056d0 0x7f96c419b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96ca000700 1 -- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96c41068d0 msgr2=0x7f96c419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96ca000700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96c41068d0 0x7f96c419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:26:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96ca000700 1 -- 172.21.15.165:0/3374422985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c41a3330 con 0x7f96c4107270 2024-05-11T13:26:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96ca000700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c4107270 0x7f96c41a2c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96bc007f10 tx=0x7f96bc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:26:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96c2000700 1 -- 172.21.15.165:0/3374422985 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc004bb0 con 0x7f96c4107270 2024-05-11T13:26:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96c2000700 1 -- 172.21.15.165:0/3374422985 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96bc004d50 con 0x7f96c4107270 2024-05-11T13:26:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c41a3620 con 0x7f96c4107270 2024-05-11T13:26:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96c2000700 1 -- 172.21.15.165:0/3374422985 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc005970 con 0x7f96c4107270 2024-05-11T13:26:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.308+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c41a3b60 con 0x7f96c4107270 2024-05-11T13:26:42.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.310+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c406a240 con 0x7f96c4107270 2024-05-11T13:26:42.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.314+0000 7f96c2000700 1 -- 172.21.15.165:0/3374422985 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f96bc01a020 con 0x7f96c4107270 2024-05-11T13:26:42.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.315+0000 7f96c2000700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ac0b2900 0x7f96ac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:26:42.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.316+0000 7f96c2000700 1 -- 172.21.15.165:0/3374422985 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f96bc0e1a20 con 0x7f96c4107270 2024-05-11T13:26:42.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.316+0000 7f96c9600700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ac0b2900 0x7f96ac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:26:42.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.317+0000 7f96c9600700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ac0b2900 0x7f96ac0b4dc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005950 tx=0x7f96b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:26:42.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.319+0000 7f96c2000700 1 -- 172.21.15.165:0/3374422985 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96bc010070 con 0x7f96c4107270 2024-05-11T13:26:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:26:42.564+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f96c4063990 con 0x7f96ac0b2900 2024-05-11T13:26:43.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:42 smithi063 ceph-mon[29809]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:26:43.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:26:43.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:26:43.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:42 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:43.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:42 smithi071 ceph-mon[33293]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:26:43.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:42 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:26:43.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:42 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:26:43.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:42 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:43.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:42 smithi165 ceph-mon[33235]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T13:26:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T13:26:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T13:26:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:42 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:44.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:43 smithi063 ceph-mon[29809]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:26:44.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:43 smithi071 ceph-mon[33293]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:26:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:43 smithi165 ceph-mon[33235]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:26:45.141 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:44 smithi165 ceph-mon[33235]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-11T13:26:45.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:44 smithi063 ceph-mon[29809]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-11T13:26:45.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:44 smithi071 ceph-mon[33293]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-11T13:26:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:46 smithi063 ceph-mon[29809]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-11T13:26:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:46 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/1782513566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]: dispatch 2024-05-11T13:26:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:46 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]: dispatch 2024-05-11T13:26:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:46 smithi063 ceph-mon[29809]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]': finished 2024-05-11T13:26:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:46 smithi063 ceph-mon[29809]: osdmap e68: 12 total, 11 up, 12 in 2024-05-11T13:26:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:26:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:46 smithi071 ceph-mon[33293]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-11T13:26:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:46 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/1782513566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]: dispatch 2024-05-11T13:26:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:46 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]: dispatch 2024-05-11T13:26:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:46 smithi071 ceph-mon[33293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]': finished 2024-05-11T13:26:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:46 smithi071 ceph-mon[33293]: osdmap e68: 12 total, 11 up, 12 in 2024-05-11T13:26:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:26:47.235 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:46 smithi165 ceph-mon[33235]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-11T13:26:47.235 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:46 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/1782513566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]: dispatch 2024-05-11T13:26:47.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:46 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]: dispatch 2024-05-11T13:26:47.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:46 smithi165 ceph-mon[33235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "222725ae-5dca-47ba-9146-2c4ef02a3272"}]': finished 2024-05-11T13:26:47.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:46 smithi165 ceph-mon[33235]: osdmap e68: 12 total, 11 up, 12 in 2024-05-11T13:26:47.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:26:48.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:47 smithi071 ceph-mon[33293]: from='client.? 172.21.15.165:0/497136128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:26:48.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:47 smithi165 ceph-mon[33235]: from='client.? 172.21.15.165:0/497136128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:26:48.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:47 smithi063 ceph-mon[29809]: from='client.? 172.21.15.165:0/497136128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T13:26:49.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:48 smithi071 ceph-mon[33293]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:26:49.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:48 smithi165 ceph-mon[33235]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:26:49.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:48 smithi063 ceph-mon[29809]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T13:26:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:50 smithi071 ceph-mon[33293]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:51.289 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:50 smithi165 ceph-mon[33235]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:51.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:50 smithi063 ceph-mon[29809]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:52 smithi165 ceph-mon[33235]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:53.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:52 smithi063 ceph-mon[29809]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:53.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:52 smithi071 ceph-mon[33293]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:55.227 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:54 smithi165 ceph-mon[33235]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:55.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:54 smithi063 ceph-mon[29809]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:55.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:54 smithi071 ceph-mon[33293]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:57.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:56 smithi165 ceph-mon[33235]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:57.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-11T13:26:57.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:56 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:57.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:56 smithi063 ceph-mon[29809]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:57.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-11T13:26:57.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:56 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:57.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:56 smithi071 ceph-mon[33293]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:57.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-11T13:26:57.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:56 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:26:58.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:57 smithi165 ceph-mon[33235]: Deploying daemon osd.11 on smithi165 2024-05-11T13:26:58.355 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:57 smithi071 ceph-mon[33293]: Deploying daemon osd.11 on smithi165 2024-05-11T13:26:58.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:57 smithi063 ceph-mon[29809]: Deploying daemon osd.11 on smithi165 2024-05-11T13:26:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:26:58 smithi165 ceph-mon[33235]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:59.382 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:26:58 smithi071 ceph-mon[33293]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:26:59.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:26:58 smithi063 ceph-mon[29809]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:01.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:00 smithi071 ceph-mon[33293]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:01.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:00 smithi165 ceph-mon[33235]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:01.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:00 smithi063 ceph-mon[29809]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:03.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:02 smithi165 ceph-mon[33235]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:03.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:02 smithi063 ceph-mon[29809]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:03.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:02 smithi071 ceph-mon[33293]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:05 smithi165 ceph-mon[33235]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:05 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:05 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:05 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:04 smithi063 ceph-mon[29809]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:04 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:05 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:05.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:05 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:05.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:05 smithi071 ceph-mon[33293]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:05.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:05 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:05.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:05 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:05.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:05 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:07.185 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:07 smithi165 ceph-mon[33235]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:07.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:07 smithi063 ceph-mon[29809]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:07.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:07 smithi071 ceph-mon[33293]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:09.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:09.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: from='osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T13:27:09.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:09 smithi165 ceph-mon[33235]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T13:27:09.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:09.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:09.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:09.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: from='osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T13:27:09.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:09 smithi063 ceph-mon[29809]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T13:27:09.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:09.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:09.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:09.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:09.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: from='osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T13:27:09.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:09 smithi071 ceph-mon[33293]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T13:27:09.541 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 11 on host 'smithi165' 2024-05-11T13:27:09.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.540+0000 7f96c2000700 1 -- 172.21.15.165:0/3374422985 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f96c4063990 con 0x7f96ac0b2900 2024-05-11T13:27:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ac0b2900 msgr2=0x7f96ac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ac0b2900 0x7f96ac0b4dc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005950 tx=0x7f96b80058e0 comp rx=0 tx=0).stop 2024-05-11T13:27:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c4107270 msgr2=0x7f96c41a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c4107270 0x7f96c41a2c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96bc007f10 tx=0x7f96bc00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:27:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 shutdown_connections 2024-05-11T13:27:09.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f96ac0b2900 0x7f96ac0b4dc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:09.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96c4107270 0x7f96c41a2c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:09.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96c41056d0 0x7f96c419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:09.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 --2- 172.21.15.165:0/3374422985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96c41068d0 0x7f96c419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:09.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.543+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 >> 172.21.15.165:0/3374422985 conn(0x7f96c4100c70 msgr2=0x7f96c4109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:09.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.544+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 shutdown_connections 2024-05-11T13:27:09.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:09.544+0000 7f96caa00700 1 -- 172.21.15.165:0/3374422985 wait complete. 2024-05-11T13:27:10.335 DEBUG:teuthology.orchestra.run.smithi165:osd.11> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.11.service 2024-05-11T13:27:10.340 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-11T13:27:10.340 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-11T13:27:10.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:10.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:10.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:10.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-11T13:27:10.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: osdmap e69: 12 total, 11 up, 12 in 2024-05-11T13:27:10.551 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:10.551 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:10.551 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: from='osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:27:10.551 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:10 smithi165 ceph-mon[33235]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:27:10.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:10.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:10.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:10.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-11T13:27:10.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: osdmap e69: 12 total, 11 up, 12 in 2024-05-11T13:27:10.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:10.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:10.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: from='osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:27:10.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:10 smithi063 ceph-mon[29809]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: osdmap e69: 12 total, 11 up, 12 in 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: from='osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:27:10.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:10 smithi071 ceph-mon[33293]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-11T13:27:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:11.723 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:11 smithi165 ceph-mon[33235]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:27:11.723 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:11 smithi165 ceph-mon[33235]: osdmap e70: 12 total, 11 up, 12 in 2024-05-11T13:27:11.723 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:11 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:11.723 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:11 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:11.723 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:11 smithi165 ceph-mon[33235]: from='osd.11 ' entity='osd.11' 2024-05-11T13:27:11.724 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 13:27:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T13:27:11.433+0000 7fafcc600700 -1 osd.11 0 waiting for initial osdmap 2024-05-11T13:27:11.724 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 13:27:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T13:27:11.467+0000 7fafc2400700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T13:27:11.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:11 smithi063 ceph-mon[29809]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:27:11.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:11 smithi063 ceph-mon[29809]: osdmap e70: 12 total, 11 up, 12 in 2024-05-11T13:27:11.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:11 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:11.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:11 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:11.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:11 smithi063 ceph-mon[29809]: from='osd.11 ' entity='osd.11' 2024-05-11T13:27:11.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:11 smithi071 ceph-mon[33293]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-11T13:27:11.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:11 smithi071 ceph-mon[33293]: osdmap e70: 12 total, 11 up, 12 in 2024-05-11T13:27:11.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:11 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:11.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:11 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:11.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:11 smithi071 ceph-mon[33293]: from='osd.11 ' entity='osd.11' 2024-05-11T13:27:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:12 smithi063 ceph-mon[29809]: purged_snaps scrub starts 2024-05-11T13:27:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:12 smithi063 ceph-mon[29809]: purged_snaps scrub ok 2024-05-11T13:27:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:12 smithi063 ceph-mon[29809]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:12 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:12 smithi063 ceph-mon[29809]: osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] boot 2024-05-11T13:27:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:12 smithi063 ceph-mon[29809]: osdmap e71: 12 total, 12 up, 12 in 2024-05-11T13:27:12.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:12 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:12 smithi165 ceph-mon[33235]: purged_snaps scrub starts 2024-05-11T13:27:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:12 smithi165 ceph-mon[33235]: purged_snaps scrub ok 2024-05-11T13:27:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:12 smithi165 ceph-mon[33235]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:12 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:12 smithi165 ceph-mon[33235]: osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] boot 2024-05-11T13:27:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:12 smithi165 ceph-mon[33235]: osdmap e71: 12 total, 12 up, 12 in 2024-05-11T13:27:12.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:12 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:12 smithi071 ceph-mon[33293]: purged_snaps scrub starts 2024-05-11T13:27:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:12 smithi071 ceph-mon[33293]: purged_snaps scrub ok 2024-05-11T13:27:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:12 smithi071 ceph-mon[33293]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-11T13:27:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:12 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:12 smithi071 ceph-mon[33293]: osd.11 [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] boot 2024-05-11T13:27:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:12 smithi071 ceph-mon[33293]: osdmap e71: 12 total, 12 up, 12 in 2024-05-11T13:27:12.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:12 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.382+0000 7f90d3200700 1 -- 172.21.15.63:0/393510159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 msgr2=0x7f90cc10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.382+0000 7f90d3200700 1 --2- 172.21.15.63:0/393510159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc10d920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009990 tx=0x7f90c4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.382+0000 7f90d3200700 1 -- 172.21.15.63:0/393510159 shutdown_connections 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.382+0000 7f90d3200700 1 --2- 172.21.15.63:0/393510159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc10d920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.382+0000 7f90d3200700 1 --2- 172.21.15.63:0/393510159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90cc077580 0x7f90cc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.382+0000 7f90d3200700 1 --2- 172.21.15.63:0/393510159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90cc075fb0 0x7f90cc076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.382+0000 7f90d3200700 1 -- 172.21.15.63:0/393510159 >> 172.21.15.63:0/393510159 conn(0x7f90cc0feb20 msgr2=0x7f90cc100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.383+0000 7f90d3200700 1 -- 172.21.15.63:0/393510159 shutdown_connections 2024-05-11T13:27:13.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.383+0000 7f90d3200700 1 -- 172.21.15.63:0/393510159 wait complete. 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 Processor -- start 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 -- start start 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90cc075fb0 0x7f90cc19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90cc077580 0x7f90cc19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc1a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc111170 con 0x7f90cc10b490 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc110ff0 con 0x7f90cc075fb0 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.384+0000 7f90d3200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1112f0 con 0x7f90cc077580 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc1a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:13.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc1a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52666/0 (socket says 172.21.15.63:52666) 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 -- 172.21.15.63:0/2818668719 learned_addr learned my addr 172.21.15.63:0/2818668719 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90cbe00700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90cc075fb0 0x7f90cc19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90c3400700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90cc077580 0x7f90cc19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 -- 172.21.15.63:0/2818668719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90cc077580 msgr2=0x7f90cc19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90cc077580 0x7f90cc19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 -- 172.21.15.63:0/2818668719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90cc075fb0 msgr2=0x7f90cc19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90cc075fb0 0x7f90cc19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90d1200700 1 -- 172.21.15.63:0/2818668719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc1a3460 con 0x7f90cc10b490 2024-05-11T13:27:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.385+0000 7f90c3400700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90cc077580 0x7f90cc19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:27:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.386+0000 7f90d1200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc1a2dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90c40047c0 tx=0x7f90c4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.386+0000 7f90c9600700 1 -- 172.21.15.63:0/2818668719 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4004830 con 0x7f90cc10b490 2024-05-11T13:27:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.386+0000 7f90c9600700 1 -- 172.21.15.63:0/2818668719 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90c4003b90 con 0x7f90cc10b490 2024-05-11T13:27:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.386+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc1a36f0 con 0x7f90cc10b490 2024-05-11T13:27:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.386+0000 7f90c9600700 1 -- 172.21.15.63:0/2818668719 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40053a0 con 0x7f90cc10b490 2024-05-11T13:27:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.386+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc1a3ae0 con 0x7f90cc10b490 2024-05-11T13:27:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.387+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc06a240 con 0x7f90cc10b490 2024-05-11T13:27:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.390+0000 7f90c9600700 1 -- 172.21.15.63:0/2818668719 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f90c4005540 con 0x7f90cc10b490 2024-05-11T13:27:13.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.391+0000 7f90c9600700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f90b00b2880 0x7f90b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.392+0000 7f90c9600700 1 -- 172.21.15.63:0/2818668719 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f90c40e31b0 con 0x7f90cc10b490 2024-05-11T13:27:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.392+0000 7f90cbe00700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f90b00b2880 0x7f90b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.392+0000 7f90cbe00700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f90b00b2880 0x7f90b00b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009550 tx=0x7f90bc0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.395+0000 7f90c9600700 1 -- 172.21.15.63:0/2818668719 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f90c4013070 con 0x7f90cc10b490 2024-05-11T13:27:13.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.636+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f90cc02e7a0 con 0x7f90cc10b490 2024-05-11T13:27:13.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.637+0000 7f90c9600700 1 -- 172.21.15.63:0/2818668719 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f90c40ab360 con 0x7f90cc10b490 2024-05-11T13:27:13.637 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:27:13.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f90b00b2880 msgr2=0x7f90b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:13.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f90b00b2880 0x7f90b00b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009550 tx=0x7f90bc0091b0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 msgr2=0x7f90cc1a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc1a2dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90c40047c0 tx=0x7f90c4009580 comp rx=0 tx=0).stop 2024-05-11T13:27:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 shutdown_connections 2024-05-11T13:27:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f90b00b2880 0x7f90b00b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90cc10b490 0x7f90cc1a2dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.640+0000 7f90d3200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90cc075fb0 0x7f90cc19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.641+0000 7f90d3200700 1 --2- 172.21.15.63:0/2818668719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90cc077580 0x7f90cc19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.641+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 >> 172.21.15.63:0/2818668719 conn(0x7f90cc0feb20 msgr2=0x7f90cc107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:13.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.641+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 shutdown_connections 2024-05-11T13:27:13.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:13.641+0000 7f90d3200700 1 -- 172.21.15.63:0/2818668719 wait complete. 2024-05-11T13:27:14.589 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715434032,"num_in_osds":12,"osd_in_since":1715434006,"num_remapped_pgs":0} 2024-05-11T13:27:14.591 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:27:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:14 smithi165 ceph-mon[33235]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:14 smithi165 ceph-mon[33235]: osdmap e72: 12 total, 12 up, 12 in 2024-05-11T13:27:14.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:14 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2818668719' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T13:27:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:14 smithi063 ceph-mon[29809]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:14 smithi063 ceph-mon[29809]: osdmap e72: 12 total, 12 up, 12 in 2024-05-11T13:27:14.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:14 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2818668719' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T13:27:14.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:14 smithi071 ceph-mon[33293]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:14.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:14 smithi071 ceph-mon[33293]: osdmap e72: 12 total, 12 up, 12 in 2024-05-11T13:27:14.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:14 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2818668719' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T13:27:15.064 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: Detected new or changed devices on smithi165 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi165 to 4272M 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.246 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: Detected new or changed devices on smithi165 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi165 to 4272M 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:16.247 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: Detected new or changed devices on smithi165 2024-05-11T13:27:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi165 to 4272M 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:17.292 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:16 smithi063 ceph-mon[29809]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:17.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:16 smithi165 ceph-mon[33235]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:17.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:16 smithi071 ceph-mon[33293]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:17.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 -- 172.21.15.63:0/3061874326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada4114750 msgr2=0x7fada4116b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 --2- 172.21.15.63:0/3061874326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada4114750 0x7fada4116b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009990 tx=0x7fad9c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 -- 172.21.15.63:0/3061874326 shutdown_connections 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 --2- 172.21.15.63:0/3061874326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada4114750 0x7fada4116b40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 --2- 172.21.15.63:0/3061874326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada4075780 0x7fada4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 --2- 172.21.15.63:0/3061874326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada40760a0 0x7fada4114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 -- 172.21.15.63:0/3061874326 >> 172.21.15.63:0/3061874326 conn(0x7fada407a700 msgr2=0x7fada407ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 -- 172.21.15.63:0/3061874326 shutdown_connections 2024-05-11T13:27:17.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.467+0000 7fada9600700 1 -- 172.21.15.63:0/3061874326 wait complete. 2024-05-11T13:27:17.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.468+0000 7fada9600700 1 Processor -- start 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.468+0000 7fada9600700 1 -- start start 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada9600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada4075780 0x7fada41a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada9600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada40760a0 0x7fada41a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada4114750 0x7fada41a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada9600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fada4119c40 con 0x7fada40760a0 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada9600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fada4119ac0 con 0x7fada4114750 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada9600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fada4119dc0 con 0x7fada4075780 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fad9a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada40760a0 0x7fada41a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fad9a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada40760a0 0x7fada41a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52686/0 (socket says 172.21.15.63:52686) 2024-05-11T13:27:17.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fad9a000700 1 -- 172.21.15.63:0/609564042 learned_addr learned my addr 172.21.15.63:0/609564042 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada2a00700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada4075780 0x7fada41a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.469+0000 7fada3400700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada4114750 0x7fada41a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9a000700 1 -- 172.21.15.63:0/609564042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada4075780 msgr2=0x7fada41a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9a000700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada4075780 0x7fada41a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9a000700 1 -- 172.21.15.63:0/609564042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada4114750 msgr2=0x7fada41a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9a000700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada4114750 0x7fada41a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9a000700 1 -- 172.21.15.63:0/609564042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada41a51e0 con 0x7fada40760a0 2024-05-11T13:27:17.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fada2a00700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada4075780 0x7fada41a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:27:17.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9a000700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada40760a0 0x7fada41a0710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d980 tx=0x7fad8c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:17.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9be00700 1 -- 172.21.15.63:0/609564042 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c016070 con 0x7fada40760a0 2024-05-11T13:27:17.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9be00700 1 -- 172.21.15.63:0/609564042 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad8c004020 con 0x7fada40760a0 2024-05-11T13:27:17.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada41a54d0 con 0x7fada40760a0 2024-05-11T13:27:17.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.470+0000 7fad9be00700 1 -- 172.21.15.63:0/609564042 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c004bb0 con 0x7fada40760a0 2024-05-11T13:27:17.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.471+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada41acd50 con 0x7fada40760a0 2024-05-11T13:27:17.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.472+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada406a240 con 0x7fada40760a0 2024-05-11T13:27:17.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.474+0000 7fad9be00700 1 -- 172.21.15.63:0/609564042 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fad8c00bb60 con 0x7fada40760a0 2024-05-11T13:27:17.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.476+0000 7fad9be00700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad880b2900 0x7fad880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:17.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.476+0000 7fad9be00700 1 -- 172.21.15.63:0/609564042 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fad8c0e1b90 con 0x7fada40760a0 2024-05-11T13:27:17.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.476+0000 7fada2a00700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad880b2900 0x7fad880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:17.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.477+0000 7fada2a00700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad880b2900 0x7fad880b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad940093a0 tx=0x7fad9400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:17.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.481+0000 7fad9be00700 1 -- 172.21.15.63:0/609564042 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad8c01f020 con 0x7fada40760a0 2024-05-11T13:27:17.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.723+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fada41acf80 con 0x7fada40760a0 2024-05-11T13:27:17.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.724+0000 7fad9be00700 1 -- 172.21.15.63:0/609564042 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18420 (secure 0 0 0) 0x7fad8c0a9cb0 con 0x7fada40760a0 2024-05-11T13:27:17.725 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:27:17.725 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":72,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:27:13.549333+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:27:17.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad880b2900 msgr2=0x7fad880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad880b2900 0x7fad880b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad940093a0 tx=0x7fad9400b040 comp rx=0 tx=0).stop 2024-05-11T13:27:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada40760a0 msgr2=0x7fada41a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada40760a0 0x7fada41a0710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d980 tx=0x7fad8c00dd40 comp rx=0 tx=0).stop 2024-05-11T13:27:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 shutdown_connections 2024-05-11T13:27:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fad880b2900 0x7fad880b4dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada4114750 0x7fada41a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada4075780 0x7fada41a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 --2- 172.21.15.63:0/609564042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada40760a0 0x7fada41a0710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:17.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.727+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 >> 172.21.15.63:0/609564042 conn(0x7fada407a700 msgr2=0x7fada4113590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:17.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.728+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 shutdown_connections 2024-05-11T13:27:17.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:17.728+0000 7fada9600700 1 -- 172.21.15.63:0/609564042 wait complete. 2024-05-11T13:27:18.273 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:17 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/609564042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:27:18.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:17 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/609564042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:27:18.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:17 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/609564042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:27:18.704 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T13:21:55.534075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T13:27:18.705 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-11T13:27:19.167 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:19.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:18 smithi165 ceph-mon[33235]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:19.327 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:18 smithi063 ceph-mon[29809]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:19.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:18 smithi071 ceph-mon[33293]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:21.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:21 smithi063 ceph-mon[29809]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:21.318 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:21 smithi165 ceph-mon[33235]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:21.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:21 smithi071 ceph-mon[33293]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 -- 172.21.15.63:0/860813912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 msgr2=0x7f785c112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 --2- 172.21.15.63:0/860813912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c112640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7854009f90 tx=0x7f785400b330 comp rx=0 tx=0).stop 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 -- 172.21.15.63:0/860813912 shutdown_connections 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 --2- 172.21.15.63:0/860813912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c112640 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 --2- 172.21.15.63:0/860813912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c10b770 0x7f785c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 --2- 172.21.15.63:0/860813912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f785c105770 0x7f785c105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 -- 172.21.15.63:0/860813912 >> 172.21.15.63:0/860813912 conn(0x7f785c101260 msgr2=0x7f785c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:21.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.478+0000 7f7861a00700 1 -- 172.21.15.63:0/860813912 shutdown_connections 2024-05-11T13:27:21.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.479+0000 7f7861a00700 1 -- 172.21.15.63:0/860813912 wait complete. 2024-05-11T13:27:21.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 Processor -- start 2024-05-11T13:27:21.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 -- start start 2024-05-11T13:27:21.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f785c105770 0x7f785c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:21.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:21.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c10b770 0x7f785c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:21.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f785c115850 con 0x7f785c106130 2024-05-11T13:27:21.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1156d0 con 0x7f785c10b770 2024-05-11T13:27:21.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f7861a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1159d0 con 0x7f785c105770 2024-05-11T13:27:21.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f785a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f785a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43528/0 (socket says 172.21.15.63:43528) 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.480+0000 7f785a000700 1 -- 172.21.15.63:0/1924635111 learned_addr learned my addr 172.21.15.63:0/1924635111 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785aa00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f785c105770 0x7f785c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785b400700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c10b770 0x7f785c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785a000700 1 -- 172.21.15.63:0/1924635111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f785c105770 msgr2=0x7f785c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785a000700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f785c105770 0x7f785c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785a000700 1 -- 172.21.15.63:0/1924635111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c10b770 msgr2=0x7f785c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785a000700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c10b770 0x7f785c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785a000700 1 -- 172.21.15.63:0/1924635111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c1a0d30 con 0x7f785c106130 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785b400700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c10b770 0x7f785c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f785a000700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c19c260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f784800e9c0 tx=0x7f784800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f7853400700 1 -- 172.21.15.63:0/1924635111 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784800cbd0 con 0x7f785c106130 2024-05-11T13:27:21.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f7853400700 1 -- 172.21.15.63:0/1924635111 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f784800cd70 con 0x7f785c106130 2024-05-11T13:27:21.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.481+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785c1a1020 con 0x7f785c106130 2024-05-11T13:27:21.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.482+0000 7f7853400700 1 -- 172.21.15.63:0/1924635111 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848005590 con 0x7f785c106130 2024-05-11T13:27:21.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.482+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785c1a88a0 con 0x7f785c106130 2024-05-11T13:27:21.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.483+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785c06a240 con 0x7f785c106130 2024-05-11T13:27:21.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.485+0000 7f7853400700 1 -- 172.21.15.63:0/1924635111 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7848004750 con 0x7f785c106130 2024-05-11T13:27:21.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.487+0000 7f7853400700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f78400b2880 0x7f78400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:21.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.487+0000 7f7853400700 1 -- 172.21.15.63:0/1924635111 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f78480e2070 con 0x7f785c106130 2024-05-11T13:27:21.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.487+0000 7f785aa00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f78400b2880 0x7f78400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:21.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.488+0000 7f785aa00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f78400b2880 0x7f78400b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f784c009bd0 tx=0x7f784c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:21.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.492+0000 7f7853400700 1 -- 172.21.15.63:0/1924635111 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78480aa190 con 0x7f785c106130 2024-05-11T13:27:21.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.729+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f785c02e570 con 0x7f785c106130 2024-05-11T13:27:21.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.730+0000 7f7853400700 1 -- 172.21.15.63:0/1924635111 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f78480a5020 con 0x7f785c106130 2024-05-11T13:27:21.730 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-11T13:27:21.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f78400b2880 msgr2=0x7f78400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:21.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f78400b2880 0x7f78400b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f784c009bd0 tx=0x7f784c009260 comp rx=0 tx=0).stop 2024-05-11T13:27:21.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 msgr2=0x7f785c19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:21.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c19c260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f784800e9c0 tx=0x7f784800ed80 comp rx=0 tx=0).stop 2024-05-11T13:27:21.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 shutdown_connections 2024-05-11T13:27:21.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f78400b2880 0x7f78400b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c10b770 0x7f785c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f785c105770 0x7f785c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 --2- 172.21.15.63:0/1924635111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f785c106130 0x7f785c19c260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:21.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.733+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 >> 172.21.15.63:0/1924635111 conn(0x7f785c101260 msgr2=0x7f785c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:21.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.734+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 shutdown_connections 2024-05-11T13:27:21.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:21.734+0000 7f7861a00700 1 -- 172.21.15.63:0/1924635111 wait complete. 2024-05-11T13:27:22.273 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:22 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1924635111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T13:27:22.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:22 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1924635111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T13:27:22.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:22 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1924635111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T13:27:22.825 INFO:tasks.cephadm:Adding mds.a on smithi063 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.d on smithi063 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.g on smithi063 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.j on smithi063 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.b on smithi071 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.e on smithi071 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.h on smithi071 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.k on smithi071 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.c on smithi165 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.f on smithi165 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.i on smithi165 2024-05-11T13:27:22.826 INFO:tasks.cephadm:Adding mds.l on smithi165 2024-05-11T13:27:22.827 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l' 2024-05-11T13:27:23.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:23 smithi165 ceph-mon[33235]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:23.263 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:27:23.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:23 smithi063 ceph-mon[29809]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:23.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:23 smithi071 ceph-mon[33293]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:24.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:24 smithi165 ceph-mon[33235]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:24.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:24 smithi063 ceph-mon[29809]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:24.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:24 smithi071 ceph-mon[33293]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:25.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 -- 172.21.15.165:0/3115624424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40075f50 msgr2=0x7f4c400763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:25.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 --2- 172.21.15.165:0/3115624424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40075f50 0x7f4c400763d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009990 tx=0x7f4c2c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 -- 172.21.15.165:0/3115624424 shutdown_connections 2024-05-11T13:27:25.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 --2- 172.21.15.165:0/3115624424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c4010b490 0x7f4c4010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 --2- 172.21.15.165:0/3115624424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c40077520 0x7f4c40075980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 --2- 172.21.15.165:0/3115624424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40075f50 0x7f4c400763d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 -- 172.21.15.165:0/3115624424 >> 172.21.15.165:0/3115624424 conn(0x7f4c400feb50 msgr2=0x7f4c40100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:25.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.637+0000 7f4c46a00700 1 -- 172.21.15.165:0/3115624424 shutdown_connections 2024-05-11T13:27:25.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.638+0000 7f4c46a00700 1 -- 172.21.15.165:0/3115624424 wait complete. 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.638+0000 7f4c46a00700 1 Processor -- start 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c46a00700 1 -- start start 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c46a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c40075f50 0x7f4c4019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c46a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40077520 0x7f4c4019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c46a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4010b490 0x7f4c401a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c46a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40111200 con 0x7f4c40075f50 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c46a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40111080 con 0x7f4c4010b490 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c46a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40111380 con 0x7f4c40077520 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c3f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40077520 0x7f4c4019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c3f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40077520 0x7f4c4019bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:46942/0 (socket says 172.21.15.165:46942) 2024-05-11T13:27:25.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.639+0000 7f4c3f400700 1 -- 172.21.15.165:0/384789541 learned_addr learned my addr 172.21.15.165:0/384789541 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3fe00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c40075f50 0x7f4c4019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c44a00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4010b490 0x7f4c401a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3f400700 1 -- 172.21.15.165:0/384789541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4010b490 msgr2=0x7f4c401a2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3f400700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4010b490 0x7f4c401a2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3f400700 1 -- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c40075f50 msgr2=0x7f4c4019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3f400700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c40075f50 0x7f4c4019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3f400700 1 -- 172.21.15.165:0/384789541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c401a3670 con 0x7f4c40077520 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3f400700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40077520 0x7f4c4019bf50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0057c0 tx=0x7f4c2c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3cc00700 1 -- 172.21.15.165:0/384789541 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c027070 con 0x7f4c40077520 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c3cc00700 1 -- 172.21.15.165:0/384789541 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c2c022470 con 0x7f4c40077520 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.640+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c401a3900 con 0x7f4c40077520 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.641+0000 7f4c3cc00700 1 -- 172.21.15.165:0/384789541 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00f6c0 con 0x7f4c40077520 2024-05-11T13:27:25.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.641+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c401a3dd0 con 0x7f4c40077520 2024-05-11T13:27:25.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.642+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4002e570 con 0x7f4c40077520 2024-05-11T13:27:25.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.644+0000 7f4c3cc00700 1 -- 172.21.15.165:0/384789541 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4c2c02a070 con 0x7f4c40077520 2024-05-11T13:27:25.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.645+0000 7f4c3cc00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4c240b2880 0x7f4c240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:25.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.646+0000 7f4c3cc00700 1 -- 172.21.15.165:0/384789541 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f4c2c0e2e10 con 0x7f4c40077520 2024-05-11T13:27:25.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.646+0000 7f4c3fe00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4c240b2880 0x7f4c240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:25.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.647+0000 7f4c3fe00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4c240b2880 0x7f4c240b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006fd0 tx=0x7f4c30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:25.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.650+0000 7f4c3cc00700 1 -- 172.21.15.165:0/384789541 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c2c0e6030 con 0x7f4c40077520 2024-05-11T13:27:25.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.895+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}) -- 0x7f4c40063a90 con 0x7f4c240b2880 2024-05-11T13:27:25.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.914+0000 7f4c3cc00700 1 -- 172.21.15.165:0/384789541 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4c40063a90 con 0x7f4c240b2880 2024-05-11T13:27:25.914 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mds.all update... 2024-05-11T13:27:25.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4c240b2880 msgr2=0x7f4c240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:25.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4c240b2880 0x7f4c240b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006fd0 tx=0x7f4c30008040 comp rx=0 tx=0).stop 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40077520 msgr2=0x7f4c4019bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40077520 0x7f4c4019bf50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0057c0 tx=0x7f4c2c009f90 comp rx=0 tx=0).stop 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 shutdown_connections 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4c240b2880 0x7f4c240b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4010b490 0x7f4c401a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c40075f50 0x7f4c4019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 --2- 172.21.15.165:0/384789541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c40077520 0x7f4c4019bf50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.917+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 >> 172.21.15.165:0/384789541 conn(0x7f4c400feb50 msgr2=0x7f4c401073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.918+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 shutdown_connections 2024-05-11T13:27:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-11T13:27:25.918+0000 7f4c46a00700 1 -- 172.21.15.165:0/384789541 wait complete. 2024-05-11T13:27:26.698 DEBUG:teuthology.orchestra.run.smithi063:mds.a> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.a.service 2024-05-11T13:27:26.706 DEBUG:teuthology.orchestra.run.smithi063:mds.d> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.d.service 2024-05-11T13:27:26.751 DEBUG:teuthology.orchestra.run.smithi063:mds.g> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.g.service 2024-05-11T13:27:26.798 DEBUG:teuthology.orchestra.run.smithi063:mds.j> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.j.service 2024-05-11T13:27:26.847 DEBUG:teuthology.orchestra.run.smithi071:mds.b> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.b.service 2024-05-11T13:27:26.854 DEBUG:teuthology.orchestra.run.smithi071:mds.e> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.e.service 2024-05-11T13:27:26.901 DEBUG:teuthology.orchestra.run.smithi071:mds.h> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.h.service 2024-05-11T13:27:26.946 DEBUG:teuthology.orchestra.run.smithi071:mds.k> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.k.service 2024-05-11T13:27:26.992 DEBUG:teuthology.orchestra.run.smithi165:mds.c> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.c.service 2024-05-11T13:27:26.999 DEBUG:teuthology.orchestra.run.smithi165:mds.f> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.f.service 2024-05-11T13:27:27.046 DEBUG:teuthology.orchestra.run.smithi165:mds.i> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.i.service 2024-05-11T13:27:27.092 DEBUG:teuthology.orchestra.run.smithi165:mds.l> sudo journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.l.service 2024-05-11T13:27:27.136 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-11T13:27:27.136 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:27:27.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:27.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:27:27.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: Saving service mds.all spec with placement smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-11T13:27:27.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:27.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:27.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:27.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:27.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:27.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:27.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:27.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:27.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:26 smithi165 ceph-mon[33235]: Deploying daemon mds.l on smithi165 2024-05-11T13:27:27.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:27.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:27:27.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: Saving service mds.all spec with placement smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-11T13:27:27.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:27.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:26 smithi063 ceph-mon[29809]: Deploying daemon mds.l on smithi165 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: Saving service mds.all spec with placement smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:27.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:26 smithi071 ceph-mon[33293]: Deploying daemon mds.l on smithi165 2024-05-11T13:27:27.602 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:29.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:28 smithi063 ceph-mon[29809]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:29.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:28 smithi071 ceph-mon[33293]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:29.298 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:28 smithi165 ceph-mon[33235]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:29.299 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:28 smithi165 systemd[1]: Starting Ceph mds.l for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:27:29.567 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 podman[62549]: 2024-05-11 13:27:29.204501738 +0000 UTC m=+0.034790748 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:27:29.567 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 podman[62549]: 2024-05-11 13:27:29.309393297 +0000 UTC m=+0.139682309 container create 3a85cb4d9fd15a3cea6a28b8bc03c92e27d813551619eb3d6bdd26094f16a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T13:27:30.067 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 podman[62549]: 2024-05-11 13:27:29.588439262 +0000 UTC m=+0.418728305 container init 3a85cb4d9fd15a3cea6a28b8bc03c92e27d813551619eb3d6bdd26094f16a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:27:30.067 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 podman[62549]: 2024-05-11 13:27:29.598569086 +0000 UTC m=+0.428858104 container start 3a85cb4d9fd15a3cea6a28b8bc03c92e27d813551619eb3d6bdd26094f16a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:27:30.068 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 ceph-mds[62568]: -- 172.21.15.165:0/1185147181 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559093a31a40 con 0x559093a04c00 2024-05-11T13:27:30.068 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l[62564]: starting mds.l at 2024-05-11T13:27:30.068 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 bash[62549]: 3a85cb4d9fd15a3cea6a28b8bc03c92e27d813551619eb3d6bdd26094f16a9be 2024-05-11T13:27:30.068 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:27:29 smithi165 systemd[1]: Started Ceph mds.l for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:27:30.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 -- 172.21.15.63:0/1658344075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406b230 msgr2=0x7f34a406b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:30.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 --2- 172.21.15.63:0/1658344075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406b230 0x7f34a406b6b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3490009990 tx=0x7f3490009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 -- 172.21.15.63:0/1658344075 shutdown_connections 2024-05-11T13:27:30.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 --2- 172.21.15.63:0/1658344075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a406bbf0 0x7f34a41103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 --2- 172.21.15.63:0/1658344075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a41095c0 0x7f34a41099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 --2- 172.21.15.63:0/1658344075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406b230 0x7f34a406b6b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 -- 172.21.15.63:0/1658344075 >> 172.21.15.63:0/1658344075 conn(0x7f34a4079400 msgr2=0x7f34a4079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:30.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 -- 172.21.15.63:0/1658344075 shutdown_connections 2024-05-11T13:27:30.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.182+0000 7f34abc00700 1 -- 172.21.15.63:0/1658344075 wait complete. 2024-05-11T13:27:30.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.183+0000 7f34abc00700 1 Processor -- start 2024-05-11T13:27:30.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.183+0000 7f34abc00700 1 -- start start 2024-05-11T13:27:30.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f34abc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a406b230 0x7f34a419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:30.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f34abc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406bbf0 0x7f34a419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:30.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f34abc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a41095c0 0x7f34a41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:30.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f34abc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41135f0 con 0x7f34a406bbf0 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f34abc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4113470 con 0x7f34a406b230 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f34abc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4113770 con 0x7f34a41095c0 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f349be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406bbf0 0x7f34a419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f349be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406bbf0 0x7f34a419c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56022/0 (socket says 172.21.15.63:56022) 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f349be00700 1 -- 172.21.15.63:0/1265702615 learned_addr learned my addr 172.21.15.63:0/1265702615 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.184+0000 7f34a9200700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a406b230 0x7f34a419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.185+0000 7f34a9c00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a41095c0 0x7f34a41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.185+0000 7f349be00700 1 -- 172.21.15.63:0/1265702615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a41095c0 msgr2=0x7f34a41a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.185+0000 7f349be00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a41095c0 0x7f34a41a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.185+0000 7f349be00700 1 -- 172.21.15.63:0/1265702615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a406b230 msgr2=0x7f34a419be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.185+0000 7f349be00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a406b230 0x7f34a419be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.185+0000 7f349be00700 1 -- 172.21.15.63:0/1265702615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a41a0e20 con 0x7f34a406bbf0 2024-05-11T13:27:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.185+0000 7f349be00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406bbf0 0x7f34a419c3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f349000b650 tx=0x7f349000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.186+0000 7f34a2000700 1 -- 172.21.15.63:0/1265702615 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490027070 con 0x7f34a406bbf0 2024-05-11T13:27:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.186+0000 7f34a2000700 1 -- 172.21.15.63:0/1265702615 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3490022470 con 0x7f34a406bbf0 2024-05-11T13:27:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.186+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a41a10b0 con 0x7f34a406bbf0 2024-05-11T13:27:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.186+0000 7f34a2000700 1 -- 172.21.15.63:0/1265702615 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490017650 con 0x7f34a406bbf0 2024-05-11T13:27:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.186+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41a8990 con 0x7f34a406bbf0 2024-05-11T13:27:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.187+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a4104c80 con 0x7f34a406bbf0 2024-05-11T13:27:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.190+0000 7f34a2000700 1 -- 172.21.15.63:0/1265702615 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3490009d10 con 0x7f34a406bbf0 2024-05-11T13:27:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.192+0000 7f34a2000700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f34880b2880 0x7f34880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.192+0000 7f34a2000700 1 -- 172.21.15.63:0/1265702615 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f34900e2d10 con 0x7f34a406bbf0 2024-05-11T13:27:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.192+0000 7f34a9200700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f34880b2880 0x7f34880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.193+0000 7f34a9200700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f34880b2880 0x7f34880b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34940045b0 tx=0x7f349400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.195+0000 7f34a2000700 1 -- 172.21.15.63:0/1265702615 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3490013070 con 0x7f34a406bbf0 2024-05-11T13:27:30.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.480+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34a419d170 con 0x7f34a406bbf0 2024-05-11T13:27:30.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.480+0000 7f34a2000700 1 -- 172.21.15.63:0/1265702615 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f34900aaec0 con 0x7f34a406bbf0 2024-05-11T13:27:30.481 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:27:30.481 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-11T13:27:30.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.483+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f34880b2880 msgr2=0x7f34880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:30.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.483+0000 7f34abc00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f34880b2880 0x7f34880b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34940045b0 tx=0x7f349400b3c0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.483+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406bbf0 msgr2=0x7f34a419c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:30.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.483+0000 7f34abc00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406bbf0 0x7f34a419c3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f349000b650 tx=0x7f349000bfc0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 shutdown_connections 2024-05-11T13:27:30.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f34880b2880 0x7f34880b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a41095c0 0x7f34a41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a406b230 0x7f34a419be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 --2- 172.21.15.63:0/1265702615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a406bbf0 0x7f34a419c3a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:30.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 >> 172.21.15.63:0/1265702615 conn(0x7f34a4079400 msgr2=0x7f34a4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:30.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 shutdown_connections 2024-05-11T13:27:30.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:30.484+0000 7f34abc00700 1 -- 172.21.15.63:0/1265702615 wait complete. 2024-05-11T13:27:30.486 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 2 2024-05-11T13:27:30.866 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:30.866 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6832/2703259087,v1:172.21.15.165:6833/2703259087] up:boot 2024-05-11T13:27:30.866 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: fsmap 1 up:standby 2024-05-11T13:27:30.866 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:27:30.866 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:30.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:30.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:30.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:30.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:30.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:30.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: Deploying daemon mds.k on smithi071 2024-05-11T13:27:30.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:30 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1265702615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6832/2703259087,v1:172.21.15.165:6833/2703259087] up:boot 2024-05-11T13:27:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: fsmap 1 up:standby 2024-05-11T13:27:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:27:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:30.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:30.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:30.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:30.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:30.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: Deploying daemon mds.k on smithi071 2024-05-11T13:27:30.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:30 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1265702615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:31.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6832/2703259087,v1:172.21.15.165:6833/2703259087] up:boot 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: fsmap 1 up:standby 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: Deploying daemon mds.k on smithi071 2024-05-11T13:27:31.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:30 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1265702615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:32.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:32 smithi063 ceph-mon[29809]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:32.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:32 smithi071 ceph-mon[33293]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:33.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:32 smithi165 ceph-mon[33235]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:33.431 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 systemd[1]: Starting Ceph mds.k for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:27:33.517 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:27:33.705 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 podman[62214]: 2024-05-11 13:27:33.349409237 +0000 UTC m=+0.032867494 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:27:33.705 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 podman[62214]: 2024-05-11 13:27:33.480776653 +0000 UTC m=+0.164234937 container create e4eb76c487caca452cf11df893a80fc4e30e8fa1dfd48de0e836e5df8e87c1b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:27:33.995 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:34.204 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 podman[62214]: 2024-05-11 13:27:33.759085566 +0000 UTC m=+0.442543831 container init e4eb76c487caca452cf11df893a80fc4e30e8fa1dfd48de0e836e5df8e87c1b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-05-11T13:27:34.204 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 podman[62214]: 2024-05-11 13:27:33.767358198 +0000 UTC m=+0.450816446 container start e4eb76c487caca452cf11df893a80fc4e30e8fa1dfd48de0e836e5df8e87c1b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-11T13:27:34.204 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 ceph-mds[62241]: -- 172.21.15.71:0/3262595306 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a915e9a40 con 0x556a915bc800 2024-05-11T13:27:34.204 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k[62232]: starting mds.k at 2024-05-11T13:27:34.204 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 bash[62214]: e4eb76c487caca452cf11df893a80fc4e30e8fa1dfd48de0e836e5df8e87c1b4 2024-05-11T13:27:34.204 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:27:33 smithi071 systemd[1]: Started Ceph mds.k for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:27:35.275 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:35.275 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.275 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.275 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.275 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:35.276 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:35.276 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:35.276 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] up:boot 2024-05-11T13:27:35.276 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: fsmap 2 up:standby 2024-05-11T13:27:35.276 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:35 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:27:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:35.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:35.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:35.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] up:boot 2024-05-11T13:27:35.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: fsmap 2 up:standby 2024-05-11T13:27:35.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:35 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:27:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:35.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:35.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] up:boot 2024-05-11T13:27:35.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: fsmap 2 up:standby 2024-05-11T13:27:35.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:35 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:27:36.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:36 smithi063 ceph-mon[29809]: Deploying daemon mds.j on smithi063 2024-05-11T13:27:36.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:36 smithi165 ceph-mon[33235]: Deploying daemon mds.j on smithi063 2024-05-11T13:27:36.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:36 smithi071 ceph-mon[33293]: Deploying daemon mds.j on smithi063 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 -- 172.21.15.63:0/4196446536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37241095c0 msgr2=0x7f37241099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 --2- 172.21.15.63:0/4196446536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37241095c0 0x7f37241099a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3714009990 tx=0x7f3714009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 -- 172.21.15.63:0/4196446536 shutdown_connections 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 --2- 172.21.15.63:0/4196446536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f372406bbf0 0x7f37241103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 --2- 172.21.15.63:0/4196446536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37241095c0 0x7f37241099a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 --2- 172.21.15.63:0/4196446536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f372406b230 0x7f372406b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 -- 172.21.15.63:0/4196446536 >> 172.21.15.63:0/4196446536 conn(0x7f3724079400 msgr2=0x7f3724079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 -- 172.21.15.63:0/4196446536 shutdown_connections 2024-05-11T13:27:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.481+0000 7f3729600700 1 -- 172.21.15.63:0/4196446536 wait complete. 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.483+0000 7f3729600700 1 Processor -- start 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.483+0000 7f3729600700 1 -- start start 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3729600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372406b230 0x7f372419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3729600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f372406bbf0 0x7f372419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3729600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37241095c0 0x7f37241a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3729600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3724113600 con 0x7f372406b230 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3729600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3724113480 con 0x7f372406bbf0 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3729600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3724113780 con 0x7f37241095c0 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372406b230 0x7f372419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372406b230 0x7f372419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56034/0 (socket says 172.21.15.63:56034) 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 -- 172.21.15.63:0/1299044547 learned_addr learned my addr 172.21.15.63:0/1299044547 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722000700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f372406bbf0 0x7f372419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:36.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3723400700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37241095c0 0x7f37241a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 -- 172.21.15.63:0/1299044547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37241095c0 msgr2=0x7f37241a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37241095c0 0x7f37241a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 -- 172.21.15.63:0/1299044547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f372406bbf0 msgr2=0x7f372419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f372406bbf0 0x7f372419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.484+0000 7f3722a00700 1 -- 172.21.15.63:0/1299044547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37241a0d60 con 0x7f372406b230 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.485+0000 7f3722a00700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372406b230 0x7f372419bd50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3714005910 tx=0x7f37140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.485+0000 7f371b400700 1 -- 172.21.15.63:0/1299044547 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714027070 con 0x7f372406b230 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.485+0000 7f371b400700 1 -- 172.21.15.63:0/1299044547 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f371400fc00 con 0x7f372406b230 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.485+0000 7f371b400700 1 -- 172.21.15.63:0/1299044547 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714022d40 con 0x7f372406b230 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.485+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37241a0ff0 con 0x7f372406b230 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.485+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241a88d0 con 0x7f372406b230 2024-05-11T13:27:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.487+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372406a240 con 0x7f372406b230 2024-05-11T13:27:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.489+0000 7f371b400700 1 -- 172.21.15.63:0/1299044547 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f371402a070 con 0x7f372406b230 2024-05-11T13:27:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.491+0000 7f371b400700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f37080b2900 0x7f37080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.491+0000 7f371b400700 1 -- 172.21.15.63:0/1299044547 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f37140e3210 con 0x7f372406b230 2024-05-11T13:27:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.491+0000 7f3722000700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f37080b2900 0x7f37080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.492+0000 7f3722000700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f37080b2900 0x7f37080b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3710005950 tx=0x7f37100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.494+0000 7f371b400700 1 -- 172.21.15.63:0/1299044547 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3714031070 con 0x7f372406b230 2024-05-11T13:27:36.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.788+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37241a8aa0 con 0x7f372406b230 2024-05-11T13:27:36.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.789+0000 7f371b400700 1 -- 172.21.15.63:0/1299044547 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f3714013070 con 0x7f372406b230 2024-05-11T13:27:36.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:27:36.790 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T13:27:36.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f37080b2900 msgr2=0x7f37080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:36.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f37080b2900 0x7f37080b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3710005950 tx=0x7f37100058e0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372406b230 msgr2=0x7f372419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372406b230 0x7f372419bd50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3714005910 tx=0x7f37140047b0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 shutdown_connections 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f37080b2900 0x7f37080b4dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37241095c0 0x7f37241a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372406b230 0x7f372419bd50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 --2- 172.21.15.63:0/1299044547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f372406bbf0 0x7f372419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.792+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 >> 172.21.15.63:0/1299044547 conn(0x7f3724079400 msgr2=0x7f372410e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.793+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 shutdown_connections 2024-05-11T13:27:36.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:36.793+0000 7f3729600700 1 -- 172.21.15.63:0/1299044547 wait complete. 2024-05-11T13:27:36.794 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 3 2024-05-11T13:27:37.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:37 smithi165 ceph-mon[33235]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:37.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:37 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1299044547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:37.355 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:37 smithi063 ceph-mon[29809]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:37.355 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:37 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1299044547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:37.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:37 smithi071 ceph-mon[33293]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:37.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:37 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1299044547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:38.307 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:38 smithi063 ceph-mon[29809]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:38.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:38 smithi165 ceph-mon[33235]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:38.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:38 smithi071 ceph-mon[33293]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:39.434 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:39 smithi063 systemd[1]: Starting Ceph mds.j for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:27:39.807 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:27:39.886 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:39 smithi063 podman[67544]: 2024-05-11 13:27:39.428418015 +0000 UTC m=+0.038690951 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:27:39.886 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:39 smithi063 podman[67544]: 2024-05-11 13:27:39.58213857 +0000 UTC m=+0.192411544 container create bba3825e142c3e46d11a4ce8fef13ce0d41b97fc1e4aab7c9576a812d5c30fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-11T13:27:40.184 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:39 smithi063 podman[67544]: 2024-05-11 13:27:39.986612156 +0000 UTC m=+0.596885137 container init bba3825e142c3e46d11a4ce8fef13ce0d41b97fc1e4aab7c9576a812d5c30fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-11T13:27:40.185 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:39 smithi063 podman[67544]: 2024-05-11 13:27:39.997287083 +0000 UTC m=+0.607560029 container start bba3825e142c3e46d11a4ce8fef13ce0d41b97fc1e4aab7c9576a812d5c30fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:27:40.185 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mds[67581]: -- 172.21.15.63:0/8456738 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592248984e0 con 0x55922486f000 2024-05-11T13:27:40.185 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mds[67581]: -- 172.21.15.63:0/8456738 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55922489ba40 con 0x55922486f000 2024-05-11T13:27:40.185 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j[67559]: starting mds.j at 2024-05-11T13:27:40.185 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:40 smithi063 bash[67544]: bba3825e142c3e46d11a4ce8fef13ce0d41b97fc1e4aab7c9576a812d5c30fd3 2024-05-11T13:27:40.185 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 13:27:40 smithi063 systemd[1]: Started Ceph mds.j for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:27:40.288 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:40.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mon[29809]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:40.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:40.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:40.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:40.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:40.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:40.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:40 smithi071 ceph-mon[33293]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:41.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:40 smithi165 ceph-mon[33235]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:41.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:41.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:41.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:41.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:41.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:41.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:41.879 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:41 smithi063 ceph-mon[29809]: Deploying daemon mds.d on smithi063 2024-05-11T13:27:41.879 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:41 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.63:6834/1042825923,v1:172.21.15.63:6835/1042825923] up:boot 2024-05-11T13:27:41.879 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:41 smithi063 ceph-mon[29809]: fsmap 3 up:standby 2024-05-11T13:27:41.879 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T13:27:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:41 smithi071 ceph-mon[33293]: Deploying daemon mds.d on smithi063 2024-05-11T13:27:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:41 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.63:6834/1042825923,v1:172.21.15.63:6835/1042825923] up:boot 2024-05-11T13:27:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:41 smithi071 ceph-mon[33293]: fsmap 3 up:standby 2024-05-11T13:27:41.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T13:27:42.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:41 smithi165 ceph-mon[33235]: Deploying daemon mds.d on smithi063 2024-05-11T13:27:42.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:41 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.63:6834/1042825923,v1:172.21.15.63:6835/1042825923] up:boot 2024-05-11T13:27:42.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:41 smithi165 ceph-mon[33235]: fsmap 3 up:standby 2024-05-11T13:27:42.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T13:27:42.861 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:42 smithi063 ceph-mon[29809]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:42.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:42 smithi071 ceph-mon[33293]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.024+0000 7f581c600700 1 -- 172.21.15.63:0/2909177094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58141096b0 msgr2=0x7f5814109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.024+0000 7f581c600700 1 --2- 172.21.15.63:0/2909177094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58141096b0 0x7f5814109a90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5808009990 tx=0x7f5808009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.025+0000 7f581c600700 1 -- 172.21.15.63:0/2909177094 shutdown_connections 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.025+0000 7f581c600700 1 --2- 172.21.15.63:0/2909177094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5814103f60 0x7f5814110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.025+0000 7f581c600700 1 --2- 172.21.15.63:0/2909177094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58141096b0 0x7f5814109a90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.025+0000 7f581c600700 1 --2- 172.21.15.63:0/2909177094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58141035a0 0x7f5814103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.025+0000 7f581c600700 1 -- 172.21.15.63:0/2909177094 >> 172.21.15.63:0/2909177094 conn(0x7f581407a700 msgr2=0x7f581407ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.025+0000 7f581c600700 1 -- 172.21.15.63:0/2909177094 shutdown_connections 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.025+0000 7f581c600700 1 -- 172.21.15.63:0/2909177094 wait complete. 2024-05-11T13:27:43.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 Processor -- start 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 -- start start 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58141035a0 0x7f581419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814103f60 0x7f581419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58141096b0 0x7f5814199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58141135f0 con 0x7f5814103f60 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5814113470 con 0x7f58141035a0 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.026+0000 7f581c600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5814113770 con 0x7f58141096b0 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814103f60 0x7f581419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814103f60 0x7f581419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39624/0 (socket says 172.21.15.63:39624) 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 -- 172.21.15.63:0/3985312943 learned_addr learned my addr 172.21.15.63:0/3985312943 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819c00700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58141035a0 0x7f581419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f581a600700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58141096b0 0x7f5814199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 -- 172.21.15.63:0/3985312943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58141096b0 msgr2=0x7f5814199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58141096b0 0x7f5814199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 -- 172.21.15.63:0/3985312943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58141035a0 msgr2=0x7f581419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58141035a0 0x7f581419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 -- 172.21.15.63:0/3985312943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581419a030 con 0x7f5814103f60 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.027+0000 7f5819200700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814103f60 0x7f581419fc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f580000c9b0 tx=0x7f580000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.028+0000 7f5812000700 1 -- 172.21.15.63:0/3985312943 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800007b00 con 0x7f5814103f60 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.028+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581419a320 con 0x7f5814103f60 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.028+0000 7f5812000700 1 -- 172.21.15.63:0/3985312943 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5800007ca0 con 0x7f5814103f60 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.028+0000 7f5812000700 1 -- 172.21.15.63:0/3985312943 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800018660 con 0x7f5814103f60 2024-05-11T13:27:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.028+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a6530 con 0x7f5814103f60 2024-05-11T13:27:43.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.029+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814104cc0 con 0x7f5814103f60 2024-05-11T13:27:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.031+0000 7f5812000700 1 -- 172.21.15.63:0/3985312943 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f580000f450 con 0x7f5814103f60 2024-05-11T13:27:43.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.033+0000 7f5812000700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57fc0b2900 0x7f57fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:43.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.033+0000 7f5812000700 1 -- 172.21.15.63:0/3985312943 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f58000e1e10 con 0x7f5814103f60 2024-05-11T13:27:43.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.033+0000 7f5819c00700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57fc0b2900 0x7f57fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.034+0000 7f5819c00700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57fc0b2900 0x7f57fc0b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f580800b5c0 tx=0x7f580801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.037+0000 7f5812000700 1 -- 172.21.15.63:0/3985312943 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58000e5030 con 0x7f5814103f60 2024-05-11T13:27:43.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:42 smithi165 ceph-mon[33235]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:43.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.326+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f581402e570 con 0x7f5814103f60 2024-05-11T13:27:43.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.327+0000 7f5812000700 1 -- 172.21.15.63:0/3985312943 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f5800014030 con 0x7f5814103f60 2024-05-11T13:27:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:27:43.328 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T13:27:43.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57fc0b2900 msgr2=0x7f57fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:43.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57fc0b2900 0x7f57fc0b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f580800b5c0 tx=0x7f580801a040 comp rx=0 tx=0).stop 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814103f60 msgr2=0x7f581419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814103f60 0x7f581419fc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f580000c9b0 tx=0x7f580000cd70 comp rx=0 tx=0).stop 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 shutdown_connections 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57fc0b2900 0x7f57fc0b4dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58141096b0 0x7f5814199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58141035a0 0x7f581419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 --2- 172.21.15.63:0/3985312943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814103f60 0x7f581419fc10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.330+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 >> 172.21.15.63:0/3985312943 conn(0x7f581407a700 msgr2=0x7f5814102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.331+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 shutdown_connections 2024-05-11T13:27:43.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:43.331+0000 7f581c600700 1 -- 172.21.15.63:0/3985312943 wait complete. 2024-05-11T13:27:43.332 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 4 2024-05-11T13:27:43.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:43 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3985312943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:43.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:43 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3985312943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:44.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:43 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3985312943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:44.905 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:44 smithi063 ceph-mon[29809]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:44.905 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 13:27:44 smithi063 podman[68010]: 2024-05-11 13:27:44.73725754 +0000 UTC m=+0.205515807 container create 4d1e9d1e3aab7a5cda8671a02e4165c096b70cc7fdb6ae243ea20d2100225bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-11T13:27:44.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:44 smithi071 ceph-mon[33293]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:44 smithi165 ceph-mon[33235]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:45.435 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 13:27:45 smithi063 podman[68010]: 2024-05-11 13:27:45.017936519 +0000 UTC m=+0.486194780 container init 4d1e9d1e3aab7a5cda8671a02e4165c096b70cc7fdb6ae243ea20d2100225bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:27:45.436 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 13:27:45 smithi063 podman[68010]: 2024-05-11 13:27:45.028274148 +0000 UTC m=+0.496532388 container start 4d1e9d1e3aab7a5cda8671a02e4165c096b70cc7fdb6ae243ea20d2100225bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:27:45.436 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 13:27:45 smithi063 ceph-mds[68029]: -- 172.21.15.63:0/3524810547 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc5d11fa40 con 0x55cc5d0f2800 2024-05-11T13:27:45.436 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 13:27:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d[68025]: starting mds.d at 2024-05-11T13:27:45.436 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 13:27:45 smithi063 bash[68010]: 4d1e9d1e3aab7a5cda8671a02e4165c096b70cc7fdb6ae243ea20d2100225bc9 2024-05-11T13:27:45.436 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 13:27:45 smithi063 systemd[1]: Started Ceph mds.d for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:27:46.398 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:27:46.528 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: Deploying daemon mds.e on smithi071 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.63:6836/3240025359,v1:172.21.15.63:6837/3240025359] up:boot 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: fsmap 4 up:standby 2024-05-11T13:27:46.529 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: Deploying daemon mds.e on smithi071 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.63:6836/3240025359,v1:172.21.15.63:6837/3240025359] up:boot 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: fsmap 4 up:standby 2024-05-11T13:27:46.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T13:27:46.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:46.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:46.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: Deploying daemon mds.e on smithi071 2024-05-11T13:27:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.63:6836/3240025359,v1:172.21.15.63:6837/3240025359] up:boot 2024-05-11T13:27:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: fsmap 4 up:standby 2024-05-11T13:27:46.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T13:27:46.864 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:48.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:48 smithi063 ceph-mon[29809]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:48.839 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:48 smithi071 ceph-mon[33293]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.986+0000 7f6efe400700 1 -- 172.21.15.63:0/1636738725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 msgr2=0x7f6ef8105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 --2- 172.21.15.63:0/1636738725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef8105bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009990 tx=0x7f6ee4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 -- 172.21.15.63:0/1636738725 shutdown_connections 2024-05-11T13:27:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 --2- 172.21.15.63:0/1636738725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef8106130 0x7f6ef8112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 --2- 172.21.15.63:0/1636738725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef810b770 0x7f6ef810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 --2- 172.21.15.63:0/1636738725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef8105bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 -- 172.21.15.63:0/1636738725 >> 172.21.15.63:0/1636738725 conn(0x7f6ef8101260 msgr2=0x7f6ef8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 -- 172.21.15.63:0/1636738725 shutdown_connections 2024-05-11T13:27:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.987+0000 7f6efe400700 1 -- 172.21.15.63:0/1636738725 wait complete. 2024-05-11T13:27:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.988+0000 7f6efe400700 1 Processor -- start 2024-05-11T13:27:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.988+0000 7f6efe400700 1 -- start start 2024-05-11T13:27:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6efe400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6efe400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef8106130 0x7f6ef81a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6efe400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef810b770 0x7f6ef819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6efe400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8115820 con 0x7f6ef8105770 2024-05-11T13:27:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6efe400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81156a0 con 0x7f6ef8106130 2024-05-11T13:27:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6efe400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81159a0 con 0x7f6ef810b770 2024-05-11T13:27:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6ef7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6ef7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef81a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39660/0 (socket says 172.21.15.63:39660) 2024-05-11T13:27:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6ef7400700 1 -- 172.21.15.63:0/2306909354 learned_addr learned my addr 172.21.15.63:0/2306909354 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6ef6a00700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef8106130 0x7f6ef81a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.989+0000 7f6ef7e00700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef810b770 0x7f6ef819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6ef7400700 1 -- 172.21.15.63:0/2306909354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef810b770 msgr2=0x7f6ef819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6ef7400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef810b770 0x7f6ef819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6ef7400700 1 -- 172.21.15.63:0/2306909354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef8106130 msgr2=0x7f6ef81a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6ef7400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef8106130 0x7f6ef81a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6ef7400700 1 -- 172.21.15.63:0/2306909354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef819c220 con 0x7f6ef8105770 2024-05-11T13:27:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6ef7e00700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef810b770 0x7f6ef819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:27:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6ef7400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef81a18b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c9b0 tx=0x7f6ee800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6eefe00700 1 -- 172.21.15.63:0/2306909354 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007b00 con 0x7f6ef8105770 2024-05-11T13:27:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6eefe00700 1 -- 172.21.15.63:0/2306909354 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ee8007ca0 con 0x7f6ef8105770 2024-05-11T13:27:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6eefe00700 1 -- 172.21.15.63:0/2306909354 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8018660 con 0x7f6ef8105770 2024-05-11T13:27:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.990+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef819c510 con 0x7f6ef8105770 2024-05-11T13:27:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.991+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef81a8780 con 0x7f6ef8105770 2024-05-11T13:27:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.992+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef806a240 con 0x7f6ef8105770 2024-05-11T13:27:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.994+0000 7f6eefe00700 1 -- 172.21.15.63:0/2306909354 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6ee800f450 con 0x7f6ef8105770 2024-05-11T13:27:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.995+0000 7f6eefe00700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6edc0b2880 0x7f6edc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:48.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.996+0000 7f6eefe00700 1 -- 172.21.15.63:0/2306909354 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f6ee80e1d80 con 0x7f6ef8105770 2024-05-11T13:27:48.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.996+0000 7f6ef6a00700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6edc0b2880 0x7f6edc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:48.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:48.997+0000 7f6ef6a00700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6edc0b2880 0x7f6edc0b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400b5c0 tx=0x7f6ee4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:49.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.000+0000 7f6eefe00700 1 -- 172.21.15.63:0/2306909354 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ee8014030 con 0x7f6ef8105770 2024-05-11T13:27:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:48 smithi165 ceph-mon[33235]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:49.142 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:27:48 smithi071 podman[62529]: 2024-05-11 13:27:48.844487245 +0000 UTC m=+0.134709846 container create b7299ece7f763dbd9c263b5a5ce4bd3c7d54547afe1e7e94cd1411c6127e434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-11T13:27:49.142 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:27:49 smithi071 podman[62529]: 2024-05-11 13:27:49.141165028 +0000 UTC m=+0.431387639 container init b7299ece7f763dbd9c263b5a5ce4bd3c7d54547afe1e7e94cd1411c6127e434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-11T13:27:49.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.288+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef802e570 con 0x7f6ef8105770 2024-05-11T13:27:49.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.289+0000 7f6eefe00700 1 -- 172.21.15.63:0/2306909354 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f6ee80a9ea0 con 0x7f6ef8105770 2024-05-11T13:27:49.290 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:27:49.290 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T13:27:49.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6edc0b2880 msgr2=0x7f6edc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:49.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6edc0b2880 0x7f6edc0b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400b5c0 tx=0x7f6ee4005960 comp rx=0 tx=0).stop 2024-05-11T13:27:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 msgr2=0x7f6ef81a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef81a18b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c9b0 tx=0x7f6ee800ccc0 comp rx=0 tx=0).stop 2024-05-11T13:27:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 shutdown_connections 2024-05-11T13:27:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f6edc0b2880 0x7f6edc0b4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef810b770 0x7f6ef819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef8105770 0x7f6ef81a18b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 --2- 172.21.15.63:0/2306909354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef8106130 0x7f6ef81a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:49.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 >> 172.21.15.63:0/2306909354 conn(0x7f6ef8101260 msgr2=0x7f6ef81018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:49.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 shutdown_connections 2024-05-11T13:27:49.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:49.293+0000 7f6efe400700 1 -- 172.21.15.63:0/2306909354 wait complete. 2024-05-11T13:27:49.296 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 5 2024-05-11T13:27:49.454 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:27:49 smithi071 podman[62529]: 2024-05-11 13:27:49.150850881 +0000 UTC m=+0.441073489 container start b7299ece7f763dbd9c263b5a5ce4bd3c7d54547afe1e7e94cd1411c6127e434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:27:49.454 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mds[62548]: -- 172.21.15.71:0/2390006048 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1215b1a40 con 0x55b121585000 2024-05-11T13:27:49.454 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:27:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e[62544]: starting mds.e at 2024-05-11T13:27:49.455 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:27:49 smithi071 bash[62529]: b7299ece7f763dbd9c263b5a5ce4bd3c7d54547afe1e7e94cd1411c6127e434d 2024-05-11T13:27:49.455 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:27:49 smithi071 systemd[1]: Started Ceph mds.e for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:27:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:49 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2306909354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:49 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:49.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:49 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:49.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:49 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:49.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:49 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:49.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:49 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:49.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:49 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2306909354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:49 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:49 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2306909354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:49 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:49 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:49 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:49 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:49 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:49 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:50.776 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:50 smithi165 ceph-mon[33235]: Deploying daemon mds.f on smithi165 2024-05-11T13:27:50.776 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:50 smithi165 ceph-mon[33235]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:50.776 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:50 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] up:boot 2024-05-11T13:27:50.776 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:50 smithi165 ceph-mon[33235]: fsmap 5 up:standby 2024-05-11T13:27:50.776 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:27:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:50 smithi063 ceph-mon[29809]: Deploying daemon mds.f on smithi165 2024-05-11T13:27:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:50 smithi063 ceph-mon[29809]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:50 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] up:boot 2024-05-11T13:27:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:50 smithi063 ceph-mon[29809]: fsmap 5 up:standby 2024-05-11T13:27:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:27:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:50 smithi071 ceph-mon[33293]: Deploying daemon mds.f on smithi165 2024-05-11T13:27:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:50 smithi071 ceph-mon[33293]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:50 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] up:boot 2024-05-11T13:27:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:50 smithi071 ceph-mon[33293]: fsmap 5 up:standby 2024-05-11T13:27:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:27:52.149 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:27:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:52 smithi165 ceph-mon[33235]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:52 smithi063 ceph-mon[29809]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:52.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:52 smithi071 ceph-mon[33293]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:52.992 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:27:52 smithi165 podman[62854]: 2024-05-11 13:27:52.795047158 +0000 UTC m=+0.143662364 container create dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-11T13:27:53.566 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:27:53 smithi165 podman[62854]: 2024-05-11 13:27:53.090572294 +0000 UTC m=+0.439187494 container init dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-05-11T13:27:53.566 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:27:53 smithi165 podman[62854]: 2024-05-11 13:27:53.101059563 +0000 UTC m=+0.449674736 container start dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T13:27:53.566 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:27:53 smithi165 ceph-mds[62880]: -- 172.21.15.165:0/2428737361 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597349c7a40 con 0x55973499ac00 2024-05-11T13:27:53.566 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:27:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f[62874]: starting mds.f at 2024-05-11T13:27:53.566 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:27:53 smithi165 bash[62854]: dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 2024-05-11T13:27:53.566 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:27:53 smithi165 systemd[1]: Started Ceph mds.f for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:27:54.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:54.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:54.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:54.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: Deploying daemon mds.c on smithi165 2024-05-11T13:27:54.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:54.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6834/4024086251,v1:172.21.15.165:6835/4024086251] up:boot 2024-05-11T13:27:54.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: fsmap 6 up:standby 2024-05-11T13:27:54.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:54 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:27:54.584 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.584 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: Deploying daemon mds.c on smithi165 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6834/4024086251,v1:172.21.15.165:6835/4024086251] up:boot 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: fsmap 6 up:standby 2024-05-11T13:27:54.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:54 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:27:54.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:54.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:54.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:54.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:54.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: Deploying daemon mds.c on smithi165 2024-05-11T13:27:54.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:54.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6834/4024086251,v1:172.21.15.165:6835/4024086251] up:boot 2024-05-11T13:27:54.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: fsmap 6 up:standby 2024-05-11T13:27:54.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:54 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:27:54.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.869+0000 7f5ec2e00700 1 -- 172.21.15.63:0/2461421012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 msgr2=0x7f5ebc10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:54.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.869+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/2461421012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc10bb40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009990 tx=0x7f5eac009ca0 comp rx=0 tx=0).stop 2024-05-11T13:27:54.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.869+0000 7f5ec2e00700 1 -- 172.21.15.63:0/2461421012 shutdown_connections 2024-05-11T13:27:54.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.869+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/2461421012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ebc106120 0x7f5ebc112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:54.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.869+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/2461421012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc10bb40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:54.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.869+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/2461421012 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ebc105760 0x7f5ebc105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:54.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.869+0000 7f5ec2e00700 1 -- 172.21.15.63:0/2461421012 >> 172.21.15.63:0/2461421012 conn(0x7f5ebc101250 msgr2=0x7f5ebc103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:54.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.870+0000 7f5ec2e00700 1 -- 172.21.15.63:0/2461421012 shutdown_connections 2024-05-11T13:27:54.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.870+0000 7f5ec2e00700 1 -- 172.21.15.63:0/2461421012 wait complete. 2024-05-11T13:27:54.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 Processor -- start 2024-05-11T13:27:54.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 -- start start 2024-05-11T13:27:54.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ebc105760 0x7f5ebc19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:54.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ebc106120 0x7f5ebc19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:54.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:54.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc115810 con 0x7f5ebc10b760 2024-05-11T13:27:54.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc115690 con 0x7f5ebc105760 2024-05-11T13:27:54.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.871+0000 7f5ec2e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc115990 con 0x7f5ebc106120 2024-05-11T13:27:54.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.872+0000 7f5ec0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:54.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.872+0000 7f5ec0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55664/0 (socket says 172.21.15.63:55664) 2024-05-11T13:27:54.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.872+0000 7f5ec0e00700 1 -- 172.21.15.63:0/3308515297 learned_addr learned my addr 172.21.15.63:0/3308515297 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:27:54.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.872+0000 7f5ebbe00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ebc105760 0x7f5ebc19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:54.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.872+0000 7f5ebb400700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ebc106120 0x7f5ebc19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:54.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.872+0000 7f5ec0e00700 1 -- 172.21.15.63:0/3308515297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ebc106120 msgr2=0x7f5ebc19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:54.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5ec0e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ebc106120 0x7f5ebc19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5ec0e00700 1 -- 172.21.15.63:0/3308515297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ebc105760 msgr2=0x7f5ebc19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5ec0e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ebc105760 0x7f5ebc19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5ec0e00700 1 -- 172.21.15.63:0/3308515297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc1a0d50 con 0x7f5ebc10b760 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5ec0e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc1a0610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4007ee0 tx=0x7f5eb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5eb8c00700 1 -- 172.21.15.63:0/3308515297 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4004bb0 con 0x7f5ebc10b760 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc1a1040 con 0x7f5ebc10b760 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5eb8c00700 1 -- 172.21.15.63:0/3308515297 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5eb4004d50 con 0x7f5ebc10b760 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5eb8c00700 1 -- 172.21.15.63:0/3308515297 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40058b0 con 0x7f5ebc10b760 2024-05-11T13:27:54.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.873+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc1a88c0 con 0x7f5ebc10b760 2024-05-11T13:27:54.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.875+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc06a240 con 0x7f5ebc10b760 2024-05-11T13:27:54.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.877+0000 7f5eb8c00700 1 -- 172.21.15.63:0/3308515297 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5eb401a020 con 0x7f5ebc10b760 2024-05-11T13:27:54.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.879+0000 7f5eb8c00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5ea00b2880 0x7f5ea00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:27:54.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.879+0000 7f5eb8c00700 1 -- 172.21.15.63:0/3308515297 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f5eb40e0ee0 con 0x7f5ebc10b760 2024-05-11T13:27:54.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.879+0000 7f5ebbe00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5ea00b2880 0x7f5ea00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:27:54.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.880+0000 7f5ebbe00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5ea00b2880 0x7f5ea00b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00b5c0 tx=0x7f5eac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:27:54.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:54.883+0000 7f5eb8c00700 1 -- 172.21.15.63:0/3308515297 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5eb4010070 con 0x7f5ebc10b760 2024-05-11T13:27:55.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.170+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ebc02e570 con 0x7f5ebc10b760 2024-05-11T13:27:55.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.171+0000 7f5eb8c00700 1 -- 172.21.15.63:0/3308515297 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f5eb40a9000 con 0x7f5ebc10b760 2024-05-11T13:27:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:27:55.171 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-11T13:27:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5ea00b2880 msgr2=0x7f5ea00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5ea00b2880 0x7f5ea00b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00b5c0 tx=0x7f5eac005960 comp rx=0 tx=0).stop 2024-05-11T13:27:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 msgr2=0x7f5ebc1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:27:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc1a0610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4007ee0 tx=0x7f5eb400d3b0 comp rx=0 tx=0).stop 2024-05-11T13:27:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 shutdown_connections 2024-05-11T13:27:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5ea00b2880 0x7f5ea00b4d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ebc10b760 0x7f5ebc1a0610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ebc105760 0x7f5ebc19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 --2- 172.21.15.63:0/3308515297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ebc106120 0x7f5ebc19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:27:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.174+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 >> 172.21.15.63:0/3308515297 conn(0x7f5ebc101250 msgr2=0x7f5ebc101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:27:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.175+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 shutdown_connections 2024-05-11T13:27:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:27:55.175+0000 7f5ec2e00700 1 -- 172.21.15.63:0/3308515297 wait complete. 2024-05-11T13:27:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 7 2024-05-11T13:27:55.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:55 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3308515297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:55.657 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:55 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3308515297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:55.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:55 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3308515297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:27:56.485 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:56 smithi165 ceph-mon[33235]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:56.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:56 smithi063 ceph-mon[29809]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:56.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:56 smithi071 ceph-mon[33293]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:56.816 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:56 smithi165 systemd[1]: Starting Ceph mds.c for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:27:57.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:56 smithi165 podman[63165]: 2024-05-11 13:27:56.749751247 +0000 UTC m=+0.035225597 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:27:57.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:56 smithi165 podman[63165]: 2024-05-11 13:27:56.874343784 +0000 UTC m=+0.159818122 container create 84d4b42d326ef755efadf6358a9221e3bc8cc195237a3a78d02d0faeea7f8fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-11T13:27:57.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:57 smithi165 podman[63165]: 2024-05-11 13:27:57.136703055 +0000 UTC m=+0.422177424 container init 84d4b42d326ef755efadf6358a9221e3bc8cc195237a3a78d02d0faeea7f8fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-11T13:27:57.566 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:57 smithi165 podman[63165]: 2024-05-11 13:27:57.146832381 +0000 UTC m=+0.432306716 container start 84d4b42d326ef755efadf6358a9221e3bc8cc195237a3a78d02d0faeea7f8fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-11T13:27:57.567 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:57 smithi165 ceph-mds[63185]: -- 172.21.15.165:0/3618933165 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc05f4da40 con 0x55bc05f20c00 2024-05-11T13:27:57.567 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c[63181]: starting mds.c at 2024-05-11T13:27:57.567 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:57 smithi165 bash[63165]: 84d4b42d326ef755efadf6358a9221e3bc8cc195237a3a78d02d0faeea7f8fcb 2024-05-11T13:27:57.567 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:27:57 smithi165 systemd[1]: Started Ceph mds.c for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:27:58.056 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:27:58.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: Deploying daemon mds.h on smithi071 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6836/2308228126,v1:172.21.15.165:6837/2308228126] up:boot 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: fsmap 7 up:standby 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:27:58.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:27:58 smithi063 ceph-mon[29809]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:58.530 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:27:58.628 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.628 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.628 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.628 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:58.628 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:58.628 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:58.628 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: Deploying daemon mds.h on smithi071 2024-05-11T13:27:58.629 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6836/2308228126,v1:172.21.15.165:6837/2308228126] up:boot 2024-05-11T13:27:58.629 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: fsmap 7 up:standby 2024-05-11T13:27:58.629 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:27:58.629 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:27:58 smithi071 ceph-mon[33293]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:27:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: Deploying daemon mds.h on smithi071 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6836/2308228126,v1:172.21.15.165:6837/2308228126] up:boot 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: fsmap 7 up:standby 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:27:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:27:58 smithi165 ceph-mon[33235]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:00.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:00 smithi071 ceph-mon[33293]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.710+0000 7f973ae00700 1 -- 172.21.15.63:0/1563984051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 msgr2=0x7f9734112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.710+0000 7f973ae00700 1 --2- 172.21.15.63:0/1563984051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f9734112640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f972c00b0a0 tx=0x7f972c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.710+0000 7f973ae00700 1 -- 172.21.15.63:0/1563984051 shutdown_connections 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.710+0000 7f973ae00700 1 --2- 172.21.15.63:0/1563984051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f9734112640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.710+0000 7f973ae00700 1 --2- 172.21.15.63:0/1563984051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f973410b770 0x7f973410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.710+0000 7f973ae00700 1 --2- 172.21.15.63:0/1563984051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9734105770 0x7f9734105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.710+0000 7f973ae00700 1 -- 172.21.15.63:0/1563984051 >> 172.21.15.63:0/1563984051 conn(0x7f9734101260 msgr2=0x7f9734103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.711+0000 7f973ae00700 1 -- 172.21.15.63:0/1563984051 shutdown_connections 2024-05-11T13:28:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.711+0000 7f973ae00700 1 -- 172.21.15.63:0/1563984051 wait complete. 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 Processor -- start 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 -- start start 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9734105770 0x7f973419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f973419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f973410b770 0x7f97341a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9734115870 con 0x7f9734106130 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97341156f0 con 0x7f9734105770 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f973ae00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97341159f0 con 0x7f973410b770 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.712+0000 7f972be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f973419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f973419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53118/0 (socket says 172.21.15.63:53118) 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 -- 172.21.15.63:0/1067154019 learned_addr learned my addr 172.21.15.63:0/1067154019 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f9733e00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9734105770 0x7f973419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:00.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f9738e00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f973410b770 0x7f97341a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:00.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 -- 172.21.15.63:0/1067154019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f973410b770 msgr2=0x7f97341a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f973410b770 0x7f97341a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 -- 172.21.15.63:0/1067154019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9734105770 msgr2=0x7f973419bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9734105770 0x7f973419bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 -- 172.21.15.63:0/1067154019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97341a0d50 con 0x7f9734106130 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f9738e00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f973410b770 0x7f97341a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.713+0000 7f972be00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f973419c280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f971c00d950 tx=0x7f971c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.714+0000 7f9731600700 1 -- 172.21.15.63:0/1067154019 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c016070 con 0x7f9734106130 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.714+0000 7f9731600700 1 -- 172.21.15.63:0/1067154019 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f971c010460 con 0x7f9734106130 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.714+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97341a1040 con 0x7f9734106130 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.714+0000 7f9731600700 1 -- 172.21.15.63:0/1067154019 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c00f620 con 0x7f9734106130 2024-05-11T13:28:00.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.714+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97341a88c0 con 0x7f9734106130 2024-05-11T13:28:00.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.715+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973406a240 con 0x7f9734106130 2024-05-11T13:28:00.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.718+0000 7f9731600700 1 -- 172.21.15.63:0/1067154019 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f971c019070 con 0x7f9734106130 2024-05-11T13:28:00.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.719+0000 7f9731600700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f97180b2900 0x7f97180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:00.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.719+0000 7f9731600700 1 -- 172.21.15.63:0/1067154019 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f971c0e19d0 con 0x7f9734106130 2024-05-11T13:28:00.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.720+0000 7f9733e00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f97180b2900 0x7f97180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:00.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.720+0000 7f9733e00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f97180b2900 0x7f97180b4dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f972400bbb0 tx=0x7f972400b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:00.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:00.722+0000 7f9731600700 1 -- 172.21.15.63:0/1067154019 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f971c0a9af0 con 0x7f9734106130 2024-05-11T13:28:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:00 smithi063 ceph-mon[29809]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:01.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.009+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f973402e570 con 0x7f9734106130 2024-05-11T13:28:01.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.011+0000 7f9731600700 1 -- 172.21.15.63:0/1067154019 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f971c0a93f0 con 0x7f9734106130 2024-05-11T13:28:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:28:01.013 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-11T13:28:01.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f97180b2900 msgr2=0x7f97180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:01.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f97180b2900 0x7f97180b4dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f972400bbb0 tx=0x7f972400b590 comp rx=0 tx=0).stop 2024-05-11T13:28:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 msgr2=0x7f973419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f973419c280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f971c00d950 tx=0x7f971c00dd10 comp rx=0 tx=0).stop 2024-05-11T13:28:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 shutdown_connections 2024-05-11T13:28:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f97180b2900 0x7f97180b4dc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f973410b770 0x7f97341a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9734105770 0x7f973419bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 --2- 172.21.15.63:0/1067154019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9734106130 0x7f973419c280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.014+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 >> 172.21.15.63:0/1067154019 conn(0x7f9734101260 msgr2=0x7f9734101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.015+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 shutdown_connections 2024-05-11T13:28:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:01.015+0000 7f973ae00700 1 -- 172.21.15.63:0/1067154019 wait complete. 2024-05-11T13:28:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 8 2024-05-11T13:28:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:00 smithi165 ceph-mon[33235]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:01.385 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:28:01 smithi071 podman[62836]: 2024-05-11 13:28:01.381874954 +0000 UTC m=+0.407895219 container init 45d2b6c5d1c8db39ad164a0a4064f9101851bd55c8e7c5db7731dc7933024f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-05-11T13:28:01.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:01 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1067154019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:01.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:01 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1067154019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:01.704 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:28:01 smithi071 podman[62836]: 2024-05-11 13:28:01.392925586 +0000 UTC m=+0.418945836 container start 45d2b6c5d1c8db39ad164a0a4064f9101851bd55c8e7c5db7731dc7933024f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T13:28:01.704 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:28:01 smithi071 ceph-mds[62862]: -- 172.21.15.71:0/2784176059 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55983da85a40 con 0x55983da58c00 2024-05-11T13:28:01.704 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:28:01 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h[62856]: starting mds.h at 2024-05-11T13:28:01.704 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:28:01 smithi071 bash[62836]: 45d2b6c5d1c8db39ad164a0a4064f9101851bd55c8e7c5db7731dc7933024f0b 2024-05-11T13:28:01.704 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:28:01 smithi071 systemd[1]: Started Ceph mds.h for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:28:02.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:01 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1067154019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] up:boot 2024-05-11T13:28:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: fsmap 8 up:standby 2024-05-11T13:28:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:28:02.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:02.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:02.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:02.936 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:02 smithi063 ceph-mon[29809]: Deploying daemon mds.i on smithi165 2024-05-11T13:28:02.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:02.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] up:boot 2024-05-11T13:28:02.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: fsmap 8 up:standby 2024-05-11T13:28:02.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:28:02.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:02.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:02.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:02.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:02 smithi071 ceph-mon[33293]: Deploying daemon mds.i on smithi165 2024-05-11T13:28:02.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:02.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] up:boot 2024-05-11T13:28:02.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: fsmap 8 up:standby 2024-05-11T13:28:02.957 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:28:02.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:02.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:02.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:02.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:02.958 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:02 smithi165 ceph-mon[33235]: Deploying daemon mds.i on smithi165 2024-05-11T13:28:03.865 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:28:04.328 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:04.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:04 smithi063 ceph-mon[29809]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:04 smithi071 ceph-mon[33293]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:04.976 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:04 smithi165 ceph-mon[33235]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:05.317 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:04 smithi165 systemd[1]: Starting Ceph mds.i for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:28:05.627 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 podman[63470]: 2024-05-11 13:28:05.232208686 +0000 UTC m=+0.032891954 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:28:05.627 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 podman[63470]: 2024-05-11 13:28:05.354336922 +0000 UTC m=+0.155020221 container create 7dfe9bca856ce347bc92c6c1091a5e2c33b4464c1f6b097fc05fc8103a7caa22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-11T13:28:05.627 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 podman[63470]: 2024-05-11 13:28:05.624991522 +0000 UTC m=+0.425674850 container init 7dfe9bca856ce347bc92c6c1091a5e2c33b4464c1f6b097fc05fc8103a7caa22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T13:28:06.066 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 podman[63470]: 2024-05-11 13:28:05.635810973 +0000 UTC m=+0.436494278 container start 7dfe9bca856ce347bc92c6c1091a5e2c33b4464c1f6b097fc05fc8103a7caa22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-11T13:28:06.066 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 ceph-mds[63489]: -- 172.21.15.165:0/246385262 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556aaf63ba40 con 0x556aaf60f000 2024-05-11T13:28:06.066 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i[63485]: starting mds.i at 2024-05-11T13:28:06.066 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 bash[63470]: 7dfe9bca856ce347bc92c6c1091a5e2c33b4464c1f6b097fc05fc8103a7caa22 2024-05-11T13:28:06.067 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:28:05 smithi165 systemd[1]: Started Ceph mds.i for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:28:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 -- 172.21.15.63:0/90134384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8284075780 msgr2=0x7f8284075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 --2- 172.21.15.63:0/90134384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8284075780 0x7f8284075b60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8274009990 tx=0x7f8274009ca0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 -- 172.21.15.63:0/90134384 shutdown_connections 2024-05-11T13:28:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 --2- 172.21.15.63:0/90134384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f828410ff50 0x7f8284112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 --2- 172.21.15.63:0/90134384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8284075780 0x7f8284075b60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 --2- 172.21.15.63:0/90134384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82840760a0 0x7f828410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 -- 172.21.15.63:0/90134384 >> 172.21.15.63:0/90134384 conn(0x7f828407a6b0 msgr2=0x7f828407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:06.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 -- 172.21.15.63:0/90134384 shutdown_connections 2024-05-11T13:28:06.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.587+0000 7f8289200700 1 -- 172.21.15.63:0/90134384 wait complete. 2024-05-11T13:28:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 Processor -- start 2024-05-11T13:28:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 -- start start 2024-05-11T13:28:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8284075780 0x7f828419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82840760a0 0x7f828419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828410ff50 0x7f82841a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8284115800 con 0x7f828410ff50 2024-05-11T13:28:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8284115680 con 0x7f82840760a0 2024-05-11T13:28:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.589+0000 7f8289200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8284115980 con 0x7f8284075780 2024-05-11T13:28:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828410ff50 0x7f82841a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828410ff50 0x7f82841a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53146/0 (socket says 172.21.15.63:53146) 2024-05-11T13:28:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 -- 172.21.15.63:0/927402095 learned_addr learned my addr 172.21.15.63:0/927402095 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8281e00700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8284075780 0x7f828419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8281400700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82840760a0 0x7f828419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 -- 172.21.15.63:0/927402095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8284075780 msgr2=0x7f828419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8284075780 0x7f828419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 -- 172.21.15.63:0/927402095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82840760a0 msgr2=0x7f828419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82840760a0 0x7f828419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.590+0000 7f8282800700 1 -- 172.21.15.63:0/927402095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82841a0dc0 con 0x7f828410ff50 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.591+0000 7f8282800700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828410ff50 0x7f82841a0680 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f827c007d40 tx=0x7f827c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.591+0000 7f827aa00700 1 -- 172.21.15.63:0/927402095 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c016020 con 0x7f828410ff50 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.591+0000 7f827aa00700 1 -- 172.21.15.63:0/927402095 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f827c00ac90 con 0x7f828410ff50 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.591+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82841a10b0 con 0x7f828410ff50 2024-05-11T13:28:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.591+0000 7f827aa00700 1 -- 172.21.15.63:0/927402095 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0113a0 con 0x7f828410ff50 2024-05-11T13:28:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.591+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82841a8930 con 0x7f828410ff50 2024-05-11T13:28:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.593+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f828406a240 con 0x7f828410ff50 2024-05-11T13:28:06.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.595+0000 7f827aa00700 1 -- 172.21.15.63:0/927402095 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f827c019020 con 0x7f828410ff50 2024-05-11T13:28:06.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.598+0000 7f827aa00700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82680b2900 0x7f82680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:06.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.598+0000 7f827aa00700 1 -- 172.21.15.63:0/927402095 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f827c0e2080 con 0x7f828410ff50 2024-05-11T13:28:06.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.599+0000 7f8281e00700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82680b2900 0x7f82680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:06.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.599+0000 7f8281e00700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82680b2900 0x7f82680b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8274003760 tx=0x7f8274005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:06.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.601+0000 7f827aa00700 1 -- 172.21.15.63:0/927402095 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f827c020020 con 0x7f828410ff50 2024-05-11T13:28:06.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.905+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f828402e570 con 0x7f828410ff50 2024-05-11T13:28:06.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.906+0000 7f827aa00700 1 -- 172.21.15.63:0/927402095 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f827c0aa1a0 con 0x7f828410ff50 2024-05-11T13:28:06.908 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:28:06.908 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82680b2900 msgr2=0x7f82680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82680b2900 0x7f82680b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8274003760 tx=0x7f8274005fd0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828410ff50 msgr2=0x7f82841a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828410ff50 0x7f82841a0680 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f827c007d40 tx=0x7f827c00a340 comp rx=0 tx=0).stop 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 shutdown_connections 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f82680b2900 0x7f82680b4dc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828410ff50 0x7f82841a0680 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8284075780 0x7f828419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 --2- 172.21.15.63:0/927402095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82840760a0 0x7f828419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:06.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.909+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 >> 172.21.15.63:0/927402095 conn(0x7f828407a6b0 msgr2=0x7f8284101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:06.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.910+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 shutdown_connections 2024-05-11T13:28:06.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:06.910+0000 7f8289200700 1 -- 172.21.15.63:0/927402095 wait complete. 2024-05-11T13:28:06.912 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 10 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: Deploying daemon mds.g on smithi063 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] up:boot 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: fsmap 9 up:standby 2024-05-11T13:28:07.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:06 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:28:07.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:07.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:07.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:07.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:07.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: Deploying daemon mds.g on smithi063 2024-05-11T13:28:07.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] up:boot 2024-05-11T13:28:07.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: fsmap 9 up:standby 2024-05-11T13:28:07.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:06 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:28:07.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:07.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: Deploying daemon mds.g on smithi063 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] up:boot 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: fsmap 9 up:standby 2024-05-11T13:28:07.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:06 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:28:08.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:07 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/927402095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:08.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:07 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/927402095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:08.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:07 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/927402095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:09.091 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:08 smithi063 ceph-mon[29809]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:09.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:08 smithi071 ceph-mon[33293]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:08 smithi165 ceph-mon[33235]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:09.435 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 systemd[1]: Starting Ceph mds.g for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:28:09.829 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 podman[68942]: 2024-05-11 13:28:09.417791659 +0000 UTC m=+0.034459936 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:28:09.829 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 podman[68942]: 2024-05-11 13:28:09.540220089 +0000 UTC m=+0.156888403 container create dda168a2ca7950a53174d4db6f5fa87a7aa1b57c5fb4d613551ae71e1693fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-11T13:28:09.829 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 podman[68942]: 2024-05-11 13:28:09.827719963 +0000 UTC m=+0.444388284 container init dda168a2ca7950a53174d4db6f5fa87a7aa1b57c5fb4d613551ae71e1693fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:28:10.184 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 podman[68942]: 2024-05-11 13:28:09.839066234 +0000 UTC m=+0.455734523 container start dda168a2ca7950a53174d4db6f5fa87a7aa1b57c5fb4d613551ae71e1693fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-11T13:28:10.184 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 ceph-mds[68961]: -- 172.21.15.63:0/160853343 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610746b1a40 con 0x561074685000 2024-05-11T13:28:10.185 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g[68957]: starting mds.g at 2024-05-11T13:28:10.185 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 bash[68942]: dda168a2ca7950a53174d4db6f5fa87a7aa1b57c5fb4d613551ae71e1693fe61 2024-05-11T13:28:10.185 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:28:09 smithi063 systemd[1]: Started Ceph mds.g for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:28:10.892 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:28:11.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:10 smithi063 ceph-mon[29809]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:11.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:11.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:11.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:10 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:11.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:10 smithi071 ceph-mon[33293]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:11.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:11.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:11.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:10 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:10 smithi165 ceph-mon[33235]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:11.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:11.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:10 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:12.124 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:11 smithi063 ceph-mon[29809]: Deploying daemon mds.a on smithi063 2024-05-11T13:28:12.124 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:11 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.63:6838/3189418422,v1:172.21.15.63:6839/3189418422] up:boot 2024-05-11T13:28:12.124 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:11 smithi063 ceph-mon[29809]: fsmap 10 up:standby 2024-05-11T13:28:12.124 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:11 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:28:12.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:11 smithi071 ceph-mon[33293]: Deploying daemon mds.a on smithi063 2024-05-11T13:28:12.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:11 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.63:6838/3189418422,v1:172.21.15.63:6839/3189418422] up:boot 2024-05-11T13:28:12.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:11 smithi071 ceph-mon[33293]: fsmap 10 up:standby 2024-05-11T13:28:12.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:11 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:28:12.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:11 smithi165 ceph-mon[33235]: Deploying daemon mds.a on smithi063 2024-05-11T13:28:12.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:11 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.63:6838/3189418422,v1:172.21.15.63:6839/3189418422] up:boot 2024-05-11T13:28:12.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:11 smithi165 ceph-mon[33235]: fsmap 10 up:standby 2024-05-11T13:28:12.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:11 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:28:12.596 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:13.174 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:12 smithi063 ceph-mon[29809]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:13.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:12 smithi071 ceph-mon[33293]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:13.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:12 smithi165 ceph-mon[33235]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:13.436 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:13 smithi063 systemd[1]: Starting Ceph mds.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:28:14.353 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 podman[69345]: 2024-05-11 13:28:14.187956948 +0000 UTC m=+0.053316404 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:28:14.353 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 podman[69345]: 2024-05-11 13:28:14.352050792 +0000 UTC m=+0.217410310 container create 8e87e1d62b62dff0374d1b53d5baf74bcb60ff09f3f46c6dc05df1f71c45e941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-11T13:28:14.915 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 podman[69345]: 2024-05-11 13:28:14.734512965 +0000 UTC m=+0.599872460 container init 8e87e1d62b62dff0374d1b53d5baf74bcb60ff09f3f46c6dc05df1f71c45e941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-11T13:28:14.915 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 podman[69345]: 2024-05-11 13:28:14.746724311 +0000 UTC m=+0.612083782 container start 8e87e1d62b62dff0374d1b53d5baf74bcb60ff09f3f46c6dc05df1f71c45e941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T13:28:14.915 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 ceph-mds[69383]: -- 172.21.15.63:0/2808501596 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56199d9e9a40 con 0x56199d9bd000 2024-05-11T13:28:14.915 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a[69379]: starting mds.a at 2024-05-11T13:28:14.915 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 bash[69345]: 8e87e1d62b62dff0374d1b53d5baf74bcb60ff09f3f46c6dc05df1f71c45e941 2024-05-11T13:28:14.915 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:28:14 smithi063 systemd[1]: Started Ceph mds.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:28:15.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:14 smithi063 ceph-mon[29809]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:15.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:14 smithi071 ceph-mon[33293]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:15.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:14 smithi165 ceph-mon[33235]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.370+0000 7f29faa00700 1 -- 172.21.15.63:0/1382809260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4103490 msgr2=0x7f29f4103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.370+0000 7f29faa00700 1 --2- 172.21.15.63:0/1382809260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4103490 0x7f29f4103910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f29e00099c0 tx=0x7f29e0009cd0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.370+0000 7f29faa00700 1 -- 172.21.15.63:0/1382809260 shutdown_connections 2024-05-11T13:28:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.370+0000 7f29faa00700 1 --2- 172.21.15.63:0/1382809260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f4103e50 0x7f29f4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.370+0000 7f29faa00700 1 --2- 172.21.15.63:0/1382809260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4109540 0x7f29f4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.371+0000 7f29faa00700 1 --2- 172.21.15.63:0/1382809260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4103490 0x7f29f4103910 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.371+0000 7f29faa00700 1 -- 172.21.15.63:0/1382809260 >> 172.21.15.63:0/1382809260 conn(0x7f29f40fef60 msgr2=0x7f29f4101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.371+0000 7f29faa00700 1 -- 172.21.15.63:0/1382809260 shutdown_connections 2024-05-11T13:28:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.371+0000 7f29faa00700 1 -- 172.21.15.63:0/1382809260 wait complete. 2024-05-11T13:28:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 Processor -- start 2024-05-11T13:28:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 -- start start 2024-05-11T13:28:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4103490 0x7f29f419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f4103e50 0x7f29f419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4109540 0x7f29f4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29f4113550 con 0x7f29f4109540 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41133d0 con 0x7f29f4103490 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.372+0000 7f29faa00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41136d0 con 0x7f29f4103e50 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4109540 0x7f29f4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4109540 0x7f29f4199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38036/0 (socket says 172.21.15.63:38036) 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 -- 172.21.15.63:0/4038390792 learned_addr learned my addr 172.21.15.63:0/4038390792 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29f9600700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4103490 0x7f29f419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29f8c00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f4103e50 0x7f29f419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 -- 172.21.15.63:0/4038390792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f4103e50 msgr2=0x7f29f419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f4103e50 0x7f29f419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 -- 172.21.15.63:0/4038390792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4103490 msgr2=0x7f29f419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4103490 0x7f29f419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 -- 172.21.15.63:0/4038390792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f419a050 con 0x7f29f4109540 2024-05-11T13:28:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29f8c00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f4103e50 0x7f29f419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:28:15.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.373+0000 7f29fa000700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4109540 0x7f29f4199770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29ec007ee0 tx=0x7f29ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:15.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.374+0000 7f29f2000700 1 -- 172.21.15.63:0/4038390792 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00dbd0 con 0x7f29f4109540 2024-05-11T13:28:15.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.374+0000 7f29f2000700 1 -- 172.21.15.63:0/4038390792 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29ec00dd70 con 0x7f29f4109540 2024-05-11T13:28:15.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.374+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f419a340 con 0x7f29f4109540 2024-05-11T13:28:15.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.374+0000 7f29f2000700 1 -- 172.21.15.63:0/4038390792 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0058d0 con 0x7f29f4109540 2024-05-11T13:28:15.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.374+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f41a65d0 con 0x7f29f4109540 2024-05-11T13:28:15.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.375+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f4104bb0 con 0x7f29f4109540 2024-05-11T13:28:15.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.377+0000 7f29f2000700 1 -- 172.21.15.63:0/4038390792 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f29ec01a020 con 0x7f29f4109540 2024-05-11T13:28:15.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.379+0000 7f29f2000700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f29dc0b2880 0x7f29dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:15.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.379+0000 7f29f2000700 1 -- 172.21.15.63:0/4038390792 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f29ec0e1c50 con 0x7f29f4109540 2024-05-11T13:28:15.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.379+0000 7f29f9600700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f29dc0b2880 0x7f29dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.380+0000 7f29f9600700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f29dc0b2880 0x7f29dc0b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f29e800ac50 tx=0x7f29e800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:15.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.382+0000 7f29f2000700 1 -- 172.21.15.63:0/4038390792 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29ec010070 con 0x7f29f4109540 2024-05-11T13:28:15.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.669+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29f419ac50 con 0x7f29f4109540 2024-05-11T13:28:15.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.671+0000 7f29f2000700 1 -- 172.21.15.63:0/4038390792 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f29ec0a9d70 con 0x7f29f4109540 2024-05-11T13:28:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:28:15.673 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-11T13:28:15.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.674+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f29dc0b2880 msgr2=0x7f29dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:15.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.674+0000 7f29faa00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f29dc0b2880 0x7f29dc0b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f29e800ac50 tx=0x7f29e800a380 comp rx=0 tx=0).stop 2024-05-11T13:28:15.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.674+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4109540 msgr2=0x7f29f4199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:15.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.674+0000 7f29faa00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4109540 0x7f29f4199770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29ec007ee0 tx=0x7f29ec00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 shutdown_connections 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f29dc0b2880 0x7f29dc0b4d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29f4109540 0x7f29f4199770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4103490 0x7f29f419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 --2- 172.21.15.63:0/4038390792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f4103e50 0x7f29f419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 >> 172.21.15.63:0/4038390792 conn(0x7f29f40fef60 msgr2=0x7f29f4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 shutdown_connections 2024-05-11T13:28:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:15.675+0000 7f29faa00700 1 -- 172.21.15.63:0/4038390792 wait complete. 2024-05-11T13:28:15.678 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 12 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.63:6840/3763187067,v1:172.21.15.63:6841/3763187067] up:boot 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: fsmap 11 up:standby 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:15.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:15 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4038390792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.63:6840/3763187067,v1:172.21.15.63:6841/3763187067] up:boot 2024-05-11T13:28:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: fsmap 11 up:standby 2024-05-11T13:28:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:28:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:16.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:16.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:15 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4038390792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.63:6840/3763187067,v1:172.21.15.63:6841/3763187067] up:boot 2024-05-11T13:28:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: fsmap 11 up:standby 2024-05-11T13:28:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:28:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T13:28:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T13:28:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:16.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:15 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4038390792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:17.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:16 smithi063 ceph-mon[29809]: Deploying daemon mds.b on smithi071 2024-05-11T13:28:17.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:16 smithi063 ceph-mon[29809]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:17.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:16 smithi071 ceph-mon[33293]: Deploying daemon mds.b on smithi071 2024-05-11T13:28:17.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:16 smithi071 ceph-mon[33293]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:17.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:16 smithi165 ceph-mon[33235]: Deploying daemon mds.b on smithi071 2024-05-11T13:28:17.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:16 smithi165 ceph-mon[33235]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:18.537 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:28:18.624 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:18 smithi071 systemd[1]: Starting Ceph mds.b for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:28:18.926 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:18 smithi071 podman[63148]: 2024-05-11 13:28:18.537951423 +0000 UTC m=+0.034190669 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:28:18.926 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:18 smithi071 podman[63148]: 2024-05-11 13:28:18.650188481 +0000 UTC m=+0.146427790 container create 29a345b07e8dcea555e59b26d4efc3d9d8875da459df6600ba4b9890424e0df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-11T13:28:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:19.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:18 smithi063 ceph-mon[29809]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:19.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:18 smithi071 ceph-mon[33293]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:19.204 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:18 smithi071 podman[63148]: 2024-05-11 13:28:18.984844231 +0000 UTC m=+0.481083503 container init 29a345b07e8dcea555e59b26d4efc3d9d8875da459df6600ba4b9890424e0df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-11T13:28:19.204 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:18 smithi071 podman[63148]: 2024-05-11 13:28:18.992870653 +0000 UTC m=+0.489109901 container start 29a345b07e8dcea555e59b26d4efc3d9d8875da459df6600ba4b9890424e0df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-11T13:28:19.205 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:19 smithi071 ceph-mds[63175]: -- 172.21.15.71:0/403859706 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b204ab1a40 con 0x55b204a84800 2024-05-11T13:28:19.205 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b[63166]: starting mds.b at 2024-05-11T13:28:19.205 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:19 smithi071 bash[63148]: 29a345b07e8dcea555e59b26d4efc3d9d8875da459df6600ba4b9890424e0df4 2024-05-11T13:28:19.205 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:19 smithi071 systemd[1]: Started Ceph mds.b for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T13:28:19.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:18 smithi165 ceph-mon[33235]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] up:boot 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: fsmap 12 up:standby 2024-05-11T13:28:20.358 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:20 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:28:20.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:20.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:28:20.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] up:boot 2024-05-11T13:28:20.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: fsmap 12 up:standby 2024-05-11T13:28:20.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:20 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:28:20.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] up:boot 2024-05-11T13:28:20.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: fsmap 12 up:standby 2024-05-11T13:28:20.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:20 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:28:21.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 -- 172.21.15.63:0/2075522586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 msgr2=0x7f8f101099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 --2- 172.21.15.63:0/2075522586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 0x7f8f101099c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000b0a0 tx=0x7f8f0000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 -- 172.21.15.63:0/2075522586 shutdown_connections 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 --2- 172.21.15.63:0/2075522586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1006c5e0 0x7f8f10110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 --2- 172.21.15.63:0/2075522586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 0x7f8f101099c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 --2- 172.21.15.63:0/2075522586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f1006bc20 0x7f8f1006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 -- 172.21.15.63:0/2075522586 >> 172.21.15.63:0/2075522586 conn(0x7f8f10079400 msgr2=0x7f8f10079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 -- 172.21.15.63:0/2075522586 shutdown_connections 2024-05-11T13:28:21.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.168+0000 7f8f17600700 1 -- 172.21.15.63:0/2075522586 wait complete. 2024-05-11T13:28:21.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.169+0000 7f8f17600700 1 Processor -- start 2024-05-11T13:28:21.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.169+0000 7f8f17600700 1 -- start start 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f17600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1006bc20 0x7f8f10102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f17600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f1006c5e0 0x7f8f10102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f17600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 0x7f8f10103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f17600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101136a0 con 0x7f8f101095e0 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f17600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10113520 con 0x7f8f1006bc20 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f17600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10113820 con 0x7f8f1006c5e0 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f15600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 0x7f8f10103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f07e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f1006c5e0 0x7f8f10102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f14c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1006bc20 0x7f8f10102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f14c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1006bc20 0x7f8f10102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:60760/0 (socket says 172.21.15.63:60760) 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f07e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f1006c5e0 0x7f8f10102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:40070/0 (socket says 172.21.15.63:40070) 2024-05-11T13:28:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.170+0000 7f8f14c00700 1 -- 172.21.15.63:0/1731704864 learned_addr learned my addr 172.21.15.63:0/1731704864 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:21.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f15600700 1 -- 172.21.15.63:0/1731704864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f1006c5e0 msgr2=0x7f8f10102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:21.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f15600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f1006c5e0 0x7f8f10102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f15600700 1 -- 172.21.15.63:0/1731704864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1006bc20 msgr2=0x7f8f10102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f15600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1006bc20 0x7f8f10102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f15600700 1 -- 172.21.15.63:0/1731704864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f101a81e0 con 0x7f8f101095e0 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f15600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 0x7f8f10103380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800b3f0 tx=0x7f8f0800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f0e000700 1 -- 172.21.15.63:0/1731704864 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08015020 con 0x7f8f101095e0 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f0e000700 1 -- 172.21.15.63:0/1731704864 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f08011ac0 con 0x7f8f101095e0 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f0e000700 1 -- 172.21.15.63:0/1731704864 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08010ac0 con 0x7f8f101095e0 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.171+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101a8470 con 0x7f8f101095e0 2024-05-11T13:28:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.172+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101a8960 con 0x7f8f101095e0 2024-05-11T13:28:21.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.173+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1010bac0 con 0x7f8f101095e0 2024-05-11T13:28:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.175+0000 7f8f0e000700 1 -- 172.21.15.63:0/1731704864 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8f08018020 con 0x7f8f101095e0 2024-05-11T13:28:21.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.177+0000 7f8f0e000700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ef40b2880 0x7f8ef40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:21.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.177+0000 7f8f0e000700 1 -- 172.21.15.63:0/1731704864 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8f080e13e0 con 0x7f8f101095e0 2024-05-11T13:28:21.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.177+0000 7f8f14c00700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ef40b2880 0x7f8ef40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.178+0000 7f8f14c00700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ef40b2880 0x7f8ef40b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000b830 tx=0x7f8f00009de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.181+0000 7f8f0e000700 1 -- 172.21.15.63:0/1731704864 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f0801f020 con 0x7f8f101095e0 2024-05-11T13:28:21.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.466+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f10103e30 con 0x7f8f101095e0 2024-05-11T13:28:21.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.467+0000 7f8f0e000700 1 -- 172.21.15.63:0/1731704864 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f8f080a9500 con 0x7f8f101095e0 2024-05-11T13:28:21.468 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:28:21.468 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-11T13:28:21.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.469+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ef40b2880 msgr2=0x7f8ef40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:21.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.469+0000 7f8f17600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ef40b2880 0x7f8ef40b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000b830 tx=0x7f8f00009de0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.469+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 msgr2=0x7f8f10103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:21.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.469+0000 7f8f17600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 0x7f8f10103380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800b3f0 tx=0x7f8f0800b700 comp rx=0 tx=0).stop 2024-05-11T13:28:21.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 shutdown_connections 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ef40b2880 0x7f8ef40b4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f101095e0 0x7f8f10103380 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1006bc20 0x7f8f10102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 --2- 172.21.15.63:0/1731704864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f1006c5e0 0x7f8f10102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 >> 172.21.15.63:0/1731704864 conn(0x7f8f10079400 msgr2=0x7f8f10101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 shutdown_connections 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:21.470+0000 7f8f17600700 1 -- 172.21.15.63:0/1731704864 wait complete. 2024-05-11T13:28:21.472 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 13 2024-05-11T13:28:21.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:21 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1731704864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:21.898 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:21 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1731704864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:21.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:21 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1731704864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:28:22.325 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-11T13:28:22.803 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:22.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:22 smithi165 ceph-mon[33235]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:22.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:22.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:22 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:22 smithi063 ceph-mon[29809]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:22 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:22.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:22 smithi071 ceph-mon[33293]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:22.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:22.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:22 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:23.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:23 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:23.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:23 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:23 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:23.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:23 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:23.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:23 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:23.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:23 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:24.773 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:24 smithi063 ceph-mon[29809]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:24.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:24 smithi165 ceph-mon[33235]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:24.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:24 smithi071 ceph-mon[33293]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:25.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.066+0000 7f9dbb200700 1 -- 172.21.15.63:0/2847157808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 msgr2=0x7f9dbc112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:25.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.066+0000 7f9dbb200700 1 --2- 172.21.15.63:0/2847157808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc112130 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9db400b0a0 tx=0x7f9db400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.066+0000 7f9dbb200700 1 -- 172.21.15.63:0/2847157808 shutdown_connections 2024-05-11T13:28:25.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.066+0000 7f9dbb200700 1 --2- 172.21.15.63:0/2847157808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc112130 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.066+0000 7f9dbb200700 1 --2- 172.21.15.63:0/2847157808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dbc075980 0x7f9dbc075d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.066+0000 7f9dbb200700 1 --2- 172.21.15.63:0/2847157808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dbc0762a0 0x7f9dbc10f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.066+0000 7f9dbb200700 1 -- 172.21.15.63:0/2847157808 >> 172.21.15.63:0/2847157808 conn(0x7f9dbc0fefd0 msgr2=0x7f9dbc1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:25.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.067+0000 7f9dbb200700 1 -- 172.21.15.63:0/2847157808 shutdown_connections 2024-05-11T13:28:25.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.067+0000 7f9dbb200700 1 -- 172.21.15.63:0/2847157808 wait complete. 2024-05-11T13:28:25.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.068+0000 7f9dbb200700 1 Processor -- start 2024-05-11T13:28:25.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.068+0000 7f9dbb200700 1 -- start start 2024-05-11T13:28:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.069+0000 7f9dbb200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dbc075980 0x7f9dbc19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.069+0000 7f9dbb200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dbc0762a0 0x7f9dbc19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.069+0000 7f9dbb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.069+0000 7f9dbb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc115840 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.069+0000 7f9dbb200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1156c0 con 0x7f9dbc0762a0 2024-05-11T13:28:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.069+0000 7f9dbb200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1159c0 con 0x7f9dbc075980 2024-05-11T13:28:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.069+0000 7f9dba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45720/0 (socket says 172.21.15.63:45720) 2024-05-11T13:28:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 -- 172.21.15.63:0/1260849049 learned_addr learned my addr 172.21.15.63:0/1260849049 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9db9e00700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dbc075980 0x7f9dbc19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9db9400700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dbc0762a0 0x7f9dbc19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 -- 172.21.15.63:0/1260849049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dbc075980 msgr2=0x7f9dbc19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dbc075980 0x7f9dbc19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 -- 172.21.15.63:0/1260849049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dbc0762a0 msgr2=0x7f9dbc19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dbc0762a0 0x7f9dbc19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 -- 172.21.15.63:0/1260849049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc1a0de0 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9db9400700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dbc0762a0 0x7f9dbc19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:28:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.070+0000 7f9dba800700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc1a06a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9db4004dc0 tx=0x7f9db4004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.071+0000 7f9db2a00700 1 -- 172.21.15.63:0/1260849049 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4021070 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.071+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc1a1070 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.071+0000 7f9db2a00700 1 -- 172.21.15.63:0/1260849049 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9db4007c00 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.071+0000 7f9db2a00700 1 -- 172.21.15.63:0/1260849049 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40052a0 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.071+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc1a8950 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.072+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc06a240 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.075+0000 7f9db2a00700 1 -- 172.21.15.63:0/1260849049 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9db4024070 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.077+0000 7f9db2a00700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9da00b2900 0x7f9da00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.078+0000 7f9db2a00700 1 -- 172.21.15.63:0/1260849049 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f9db400f020 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.078+0000 7f9db9e00700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9da00b2900 0x7f9da00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:25.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.079+0000 7f9db9e00700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9da00b2900 0x7f9da00b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009a10 tx=0x7f9dac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:25.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.081+0000 7f9db2a00700 1 -- 172.21.15.63:0/1260849049 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9db402b020 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.370+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9dbc02e570 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.371+0000 7f9db2a00700 1 -- 172.21.15.63:0/1260849049 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f9db40aba60 con 0x7f9dbc10fcc0 2024-05-11T13:28:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:28:25.373 INFO:teuthology.orchestra.run.smithi063.stdout:[ 2024-05-11T13:28:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:28:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "poolnum": 1, 2024-05-11T13:28:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "poolname": ".mgr" 2024-05-11T13:28:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:28:25.374 INFO:teuthology.orchestra.run.smithi063.stdout:] 2024-05-11T13:28:25.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9da00b2900 msgr2=0x7f9da00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:25.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9da00b2900 0x7f9da00b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009a10 tx=0x7f9dac009190 comp rx=0 tx=0).stop 2024-05-11T13:28:25.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 msgr2=0x7f9dbc1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:25.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc1a06a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9db4004dc0 tx=0x7f9db4004df0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 shutdown_connections 2024-05-11T13:28:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9da00b2900 0x7f9da00b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc10fcc0 0x7f9dbc1a06a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dbc075980 0x7f9dbc19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 --2- 172.21.15.63:0/1260849049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dbc0762a0 0x7f9dbc19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 >> 172.21.15.63:0/1260849049 conn(0x7f9dbc0fefd0 msgr2=0x7f9dbc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 shutdown_connections 2024-05-11T13:28:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:25.375+0000 7f9dbb200700 1 -- 172.21.15.63:0/1260849049 wait complete. 2024-05-11T13:28:26.207 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T13:28:26.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:26 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:26 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1260849049' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-11T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:26 smithi063 ceph-mon[29809]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:26.661 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:26 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:26 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1260849049' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-11T13:28:26.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:26 smithi071 ceph-mon[33293]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:26 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:26 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1260849049' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-11T13:28:26.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:26 smithi165 ceph-mon[33235]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:27.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:27.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:28:27.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:27.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:27 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:28:27.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:27.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:28:27.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:27.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:27 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:28:27.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:27.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:28:27.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:27.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:27 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:28:28.608 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:28 smithi063 ceph-mon[29809]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:28.609 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:28.609 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:28 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:28.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:28 smithi165 ceph-mon[33235]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:28.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:28.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:28 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:28.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:28 smithi071 ceph-mon[33293]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:28.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:28.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:28 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.774+0000 7fb275800700 1 -- 172.21.15.63:0/2601163970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 msgr2=0x7fb27010fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.774+0000 7fb275800700 1 --2- 172.21.15.63:0/2601163970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb27010fa10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009990 tx=0x7fb25c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.774+0000 7fb275800700 1 -- 172.21.15.63:0/2601163970 shutdown_connections 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.774+0000 7fb275800700 1 --2- 172.21.15.63:0/2601163970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb27010ff50 0x7fb270112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.774+0000 7fb275800700 1 --2- 172.21.15.63:0/2601163970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb270075780 0x7fb270075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.774+0000 7fb275800700 1 --2- 172.21.15.63:0/2601163970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb27010fa10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.774+0000 7fb275800700 1 -- 172.21.15.63:0/2601163970 >> 172.21.15.63:0/2601163970 conn(0x7fb27007a6b0 msgr2=0x7fb27007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.775+0000 7fb275800700 1 -- 172.21.15.63:0/2601163970 shutdown_connections 2024-05-11T13:28:28.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.775+0000 7fb275800700 1 -- 172.21.15.63:0/2601163970 wait complete. 2024-05-11T13:28:28.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.776+0000 7fb275800700 1 Processor -- start 2024-05-11T13:28:28.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.776+0000 7fb275800700 1 -- start start 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb275800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb270075780 0x7fb2701a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb275800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb2701a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb275800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb27010ff50 0x7fb27019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb275800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701157c0 con 0x7fb2700760a0 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb275800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb270115640 con 0x7fb27010ff50 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb275800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb270115940 con 0x7fb270075780 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb26e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb2701a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:28.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb26e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb2701a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45732/0 (socket says 172.21.15.63:45732) 2024-05-11T13:28:28.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb26e000700 1 -- 172.21.15.63:0/643991501 learned_addr learned my addr 172.21.15.63:0/643991501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:28.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb26f400700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb27010ff50 0x7fb27019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb26ea00700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb270075780 0x7fb2701a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.777+0000 7fb26e000700 1 -- 172.21.15.63:0/643991501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb270075780 msgr2=0x7fb2701a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb26e000700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb270075780 0x7fb2701a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb26e000700 1 -- 172.21.15.63:0/643991501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb27010ff50 msgr2=0x7fb27019ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb26e000700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb27010ff50 0x7fb27019ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb26e000700 1 -- 172.21.15.63:0/643991501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27019c1e0 con 0x7fb2700760a0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb26f400700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb27010ff50 0x7fb27019ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb26ea00700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb270075780 0x7fb2701a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb26e000700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb2701a1db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb25c000c00 tx=0x7fb25c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb267400700 1 -- 172.21.15.63:0/643991501 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c027070 con 0x7fb2700760a0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb267400700 1 -- 172.21.15.63:0/643991501 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb25c00fe20 con 0x7fb2700760a0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27019c470 con 0x7fb2700760a0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb267400700 1 -- 172.21.15.63:0/643991501 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c022d00 con 0x7fb2700760a0 2024-05-11T13:28:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.778+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2701a8740 con 0x7fb2700760a0 2024-05-11T13:28:28.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.780+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27006a240 con 0x7fb2700760a0 2024-05-11T13:28:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.783+0000 7fb267400700 1 -- 172.21.15.63:0/643991501 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb25c02a070 con 0x7fb2700760a0 2024-05-11T13:28:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.785+0000 7fb267400700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2540b2880 0x7fb2540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.785+0000 7fb267400700 1 -- 172.21.15.63:0/643991501 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb25c0e2f20 con 0x7fb2700760a0 2024-05-11T13:28:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.786+0000 7fb26ea00700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2540b2880 0x7fb2540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.786+0000 7fb26ea00700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2540b2880 0x7fb2540b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb260005fd0 tx=0x7fb260005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:28.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:28.787+0000 7fb267400700 1 -- 172.21.15.63:0/643991501 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb25c031080 con 0x7fb2700760a0 2024-05-11T13:28:29.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.058+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb27002e570 con 0x7fb2700760a0 2024-05-11T13:28:29.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.395+0000 7fb267400700 1 -- 172.21.15.63:0/643991501 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fb25c013070 con 0x7fb2700760a0 2024-05-11T13:28:29.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2540b2880 msgr2=0x7fb2540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:29.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2540b2880 0x7fb2540b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb260005fd0 tx=0x7fb260005e40 comp rx=0 tx=0).stop 2024-05-11T13:28:29.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 msgr2=0x7fb2701a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:29.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb2701a1db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb25c000c00 tx=0x7fb25c009f90 comp rx=0 tx=0).stop 2024-05-11T13:28:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 shutdown_connections 2024-05-11T13:28:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb2540b2880 0x7fb2540b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb27010ff50 0x7fb27019ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb270075780 0x7fb2701a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 --2- 172.21.15.63:0/643991501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2700760a0 0x7fb2701a1db0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.401+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 >> 172.21.15.63:0/643991501 conn(0x7fb27007a6b0 msgr2=0x7fb270100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.402+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 shutdown_connections 2024-05-11T13:28:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:29.402+0000 7fb275800700 1 -- 172.21.15.63:0/643991501 wait complete. 2024-05-11T13:28:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_metadata' created 2024-05-11T13:28:29.604 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:29 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/643991501' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T13:28:29.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:29 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/643991501' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T13:28:29.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:29 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/643991501' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T13:28:30.373 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-11T13:28:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:30 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/643991501' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-11T13:28:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:30 smithi063 ceph-mon[29809]: osdmap e73: 12 total, 12 up, 12 in 2024-05-11T13:28:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:30 smithi063 ceph-mon[29809]: pgmap v358: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:30 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:30 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/643991501' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-11T13:28:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:30 smithi071 ceph-mon[33293]: osdmap e73: 12 total, 12 up, 12 in 2024-05-11T13:28:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:30 smithi071 ceph-mon[33293]: pgmap v358: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:30 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:30 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/643991501' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-11T13:28:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:30 smithi165 ceph-mon[33235]: osdmap e73: 12 total, 12 up, 12 in 2024-05-11T13:28:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:30 smithi165 ceph-mon[33235]: pgmap v358: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:30 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:30.834 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:31.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:31 smithi063 ceph-mon[29809]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:31.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:31 smithi063 ceph-mon[29809]: osdmap e74: 12 total, 12 up, 12 in 2024-05-11T13:28:31.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:31 smithi071 ceph-mon[33293]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:31.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:31 smithi071 ceph-mon[33293]: osdmap e74: 12 total, 12 up, 12 in 2024-05-11T13:28:31.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:31 smithi165 ceph-mon[33235]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:31.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:31 smithi165 ceph-mon[33235]: osdmap e74: 12 total, 12 up, 12 in 2024-05-11T13:28:32.544 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:32 smithi063 ceph-mon[29809]: osdmap e75: 12 total, 12 up, 12 in 2024-05-11T13:28:32.545 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:32 smithi063 ceph-mon[29809]: pgmap v361: 33 pgs: 2 creating+activating, 6 active+clean, 25 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:32.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:32 smithi071 ceph-mon[33293]: osdmap e75: 12 total, 12 up, 12 in 2024-05-11T13:28:32.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:32 smithi071 ceph-mon[33293]: pgmap v361: 33 pgs: 2 creating+activating, 6 active+clean, 25 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:32 smithi165 ceph-mon[33235]: osdmap e75: 12 total, 12 up, 12 in 2024-05-11T13:28:32.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:32 smithi165 ceph-mon[33235]: pgmap v361: 33 pgs: 2 creating+activating, 6 active+clean, 25 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:33.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 -- 172.21.15.63:0/1120665524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a8075780 msgr2=0x7fb3a8075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 --2- 172.21.15.63:0/1120665524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a8075780 0x7fb3a8075b60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009990 tx=0x7fb39c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 -- 172.21.15.63:0/1120665524 shutdown_connections 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 --2- 172.21.15.63:0/1120665524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a8114780 0x7fb3a8116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 --2- 172.21.15.63:0/1120665524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a8075780 0x7fb3a8075b60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 --2- 172.21.15.63:0/1120665524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3a80760a0 0x7fb3a8114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 -- 172.21.15.63:0/1120665524 >> 172.21.15.63:0/1120665524 conn(0x7fb3a807a6a0 msgr2=0x7fb3a807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 -- 172.21.15.63:0/1120665524 shutdown_connections 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.094+0000 7fb3b0800700 1 -- 172.21.15.63:0/1120665524 wait complete. 2024-05-11T13:28:33.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.096+0000 7fb3b0800700 1 Processor -- start 2024-05-11T13:28:33.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.096+0000 7fb3b0800700 1 -- start start 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.096+0000 7fb3b0800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a8075780 0x7fb3a81a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.096+0000 7fb3b0800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a80760a0 0x7fb3a81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.096+0000 7fb3b0800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3a8114780 0x7fb3a81a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3b0800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8119c60 con 0x7fb3a80760a0 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3b0800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8119ae0 con 0x7fb3a8114780 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3b0800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8119de0 con 0x7fb3a8075780 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a80760a0 0x7fb3a81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a80760a0 0x7fb3a81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34416/0 (socket says 172.21.15.63:34416) 2024-05-11T13:28:33.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3ad400700 1 -- 172.21.15.63:0/2293662390 learned_addr learned my addr 172.21.15.63:0/2293662390 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3ade00700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a8075780 0x7fb3a81a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.097+0000 7fb3ae800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3a8114780 0x7fb3a81a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3ad400700 1 -- 172.21.15.63:0/2293662390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a8075780 msgr2=0x7fb3a81a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3ad400700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a8075780 0x7fb3a81a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3ad400700 1 -- 172.21.15.63:0/2293662390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3a8114780 msgr2=0x7fb3a81a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3ad400700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3a8114780 0x7fb3a81a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3ad400700 1 -- 172.21.15.63:0/2293662390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a81a5180 con 0x7fb3a80760a0 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3ae800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3a8114780 0x7fb3a81a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3ad400700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a80760a0 0x7fb3a81a06b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb39800d950 tx=0x7fb39800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3a2a00700 1 -- 172.21.15.63:0/2293662390 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398016070 con 0x7fb3a80760a0 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3a2a00700 1 -- 172.21.15.63:0/2293662390 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb398009dd0 con 0x7fb3a80760a0 2024-05-11T13:28:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.098+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a81a5470 con 0x7fb3a80760a0 2024-05-11T13:28:33.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.099+0000 7fb3a2a00700 1 -- 172.21.15.63:0/2293662390 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39800f3c0 con 0x7fb3a80760a0 2024-05-11T13:28:33.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.099+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a81accf0 con 0x7fb3a80760a0 2024-05-11T13:28:33.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.101+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a8076ea0 con 0x7fb3a80760a0 2024-05-11T13:28:33.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.104+0000 7fb3a2a00700 1 -- 172.21.15.63:0/2293662390 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb398019070 con 0x7fb3a80760a0 2024-05-11T13:28:33.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.106+0000 7fb3a2a00700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb3900b2900 0x7fb3900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:33.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.106+0000 7fb3a2a00700 1 -- 172.21.15.63:0/2293662390 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7fb3980e1f20 con 0x7fb3a80760a0 2024-05-11T13:28:33.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.106+0000 7fb3ade00700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb3900b2900 0x7fb3900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:33.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.107+0000 7fb3ade00700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb3900b2900 0x7fb3900b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006010 tx=0x7fb39c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:33.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.111+0000 7fb3a2a00700 1 -- 172.21.15.63:0/2293662390 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3980a9ed0 con 0x7fb3a80760a0 2024-05-11T13:28:33.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:33.367+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb3a81acf20 con 0x7fb3a80760a0 2024-05-11T13:28:34.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.163+0000 7fb3a2a00700 1 -- 172.21.15.63:0/2293662390 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7fb3980a97d0 con 0x7fb3a80760a0 2024-05-11T13:28:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb3900b2900 msgr2=0x7fb3900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb3900b2900 0x7fb3900b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006010 tx=0x7fb39c00b540 comp rx=0 tx=0).stop 2024-05-11T13:28:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a80760a0 msgr2=0x7fb3a81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a80760a0 0x7fb3a81a06b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb39800d950 tx=0x7fb39800dc60 comp rx=0 tx=0).stop 2024-05-11T13:28:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 shutdown_connections 2024-05-11T13:28:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb3900b2900 0x7fb3900b4dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:34.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3a8114780 0x7fb3a81a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:34.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a8075780 0x7fb3a81a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:34.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 --2- 172.21.15.63:0/2293662390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3a80760a0 0x7fb3a81a06b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:34.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 >> 172.21.15.63:0/2293662390 conn(0x7fb3a807a6a0 msgr2=0x7fb3a8113620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:34.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.168+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 shutdown_connections 2024-05-11T13:28:34.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:34.169+0000 7fb3b0800700 1 -- 172.21.15.63:0/2293662390 wait complete. 2024-05-11T13:28:34.172 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_data' created 2024-05-11T13:28:34.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:34.427 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:28:34.427 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.427 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:34 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2293662390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-11T13:28:34.427 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:34 smithi165 ceph-mon[33235]: pgmap v362: 33 pgs: 2 creating+activating, 23 active+clean, 8 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:34.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:34.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:28:34.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:34 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2293662390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-11T13:28:34.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:34 smithi063 ceph-mon[29809]: pgmap v362: 33 pgs: 2 creating+activating, 23 active+clean, 8 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:34.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:28:34.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:28:34.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:28:34.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:34 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2293662390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-11T13:28:34.455 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:34 smithi071 ceph-mon[33293]: pgmap v362: 33 pgs: 2 creating+activating, 23 active+clean, 8 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:34.989 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T13:28:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:35 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2293662390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-11T13:28:35.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:35 smithi071 ceph-mon[33293]: osdmap e76: 12 total, 12 up, 12 in 2024-05-11T13:28:35.464 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:35.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:35 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2293662390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-11T13:28:35.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:35 smithi165 ceph-mon[33235]: osdmap e76: 12 total, 12 up, 12 in 2024-05-11T13:28:35.617 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:35 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2293662390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-11T13:28:35.771 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:35 smithi063 ceph-mon[29809]: osdmap e76: 12 total, 12 up, 12 in 2024-05-11T13:28:36.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:36 smithi063 ceph-mon[29809]: osdmap e77: 12 total, 12 up, 12 in 2024-05-11T13:28:36.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:36 smithi063 ceph-mon[29809]: pgmap v365: 97 pgs: 64 unknown, 2 creating+activating, 31 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:36.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:36 smithi071 ceph-mon[33293]: osdmap e77: 12 total, 12 up, 12 in 2024-05-11T13:28:36.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:36 smithi071 ceph-mon[33293]: pgmap v365: 97 pgs: 64 unknown, 2 creating+activating, 31 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:36.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:36 smithi165 ceph-mon[33235]: osdmap e77: 12 total, 12 up, 12 in 2024-05-11T13:28:36.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:36 smithi165 ceph-mon[33235]: pgmap v365: 97 pgs: 64 unknown, 2 creating+activating, 31 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:37.314 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:37 smithi063 ceph-mon[29809]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:37.314 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:37 smithi063 ceph-mon[29809]: osdmap e78: 12 total, 12 up, 12 in 2024-05-11T13:28:37.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:37 smithi071 ceph-mon[33293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:37.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:37 smithi071 ceph-mon[33293]: osdmap e78: 12 total, 12 up, 12 in 2024-05-11T13:28:37.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:37 smithi165 ceph-mon[33235]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:37.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:37 smithi165 ceph-mon[33235]: osdmap e78: 12 total, 12 up, 12 in 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 -- 172.21.15.63:0/1102127134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 msgr2=0x7f9a3c103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 --2- 172.21.15.63:0/1102127134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c103920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a240099c0 tx=0x7f9a24009cd0 comp rx=0 tx=0).stop 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 -- 172.21.15.63:0/1102127134 shutdown_connections 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 --2- 172.21.15.63:0/1102127134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a3c103e60 0x7f9a3c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 --2- 172.21.15.63:0/1102127134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a3c109550 0x7f9a3c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 --2- 172.21.15.63:0/1102127134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c103920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 -- 172.21.15.63:0/1102127134 >> 172.21.15.63:0/1102127134 conn(0x7f9a3c0fefd0 msgr2=0x7f9a3c1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 -- 172.21.15.63:0/1102127134 shutdown_connections 2024-05-11T13:28:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.841+0000 7f9a42200700 1 -- 172.21.15.63:0/1102127134 wait complete. 2024-05-11T13:28:37.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.842+0000 7f9a42200700 1 Processor -- start 2024-05-11T13:28:37.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.842+0000 7f9a42200700 1 -- start start 2024-05-11T13:28:37.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a42200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:37.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a42200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a3c103e60 0x7f9a3c1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:37.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a42200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a3c109550 0x7f9a3c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:37.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a42200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c113630 con 0x7f9a3c1034a0 2024-05-11T13:28:37.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a42200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c1134b0 con 0x7f9a3c109550 2024-05-11T13:28:37.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a42200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c1137b0 con 0x7f9a3c103e60 2024-05-11T13:28:37.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a40e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:37.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a40e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34432/0 (socket says 172.21.15.63:34432) 2024-05-11T13:28:37.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.843+0000 7f9a40e00700 1 -- 172.21.15.63:0/3976879286 learned_addr learned my addr 172.21.15.63:0/3976879286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a33e00700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a3c103e60 0x7f9a3c1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a41800700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a3c109550 0x7f9a3c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a40e00700 1 -- 172.21.15.63:0/3976879286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a3c103e60 msgr2=0x7f9a3c1a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a40e00700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a3c103e60 0x7f9a3c1a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a40e00700 1 -- 172.21.15.63:0/3976879286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a3c109550 msgr2=0x7f9a3c19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a40e00700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a3c109550 0x7f9a3c19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a40e00700 1 -- 172.21.15.63:0/3976879286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3c19c410 con 0x7f9a3c1034a0 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a41800700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a3c109550 0x7f9a3c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:28:37.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a40e00700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c1a1a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00b530 tx=0x7f9a2c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a3a000700 1 -- 172.21.15.63:0/3976879286 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c015020 con 0x7f9a3c1034a0 2024-05-11T13:28:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a3a000700 1 -- 172.21.15.63:0/3976879286 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a2c011cc0 con 0x7f9a3c1034a0 2024-05-11T13:28:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a3a000700 1 -- 172.21.15.63:0/3976879286 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c010d30 con 0x7f9a3c1034a0 2024-05-11T13:28:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.844+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a3c19c700 con 0x7f9a3c1034a0 2024-05-11T13:28:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.845+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a3c1a89e0 con 0x7f9a3c1034a0 2024-05-11T13:28:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.846+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3c104bc0 con 0x7f9a3c1034a0 2024-05-11T13:28:37.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.848+0000 7f9a3a000700 1 -- 172.21.15.63:0/3976879286 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9a2c018020 con 0x7f9a3c1034a0 2024-05-11T13:28:37.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.850+0000 7f9a3a000700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9a200b2900 0x7f9a200b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:37.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.850+0000 7f9a3a000700 1 -- 172.21.15.63:0/3976879286 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f9a2c0e1e90 con 0x7f9a3c1034a0 2024-05-11T13:28:37.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.850+0000 7f9a33e00700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9a200b2900 0x7f9a200b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:37.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.851+0000 7f9a33e00700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9a200b2900 0x7f9a200b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000c00 tx=0x7f9a2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:37.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:37.853+0000 7f9a3a000700 1 -- 172.21.15.63:0/3976879286 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a2c01f020 con 0x7f9a3c1034a0 2024-05-11T13:28:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.141+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9a3c1a8bb0 con 0x7f9a3c1034a0 2024-05-11T13:28:38.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.216+0000 7f9a3a000700 1 -- 172.21.15.63:0/3976879286 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f9a2c0a9cc0 con 0x7f9a3c1034a0 2024-05-11T13:28:38.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.220+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9a200b2900 msgr2=0x7f9a200b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:38.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.220+0000 7f9a42200700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9a200b2900 0x7f9a200b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000c00 tx=0x7f9a2401a040 comp rx=0 tx=0).stop 2024-05-11T13:28:38.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.220+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 msgr2=0x7f9a3c1a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.220+0000 7f9a42200700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c1a1a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00b530 tx=0x7f9a2c00b8f0 comp rx=0 tx=0).stop 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 shutdown_connections 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9a200b2900 0x7f9a200b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a3c109550 0x7f9a3c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a3c1034a0 0x7f9a3c1a1a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 --2- 172.21.15.63:0/3976879286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a3c103e60 0x7f9a3c1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 >> 172.21.15.63:0/3976879286 conn(0x7f9a3c0fefd0 msgr2=0x7f9a3c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 shutdown_connections 2024-05-11T13:28:38.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:38.221+0000 7f9a42200700 1 -- 172.21.15.63:0/3976879286 wait complete. 2024-05-11T13:28:38.224 INFO:teuthology.orchestra.run.smithi063.stderr:new fs with metadata pool 2 and data pool 3 2024-05-11T13:28:38.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:38 smithi063 ceph-mon[29809]: pgmap v367: 97 pgs: 41 unknown, 56 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:38.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:38 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3976879286' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-11T13:28:38.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:38 smithi063 ceph-mon[29809]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-11T13:28:38.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:38 smithi165 ceph-mon[33235]: pgmap v367: 97 pgs: 41 unknown, 56 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:38.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:38 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3976879286' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-11T13:28:38.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:38 smithi165 ceph-mon[33235]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-11T13:28:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:38 smithi071 ceph-mon[33293]: pgmap v367: 97 pgs: 41 unknown, 56 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:38 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3976879286' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-11T13:28:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:38 smithi071 ceph-mon[33293]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-11T13:28:39.220 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-11T13:28:39.220 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-11T13:28:39.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:39 smithi063 ceph-mon[29809]: osdmap e79: 12 total, 12 up, 12 in 2024-05-11T13:28:39.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:39 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3976879286' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-11T13:28:39.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:39 smithi063 ceph-mon[29809]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-11T13:28:39.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:39 smithi063 ceph-mon[29809]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-11T13:28:39.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:39 smithi165 ceph-mon[33235]: osdmap e79: 12 total, 12 up, 12 in 2024-05-11T13:28:39.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:39 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3976879286' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-11T13:28:39.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:39 smithi165 ceph-mon[33235]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-11T13:28:39.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:39 smithi165 ceph-mon[33235]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-11T13:28:39.701 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:39.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:39 smithi071 ceph-mon[33293]: osdmap e79: 12 total, 12 up, 12 in 2024-05-11T13:28:39.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:39 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3976879286' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-11T13:28:39.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:39 smithi071 ceph-mon[33293]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-11T13:28:39.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:39 smithi071 ceph-mon[33293]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-11T13:28:40.204 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:28:39 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b[63166]: 2024-05-11T13:28:39.945+0000 7ff8b9000700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T13:28:40.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:40 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] up:active 2024-05-11T13:28:40.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:40 smithi063 ceph-mon[29809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:40.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:40 smithi063 ceph-mon[29809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:40.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:40 smithi063 ceph-mon[29809]: pgmap v369: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-11T13:28:40.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:40 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] up:active 2024-05-11T13:28:40.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:40 smithi165 ceph-mon[33235]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:40.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:40 smithi165 ceph-mon[33235]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:40.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:40 smithi165 ceph-mon[33235]: pgmap v369: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-11T13:28:40.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:40 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] up:active 2024-05-11T13:28:40.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:40 smithi071 ceph-mon[33293]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:40.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:40 smithi071 ceph-mon[33293]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:40.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:40 smithi071 ceph-mon[33293]: pgmap v369: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-11T13:28:41.552 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:41 smithi063 ceph-mon[29809]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-11T13:28:41.552 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:41 smithi063 ceph-mon[29809]: Cluster is now healthy 2024-05-11T13:28:41.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:41 smithi165 ceph-mon[33235]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-11T13:28:41.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:41 smithi165 ceph-mon[33235]: Cluster is now healthy 2024-05-11T13:28:41.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:41 smithi071 ceph-mon[33293]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-11T13:28:41.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:41 smithi071 ceph-mon[33293]: Cluster is now healthy 2024-05-11T13:28:42.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 -- 172.21.15.63:0/3235373281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae0075780 msgr2=0x7f0ae0075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:42.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 --2- 172.21.15.63:0/3235373281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae0075780 0x7f0ae0075b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009990 tx=0x7f0ad0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:28:42.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 -- 172.21.15.63:0/3235373281 shutdown_connections 2024-05-11T13:28:42.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 --2- 172.21.15.63:0/3235373281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ae010ff50 0x7f0ae0112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:42.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 --2- 172.21.15.63:0/3235373281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae0075780 0x7f0ae0075b60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:42.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 --2- 172.21.15.63:0/3235373281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae00760a0 0x7f0ae010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:42.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 -- 172.21.15.63:0/3235373281 >> 172.21.15.63:0/3235373281 conn(0x7f0ae007a6b0 msgr2=0x7f0ae007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:42.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 -- 172.21.15.63:0/3235373281 shutdown_connections 2024-05-11T13:28:42.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.100+0000 7f0ae5200700 1 -- 172.21.15.63:0/3235373281 wait complete. 2024-05-11T13:28:42.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.101+0000 7f0ae5200700 1 Processor -- start 2024-05-11T13:28:42.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ae5200700 1 -- start start 2024-05-11T13:28:42.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ae5200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ae0075780 0x7f0ae01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:42.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ae5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae00760a0 0x7f0ae01a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:42.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ae5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae010ff50 0x7f0ae019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:42.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ae5200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0115800 con 0x7f0ae00760a0 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ae5200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0115680 con 0x7f0ae010ff50 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ae5200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0115980 con 0x7f0ae0075780 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ad5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae00760a0 0x7f0ae01a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ad5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae00760a0 0x7f0ae01a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46988/0 (socket says 172.21.15.63:46988) 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.102+0000 7f0ad5400700 1 -- 172.21.15.63:0/2547227471 learned_addr learned my addr 172.21.15.63:0/2547227471 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0adde00700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ae0075780 0x7f0ae01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ade800700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae010ff50 0x7f0ae019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ad5400700 1 -- 172.21.15.63:0/2547227471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ae0075780 msgr2=0x7f0ae01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ad5400700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ae0075780 0x7f0ae01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ad5400700 1 -- 172.21.15.63:0/2547227471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae010ff50 msgr2=0x7f0ae019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ad5400700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae010ff50 0x7f0ae019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ad5400700 1 -- 172.21.15.63:0/2547227471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae019c1a0 con 0x7f0ae00760a0 2024-05-11T13:28:42.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ade800700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae010ff50 0x7f0ae019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:28:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ad5400700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae00760a0 0x7f0ae01a1dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c990 tx=0x7f0ac800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.103+0000 7f0ad7400700 1 -- 172.21.15.63:0/2547227471 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8007b00 con 0x7f0ae00760a0 2024-05-11T13:28:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.104+0000 7f0ad7400700 1 -- 172.21.15.63:0/2547227471 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac8007ca0 con 0x7f0ae00760a0 2024-05-11T13:28:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.104+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae019c490 con 0x7f0ae00760a0 2024-05-11T13:28:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.104+0000 7f0ad7400700 1 -- 172.21.15.63:0/2547227471 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8018660 con 0x7f0ae00760a0 2024-05-11T13:28:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.104+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a8750 con 0x7f0ae00760a0 2024-05-11T13:28:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.105+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae006a240 con 0x7f0ae00760a0 2024-05-11T13:28:42.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.107+0000 7f0ad7400700 1 -- 172.21.15.63:0/2547227471 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0ac8004020 con 0x7f0ae00760a0 2024-05-11T13:28:42.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.109+0000 7f0ad7400700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0ac40b2900 0x7f0ac40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:42.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.109+0000 7f0ad7400700 1 -- 172.21.15.63:0/2547227471 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f0ac80e2210 con 0x7f0ae00760a0 2024-05-11T13:28:42.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.109+0000 7f0adde00700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0ac40b2900 0x7f0ac40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:42.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.110+0000 7f0adde00700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0ac40b2900 0x7f0ac40b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b5c0 tx=0x7f0ad001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:42.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.111+0000 7f0ad7400700 1 -- 172.21.15.63:0/2547227471 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ac8053df0 con 0x7f0ae00760a0 2024-05-11T13:28:42.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:42.366+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0ae019cf70 con 0x7f0ae00760a0 2024-05-11T13:28:42.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:42 smithi165 ceph-mon[33235]: pgmap v370: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-11T13:28:42.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:42 smithi063 ceph-mon[29809]: pgmap v370: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-11T13:28:42.703 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:42 smithi071 ceph-mon[33293]: pgmap v370: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-11T13:28:43.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.262+0000 7f0ad7400700 1 -- 172.21.15.63:0/2547227471 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f0ae019cf70 con 0x7f0ae00760a0 2024-05-11T13:28:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0ac40b2900 msgr2=0x7f0ac40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0ac40b2900 0x7f0ac40b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b5c0 tx=0x7f0ad001a040 comp rx=0 tx=0).stop 2024-05-11T13:28:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae00760a0 msgr2=0x7f0ae01a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae00760a0 0x7f0ae01a1dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c990 tx=0x7f0ac800cd50 comp rx=0 tx=0).stop 2024-05-11T13:28:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 shutdown_connections 2024-05-11T13:28:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0ac40b2900 0x7f0ac40b4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae010ff50 0x7f0ae019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ae0075780 0x7f0ae01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 --2- 172.21.15.63:0/2547227471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ae00760a0 0x7f0ae01a1dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.268+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 >> 172.21.15.63:0/2547227471 conn(0x7f0ae007a6b0 msgr2=0x7f0ae0101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.269+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 shutdown_connections 2024-05-11T13:28:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:43.269+0000 7f0ae5200700 1 -- 172.21.15.63:0/2547227471 wait complete. 2024-05-11T13:28:43.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:43 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2547227471' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-11T13:28:43.651 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:43 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2547227471' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-11T13:28:43.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:43 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2547227471' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-11T13:28:44.119 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T13:28:44.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:44 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2547227471' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-11T13:28:44.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:44 smithi063 ceph-mon[29809]: osdmap e80: 12 total, 12 up, 12 in 2024-05-11T13:28:44.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:44 smithi063 ceph-mon[29809]: pgmap v372: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-11T13:28:44.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:44 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2547227471' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-11T13:28:44.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:44 smithi165 ceph-mon[33235]: osdmap e80: 12 total, 12 up, 12 in 2024-05-11T13:28:44.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:44 smithi165 ceph-mon[33235]: pgmap v372: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-11T13:28:44.590 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:44.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:44 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2547227471' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-11T13:28:44.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:44 smithi071 ceph-mon[33293]: osdmap e80: 12 total, 12 up, 12 in 2024-05-11T13:28:44.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:44 smithi071 ceph-mon[33293]: pgmap v372: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-11T13:28:46.767 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:46 smithi063 ceph-mon[29809]: pgmap v373: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-11T13:28:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:46 smithi071 ceph-mon[33293]: pgmap v373: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-11T13:28:47.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4010990131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc1086e0 msgr2=0x7fbcbc1102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4010990131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc1086e0 0x7fbcbc1102b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400b0d0 tx=0x7fbcb400b3e0 comp rx=0 tx=0).stop 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4010990131 shutdown_connections 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4010990131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc1086e0 0x7fbcbc1102b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4010990131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcbc103600 0x7fbcbc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4010990131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcbc103fb0 0x7fbcbc108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4010990131 >> 172.21.15.63:0/4010990131 conn(0x7fbcbc0fef30 msgr2=0x7fbcbc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4010990131 shutdown_connections 2024-05-11T13:28:47.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.055+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4010990131 wait complete. 2024-05-11T13:28:47.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.056+0000 7fbcbbe00700 1 Processor -- start 2024-05-11T13:28:47.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbbe00700 1 -- start start 2024-05-11T13:28:47.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc103600 0x7fbcbc1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:47.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcbc103fb0 0x7fbcbc1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:47.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcbc1086e0 0x7fbcbc19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbbe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc1134a0 con 0x7fbcbc103600 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbbe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc113320 con 0x7fbcbc1086e0 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbbe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc113620 con 0x7fbcbc103fb0 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc103600 0x7fbcbc1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc103600 0x7fbcbc1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46998/0 (socket says 172.21.15.63:46998) 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.057+0000 7fbcbaa00700 1 -- 172.21.15.63:0/4270351917 learned_addr learned my addr 172.21.15.63:0/4270351917 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcba000700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcbc103fb0 0x7fbcbc1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcbb400700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcbc1086e0 0x7fbcbc19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcbaa00700 1 -- 172.21.15.63:0/4270351917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcbc103fb0 msgr2=0x7fbcbc1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcbaa00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcbc103fb0 0x7fbcbc1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:47.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcbaa00700 1 -- 172.21.15.63:0/4270351917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcbc1086e0 msgr2=0x7fbcbc19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:47.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcbaa00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcbc1086e0 0x7fbcbc19b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:47.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcbaa00700 1 -- 172.21.15.63:0/4270351917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc19c0c0 con 0x7fbcbc103600 2024-05-11T13:28:47.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcba000700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcbc103fb0 0x7fbcbc1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:28:47.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcbaa00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc103600 0x7fbcbc1a17a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00d950 tx=0x7fbcac00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:47.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcb3400700 1 -- 172.21.15.63:0/4270351917 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac016070 con 0x7fbcbc103600 2024-05-11T13:28:47.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.058+0000 7fbcb3400700 1 -- 172.21.15.63:0/4270351917 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcac010460 con 0x7fbcbc103600 2024-05-11T13:28:47.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.059+0000 7fbcb3400700 1 -- 172.21.15.63:0/4270351917 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00f620 con 0x7fbcbc103600 2024-05-11T13:28:47.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.059+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc19c3b0 con 0x7fbcbc103600 2024-05-11T13:28:47.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.059+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc1a8670 con 0x7fbcbc103600 2024-05-11T13:28:47.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.060+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc064bb0 con 0x7fbcbc103600 2024-05-11T13:28:47.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.062+0000 7fbcb3400700 1 -- 172.21.15.63:0/4270351917 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbcac019070 con 0x7fbcbc103600 2024-05-11T13:28:47.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.064+0000 7fbcb3400700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbca00b29b0 0x7fbca00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:47.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.064+0000 7fbcb3400700 1 -- 172.21.15.63:0/4270351917 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7fbcac0e1e30 con 0x7fbcbc103600 2024-05-11T13:28:47.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.064+0000 7fbcba000700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbca00b29b0 0x7fbca00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:46 smithi165 ceph-mon[33235]: pgmap v373: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-11T13:28:47.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.065+0000 7fbcba000700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbca00b29b0 0x7fbca00b4e70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc1106f0 tx=0x7fbca8009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:47.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.069+0000 7fbcb3400700 1 -- 172.21.15.63:0/4270351917 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcac0a9ac0 con 0x7fbcbc103600 2024-05-11T13:28:47.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:47.312+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbcbc19ce90 con 0x7fbcbc103600 2024-05-11T13:28:47.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:47 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4270351917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-11T13:28:47.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:47 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4270351917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-11T13:28:48.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:47 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4270351917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-11T13:28:48.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.634+0000 7fbcb3400700 1 -- 172.21.15.63:0/4270351917 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7fbcac0a93c0 con 0x7fbcbc103600 2024-05-11T13:28:48.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.639+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbca00b29b0 msgr2=0x7fbca00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:48.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.639+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbca00b29b0 0x7fbca00b4e70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc1106f0 tx=0x7fbca8009550 comp rx=0 tx=0).stop 2024-05-11T13:28:48.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.639+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc103600 msgr2=0x7fbcbc1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:48.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.639+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc103600 0x7fbcbc1a17a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00d950 tx=0x7fbcac00dd10 comp rx=0 tx=0).stop 2024-05-11T13:28:48.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 shutdown_connections 2024-05-11T13:28:48.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbca00b29b0 0x7fbca00b4e70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:48.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcbc1086e0 0x7fbcbc19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:48.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcbc103600 0x7fbcbc1a17a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:48.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 --2- 172.21.15.63:0/4270351917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcbc103fb0 0x7fbcbc1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:48.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 >> 172.21.15.63:0/4270351917 conn(0x7fbcbc0fef30 msgr2=0x7fbcbc0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:48.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 shutdown_connections 2024-05-11T13:28:48.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:48.640+0000 7fbcbbe00700 1 -- 172.21.15.63:0/4270351917 wait complete. 2024-05-11T13:28:48.644 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_data_ec' created 2024-05-11T13:28:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:48 smithi063 ceph-mon[29809]: pgmap v374: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-11T13:28:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:48 smithi063 ceph-mon[29809]: osdmap e81: 12 total, 12 up, 12 in 2024-05-11T13:28:48.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:48 smithi071 ceph-mon[33293]: pgmap v374: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-11T13:28:48.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:48 smithi071 ceph-mon[33293]: osdmap e81: 12 total, 12 up, 12 in 2024-05-11T13:28:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:48 smithi165 ceph-mon[33235]: pgmap v374: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-11T13:28:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:48 smithi165 ceph-mon[33235]: osdmap e81: 12 total, 12 up, 12 in 2024-05-11T13:28:49.456 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-11T13:28:49.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:49 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4270351917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-11T13:28:49.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:49 smithi063 ceph-mon[29809]: osdmap e82: 12 total, 12 up, 12 in 2024-05-11T13:28:49.931 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:49.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:49 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4270351917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-11T13:28:49.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:49 smithi071 ceph-mon[33293]: osdmap e82: 12 total, 12 up, 12 in 2024-05-11T13:28:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:49 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4270351917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-11T13:28:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:49 smithi165 ceph-mon[33235]: osdmap e82: 12 total, 12 up, 12 in 2024-05-11T13:28:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:50 smithi063 ceph-mon[29809]: pgmap v377: 129 pgs: 13 creating+peering, 19 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:50 smithi063 ceph-mon[29809]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:50 smithi063 ceph-mon[29809]: osdmap e83: 12 total, 12 up, 12 in 2024-05-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:50 smithi071 ceph-mon[33293]: pgmap v377: 129 pgs: 13 creating+peering, 19 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:50 smithi071 ceph-mon[33293]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:50 smithi071 ceph-mon[33293]: osdmap e83: 12 total, 12 up, 12 in 2024-05-11T13:28:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:50 smithi165 ceph-mon[33235]: pgmap v377: 129 pgs: 13 creating+peering, 19 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:50 smithi165 ceph-mon[33235]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T13:28:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:50 smithi165 ceph-mon[33235]: osdmap e83: 12 total, 12 up, 12 in 2024-05-11T13:28:51.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:51 smithi063 ceph-mon[29809]: osdmap e84: 12 total, 12 up, 12 in 2024-05-11T13:28:51.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:51 smithi071 ceph-mon[33293]: osdmap e84: 12 total, 12 up, 12 in 2024-05-11T13:28:52.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:51 smithi165 ceph-mon[33235]: osdmap e84: 12 total, 12 up, 12 in 2024-05-11T13:28:52.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 -- 172.21.15.63:0/1955346765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8105770 msgr2=0x7efcc8105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:52.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 --2- 172.21.15.63:0/1955346765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8105770 0x7efcc8105bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009990 tx=0x7efcb4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 -- 172.21.15.63:0/1955346765 shutdown_connections 2024-05-11T13:28:52.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 --2- 172.21.15.63:0/1955346765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc8106130 0x7efcc8112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 --2- 172.21.15.63:0/1955346765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc810b770 0x7efcc810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 --2- 172.21.15.63:0/1955346765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8105770 0x7efcc8105bf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 -- 172.21.15.63:0/1955346765 >> 172.21.15.63:0/1955346765 conn(0x7efcc8101260 msgr2=0x7efcc8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:52.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 -- 172.21.15.63:0/1955346765 shutdown_connections 2024-05-11T13:28:52.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.228+0000 7efccee00700 1 -- 172.21.15.63:0/1955346765 wait complete. 2024-05-11T13:28:52.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.229+0000 7efccee00700 1 Processor -- start 2024-05-11T13:28:52.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efccee00700 1 -- start start 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efccee00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8105770 0x7efcc819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efccee00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8106130 0x7efcc819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efccee00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc810b770 0x7efcc81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efccee00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8115820 con 0x7efcc8106130 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efccee00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81156a0 con 0x7efcc810b770 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efccee00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81159a0 con 0x7efcc8105770 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efcc7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8106130 0x7efcc819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.230+0000 7efcc7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8106130 0x7efcc819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45252/0 (socket says 172.21.15.63:45252) 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7400700 1 -- 172.21.15.63:0/2534811316 learned_addr learned my addr 172.21.15.63:0/2534811316 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7e00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8105770 0x7efcc819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efccce00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc810b770 0x7efcc81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7400700 1 -- 172.21.15.63:0/2534811316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8105770 msgr2=0x7efcc819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7400700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8105770 0x7efcc819bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7400700 1 -- 172.21.15.63:0/2534811316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc810b770 msgr2=0x7efcc81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:52.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7400700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc810b770 0x7efcc81a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7400700 1 -- 172.21.15.63:0/2534811316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc81a0d30 con 0x7efcc8106130 2024-05-11T13:28:52.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.231+0000 7efcc7400700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8106130 0x7efcc819c2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efcb40057c0 tx=0x7efcb4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:52.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.232+0000 7efcc4c00700 1 -- 172.21.15.63:0/2534811316 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4027070 con 0x7efcc8106130 2024-05-11T13:28:52.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.232+0000 7efcc4c00700 1 -- 172.21.15.63:0/2534811316 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcb400fda0 con 0x7efcc8106130 2024-05-11T13:28:52.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.232+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc81a0fc0 con 0x7efcc8106130 2024-05-11T13:28:52.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.232+0000 7efcc4c00700 1 -- 172.21.15.63:0/2534811316 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4022ca0 con 0x7efcc8106130 2024-05-11T13:28:52.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.232+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc81a88a0 con 0x7efcc8106130 2024-05-11T13:28:52.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.233+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc806a240 con 0x7efcc8106130 2024-05-11T13:28:52.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.236+0000 7efcc4c00700 1 -- 172.21.15.63:0/2534811316 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efcb402a070 con 0x7efcc8106130 2024-05-11T13:28:52.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.237+0000 7efcc4c00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efcac0b2880 0x7efcac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:52.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.238+0000 7efcc4c00700 1 -- 172.21.15.63:0/2534811316 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7efcb40e34a0 con 0x7efcc8106130 2024-05-11T13:28:52.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.238+0000 7efcc7e00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efcac0b2880 0x7efcac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:52.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.238+0000 7efcc7e00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efcac0b2880 0x7efcac0b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efcb800bbb0 tx=0x7efcb800b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:52.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.241+0000 7efcc4c00700 1 -- 172.21.15.63:0/2534811316 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efcb4013070 con 0x7efcc8106130 2024-05-11T13:28:52.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.479+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7efcc802e570 con 0x7efcc8106130 2024-05-11T13:28:52.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.669+0000 7efcc4c00700 1 -- 172.21.15.63:0/2534811316 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7efcb40095a0 con 0x7efcc8106130 2024-05-11T13:28:52.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efcac0b2880 msgr2=0x7efcac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:52.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efcac0b2880 0x7efcac0b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efcb800bbb0 tx=0x7efcb800b590 comp rx=0 tx=0).stop 2024-05-11T13:28:52.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8106130 msgr2=0x7efcc819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:52.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8106130 0x7efcc819c2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efcb40057c0 tx=0x7efcb4009f90 comp rx=0 tx=0).stop 2024-05-11T13:28:52.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 shutdown_connections 2024-05-11T13:28:52.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efcac0b2880 0x7efcac0b4d40 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efcb800bbb0 tx=0x7efcb800b590 comp rx=0 tx=0).stop 2024-05-11T13:28:52.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc810b770 0x7efcc81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8105770 0x7efcc819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.674+0000 7efccee00700 1 --2- 172.21.15.63:0/2534811316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcc8106130 0x7efcc819c2b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:52.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.675+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 >> 172.21.15.63:0/2534811316 conn(0x7efcc8101260 msgr2=0x7efcc8101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:52.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.675+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 shutdown_connections 2024-05-11T13:28:52.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:52.675+0000 7efccee00700 1 -- 172.21.15.63:0/2534811316 wait complete. 2024-05-11T13:28:52.677 INFO:teuthology.orchestra.run.smithi063.stderr:set pool 4 allow_ec_overwrites to true 2024-05-11T13:28:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:52 smithi063 ceph-mon[29809]: pgmap v380: 129 pgs: 13 creating+peering, 14 unknown, 102 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:52 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2534811316' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-11T13:28:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:52 smithi071 ceph-mon[33293]: pgmap v380: 129 pgs: 13 creating+peering, 14 unknown, 102 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:52 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2534811316' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-11T13:28:52.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:52 smithi165 ceph-mon[33235]: pgmap v380: 129 pgs: 13 creating+peering, 14 unknown, 102 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:52.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:52 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2534811316' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-11T13:28:53.602 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-11T13:28:53.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:53 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2534811316' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-11T13:28:53.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:53 smithi063 ceph-mon[29809]: osdmap e85: 12 total, 12 up, 12 in 2024-05-11T13:28:53.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2534811316' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-11T13:28:53.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:53 smithi071 ceph-mon[33293]: osdmap e85: 12 total, 12 up, 12 in 2024-05-11T13:28:54.000 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:53 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2534811316' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-11T13:28:54.001 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:53 smithi165 ceph-mon[33235]: osdmap e85: 12 total, 12 up, 12 in 2024-05-11T13:28:54.069 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:54.733 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:54 smithi063 ceph-mon[29809]: pgmap v382: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:54.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:54 smithi071 ceph-mon[33293]: pgmap v382: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:55.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:54 smithi165 ceph-mon[33235]: pgmap v382: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:56.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 -- 172.21.15.63:0/3463419185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf1010b770 msgr2=0x7fbf1010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 --2- 172.21.15.63:0/3463419185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf1010b770 0x7fbf1010bb50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009990 tx=0x7fbf04009ca0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 -- 172.21.15.63:0/3463419185 shutdown_connections 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 --2- 172.21.15.63:0/3463419185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf10106130 0x7fbf10112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 --2- 172.21.15.63:0/3463419185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf1010b770 0x7fbf1010bb50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 --2- 172.21.15.63:0/3463419185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf10105770 0x7fbf10105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 -- 172.21.15.63:0/3463419185 >> 172.21.15.63:0/3463419185 conn(0x7fbf10101260 msgr2=0x7fbf10103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 -- 172.21.15.63:0/3463419185 shutdown_connections 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.366+0000 7fbf17400700 1 -- 172.21.15.63:0/3463419185 wait complete. 2024-05-11T13:28:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 Processor -- start 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 -- start start 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf10105770 0x7fbf1019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf10106130 0x7fbf1019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1010b770 0x7fbf101a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10115850 con 0x7fbf10106130 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101156d0 con 0x7fbf1010b770 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.367+0000 7fbf17400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101159d0 con 0x7fbf10105770 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf10106130 0x7fbf1019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf10106130 0x7fbf1019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45264/0 (socket says 172.21.15.63:45264) 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 -- 172.21.15.63:0/2861129019 learned_addr learned my addr 172.21.15.63:0/2861129019 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf14a00700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf10105770 0x7fbf1019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf15400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1010b770 0x7fbf101a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 -- 172.21.15.63:0/2861129019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf10105770 msgr2=0x7fbf1019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf10105770 0x7fbf1019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 -- 172.21.15.63:0/2861129019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1010b770 msgr2=0x7fbf101a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1010b770 0x7fbf101a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf0fe00700 1 -- 172.21.15.63:0/2861129019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf101a0d60 con 0x7fbf10106130 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.368+0000 7fbf15400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1010b770 0x7fbf101a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.369+0000 7fbf0fe00700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf10106130 0x7fbf1019c2e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000d950 tx=0x7fbf0000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.369+0000 7fbf0d600700 1 -- 172.21.15.63:0/2861129019 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00016070 con 0x7fbf10106130 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.369+0000 7fbf0d600700 1 -- 172.21.15.63:0/2861129019 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf00010460 con 0x7fbf10106130 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.369+0000 7fbf0d600700 1 -- 172.21.15.63:0/2861129019 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000f620 con 0x7fbf10106130 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.369+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf101a1050 con 0x7fbf10106130 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.369+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101a88d0 con 0x7fbf10106130 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.370+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1006a240 con 0x7fbf10106130 2024-05-11T13:28:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.373+0000 7fbf0d600700 1 -- 172.21.15.63:0/2861129019 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbf00019070 con 0x7fbf10106130 2024-05-11T13:28:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.376+0000 7fbf0d600700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbef80b2900 0x7fbef80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:28:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.376+0000 7fbf14a00700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbef80b2900 0x7fbef80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:28:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.376+0000 7fbf0d600700 1 -- 172.21.15.63:0/2861129019 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7fbf000e1ff0 con 0x7fbf10106130 2024-05-11T13:28:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.377+0000 7fbf14a00700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbef80b2900 0x7fbef80b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b5c0 tx=0x7fbf0401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:28:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.377+0000 7fbf0d600700 1 -- 172.21.15.63:0/2861129019 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf000a9af0 con 0x7fbf10106130 2024-05-11T13:28:56.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.665+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fbf1002e570 con 0x7fbf10106130 2024-05-11T13:28:56.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.700+0000 7fbf0d600700 1 -- 172.21.15.63:0/2861129019 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fbf000a93f0 con 0x7fbf10106130 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbef80b2900 msgr2=0x7fbef80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbef80b2900 0x7fbef80b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b5c0 tx=0x7fbf0401a040 comp rx=0 tx=0).stop 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf10106130 msgr2=0x7fbf1019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf10106130 0x7fbf1019c2e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000d950 tx=0x7fbf0000dd10 comp rx=0 tx=0).stop 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 shutdown_connections 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbef80b2900 0x7fbef80b4dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1010b770 0x7fbf101a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf10105770 0x7fbf1019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 --2- 172.21.15.63:0/2861129019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf10106130 0x7fbf1019c2e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:28:56.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.704+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 >> 172.21.15.63:0/2861129019 conn(0x7fbf10101260 msgr2=0x7fbf10101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:28:56.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.705+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 shutdown_connections 2024-05-11T13:28:56.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:28:56.705+0000 7fbf17400700 1 -- 172.21.15.63:0/2861129019 wait complete. 2024-05-11T13:28:56.708 INFO:teuthology.orchestra.run.smithi063.stderr:added data pool 4 to fsmap 2024-05-11T13:28:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:56 smithi063 ceph-mon[29809]: pgmap v383: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:56.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:56 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2861129019' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-11T13:28:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:56 smithi165 ceph-mon[33235]: pgmap v383: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:56 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2861129019' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-11T13:28:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:56 smithi071 ceph-mon[33293]: pgmap v383: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:56 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2861129019' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-11T13:28:57.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:28:57.838 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:57 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2861129019' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-11T13:28:57.839 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:57 smithi063 ceph-mon[29809]: osdmap e86: 12 total, 12 up, 12 in 2024-05-11T13:28:57.839 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:57 smithi063 ceph-mon[29809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:58.009 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:28:58.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:57 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2861129019' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-11T13:28:58.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:57 smithi165 ceph-mon[33235]: osdmap e86: 12 total, 12 up, 12 in 2024-05-11T13:28:58.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:57 smithi165 ceph-mon[33235]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:57 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2861129019' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-11T13:28:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:57 smithi071 ceph-mon[33293]: osdmap e86: 12 total, 12 up, 12 in 2024-05-11T13:28:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:57 smithi071 ceph-mon[33293]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:28:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:58 smithi063 ceph-mon[29809]: pgmap v385: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:58 smithi063 ceph-mon[29809]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-11T13:28:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:28:58 smithi063 ceph-mon[29809]: Cluster is now healthy 2024-05-11T13:28:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:58 smithi165 ceph-mon[33235]: pgmap v385: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:58 smithi165 ceph-mon[33235]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-11T13:28:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:28:58 smithi165 ceph-mon[33235]: Cluster is now healthy 2024-05-11T13:28:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:58 smithi071 ceph-mon[33293]: pgmap v385: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:28:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:58 smithi071 ceph-mon[33293]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-11T13:28:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:28:58 smithi071 ceph-mon[33293]: Cluster is now healthy 2024-05-11T13:29:00.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 -- 172.21.15.63:0/4044949688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc1041b0 msgr2=0x7f06dc1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 --2- 172.21.15.63:0/4044949688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc1041b0 0x7f06dc1081d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f06c80099c0 tx=0x7f06c8009cd0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 -- 172.21.15.63:0/4044949688 shutdown_connections 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 --2- 172.21.15.63:0/4044949688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06dc108900 0x7f06dc110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 --2- 172.21.15.63:0/4044949688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06dc103800 0x7f06dc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 --2- 172.21.15.63:0/4044949688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc1041b0 0x7f06dc1081d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 -- 172.21.15.63:0/4044949688 >> 172.21.15.63:0/4044949688 conn(0x7f06dc07a6b0 msgr2=0x7f06dc07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 -- 172.21.15.63:0/4044949688 shutdown_connections 2024-05-11T13:29:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.335+0000 7f06e1200700 1 -- 172.21.15.63:0/4044949688 wait complete. 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.336+0000 7f06e1200700 1 Processor -- start 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.336+0000 7f06e1200700 1 -- start start 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06e1200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06dc103800 0x7f06dc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06e1200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06dc1041b0 0x7f06dc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06e1200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc108900 0x7f06dc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06e1200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc113590 con 0x7f06dc108900 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06e1200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc113410 con 0x7f06dc1041b0 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06e1200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc113710 con 0x7f06dc103800 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc108900 0x7f06dc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc108900 0x7f06dc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58368/0 (socket says 172.21.15.63:58368) 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.337+0000 7f06da800700 1 -- 172.21.15.63:0/478244647 learned_addr learned my addr 172.21.15.63:0/478244647 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06d9e00700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06dc103800 0x7f06dc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06d9400700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06dc1041b0 0x7f06dc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06da800700 1 -- 172.21.15.63:0/478244647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06dc103800 msgr2=0x7f06dc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06da800700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06dc103800 0x7f06dc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06da800700 1 -- 172.21.15.63:0/478244647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06dc1041b0 msgr2=0x7f06dc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06da800700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06dc1041b0 0x7f06dc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06da800700 1 -- 172.21.15.63:0/478244647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc199ff0 con 0x7f06dc108900 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06da800700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc108900 0x7f06dc199890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06d4007ee0 tx=0x7f06d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06d2a00700 1 -- 172.21.15.63:0/478244647 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d400dbd0 con 0x7f06dc108900 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.338+0000 7f06d2a00700 1 -- 172.21.15.63:0/478244647 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06d400dd70 con 0x7f06dc108900 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.339+0000 7f06d2a00700 1 -- 172.21.15.63:0/478244647 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d40058d0 con 0x7f06dc108900 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.339+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc19a2e0 con 0x7f06dc108900 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.339+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc1a65a0 con 0x7f06dc108900 2024-05-11T13:29:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.340+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06dc104c90 con 0x7f06dc108900 2024-05-11T13:29:00.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.344+0000 7f06d2a00700 1 -- 172.21.15.63:0/478244647 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f06d401a020 con 0x7f06dc108900 2024-05-11T13:29:00.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.345+0000 7f06d2a00700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06c00b2900 0x7f06c00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:00.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.346+0000 7f06d2a00700 1 -- 172.21.15.63:0/478244647 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f06d40e23b0 con 0x7f06dc108900 2024-05-11T13:29:00.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.346+0000 7f06d9e00700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06c00b2900 0x7f06c00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:00.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.347+0000 7f06d9e00700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06c00b2900 0x7f06c00b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005950 tx=0x7f06cc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:00.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.348+0000 7f06d2a00700 1 -- 172.21.15.63:0/478244647 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06d4010070 con 0x7f06dc108900 2024-05-11T13:29:00.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.638+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06dc02e570 con 0x7f06dc108900 2024-05-11T13:29:00.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.639+0000 7f06d2a00700 1 -- 172.21.15.63:0/478244647 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7f06d40a9e30 con 0x7f06dc108900 2024-05-11T13:29:00.640 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:00.640 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:28:56.667019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34472,"qdb_cluster":[34472]},"id":1}]} 2024-05-11T13:29:00.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06c00b2900 msgr2=0x7f06c00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:00.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06c00b2900 0x7f06c00b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005950 tx=0x7f06cc0058e0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc108900 msgr2=0x7f06dc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc108900 0x7f06dc199890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06d4007ee0 tx=0x7f06d400d3b0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 shutdown_connections 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06c00b2900 0x7f06c00b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc108900 0x7f06dc199890 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06dc103800 0x7f06dc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 --2- 172.21.15.63:0/478244647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06dc1041b0 0x7f06dc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 >> 172.21.15.63:0/478244647 conn(0x7f06dc07a6b0 msgr2=0x7f06dc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 shutdown_connections 2024-05-11T13:29:00.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:00.643+0000 7f06e1200700 1 -- 172.21.15.63:0/478244647 wait complete. 2024-05-11T13:29:00.645 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 17 2024-05-11T13:29:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:00 smithi063 ceph-mon[29809]: pgmap v386: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:00 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/478244647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:29:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:00 smithi165 ceph-mon[33235]: pgmap v386: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:00 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/478244647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:29:01.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:00 smithi071 ceph-mon[33293]: pgmap v386: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:01.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:00 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/478244647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:29:01.598 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:29:02.086 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:02.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:02 smithi063 ceph-mon[29809]: pgmap v387: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:02 smithi165 ceph-mon[33235]: pgmap v387: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:03.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:02 smithi071 ceph-mon[33293]: pgmap v387: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:04.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.467+0000 7fe6bf200700 1 -- 172.21.15.63:0/3024404028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102220 msgr2=0x7fe6c0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:04.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.467+0000 7fe6bf200700 1 --2- 172.21.15.63:0/3024404028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102220 0x7fe6c0102600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009990 tx=0x7fe6b0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.468+0000 7fe6bf200700 1 -- 172.21.15.63:0/3024404028 shutdown_connections 2024-05-11T13:29:04.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.468+0000 7fe6bf200700 1 --2- 172.21.15.63:0/3024404028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c010fdd0 0x7fe6c01121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.468+0000 7fe6bf200700 1 --2- 172.21.15.63:0/3024404028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102220 0x7fe6c0102600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.468+0000 7fe6bf200700 1 --2- 172.21.15.63:0/3024404028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c0102bd0 0x7fe6c010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.468+0000 7fe6bf200700 1 -- 172.21.15.63:0/3024404028 >> 172.21.15.63:0/3024404028 conn(0x7fe6c00fdee0 msgr2=0x7fe6c0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:04.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.468+0000 7fe6bf200700 1 -- 172.21.15.63:0/3024404028 shutdown_connections 2024-05-11T13:29:04.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.468+0000 7fe6bf200700 1 -- 172.21.15.63:0/3024404028 wait complete. 2024-05-11T13:29:04.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 Processor -- start 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 -- start start 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c0102220 0x7fe6c01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102bd0 0x7fe6c01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c010fdd0 0x7fe6c019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0115770 con 0x7fe6c0102bd0 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01155f0 con 0x7fe6c0102220 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.469+0000 7fe6bf200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01158f0 con 0x7fe6c010fdd0 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102bd0 0x7fe6c01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102bd0 0x7fe6c01a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58378/0 (socket says 172.21.15.63:58378) 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 -- 172.21.15.63:0/4203983632 learned_addr learned my addr 172.21.15.63:0/4203983632 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bde00700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c0102220 0x7fe6c01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6be800700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c010fdd0 0x7fe6c019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 -- 172.21.15.63:0/4203983632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c010fdd0 msgr2=0x7fe6c019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c010fdd0 0x7fe6c019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 -- 172.21.15.63:0/4203983632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c0102220 msgr2=0x7fe6c01a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:04.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c0102220 0x7fe6c01a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bd400700 1 -- 172.21.15.63:0/4203983632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c019c250 con 0x7fe6c0102bd0 2024-05-11T13:29:04.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.470+0000 7fe6bde00700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c0102220 0x7fe6c01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:29:04.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.471+0000 7fe6bd400700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102bd0 0x7fe6c01a1e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00d950 tx=0x7fe6ac00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:04.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.471+0000 7fe6b6a00700 1 -- 172.21.15.63:0/4203983632 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac016070 con 0x7fe6c0102bd0 2024-05-11T13:29:04.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.471+0000 7fe6b6a00700 1 -- 172.21.15.63:0/4203983632 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6ac010460 con 0x7fe6c0102bd0 2024-05-11T13:29:04.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.471+0000 7fe6b6a00700 1 -- 172.21.15.63:0/4203983632 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00f620 con 0x7fe6c0102bd0 2024-05-11T13:29:04.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.471+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c019c540 con 0x7fe6c0102bd0 2024-05-11T13:29:04.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.471+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a87b0 con 0x7fe6c0102bd0 2024-05-11T13:29:04.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.472+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c006a240 con 0x7fe6c0102bd0 2024-05-11T13:29:04.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.476+0000 7fe6b6a00700 1 -- 172.21.15.63:0/4203983632 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe6ac019070 con 0x7fe6c0102bd0 2024-05-11T13:29:04.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.478+0000 7fe6b6a00700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe6a40b2900 0x7fe6a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:04.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.478+0000 7fe6b6a00700 1 -- 172.21.15.63:0/4203983632 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fe6ac0e2070 con 0x7fe6c0102bd0 2024-05-11T13:29:04.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.478+0000 7fe6bde00700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe6a40b2900 0x7fe6a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:04.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.479+0000 7fe6bde00700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe6a40b2900 0x7fe6a40b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0003760 tx=0x7fe6b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:04.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.480+0000 7fe6b6a00700 1 -- 172.21.15.63:0/4203983632 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6ac0a9af0 con 0x7fe6c0102bd0 2024-05-11T13:29:04.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.719+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6c002e570 con 0x7fe6c0102bd0 2024-05-11T13:29:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.722+0000 7fe6b6a00700 1 -- 172.21.15.63:0/4203983632 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24092 (secure 0 0 0) 0x7fe6ac0a93f0 con 0x7fe6c0102bd0 2024-05-11T13:29:04.723 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:04.723 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":86,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:28:56.666760+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:29:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe6a40b2900 msgr2=0x7fe6a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe6a40b2900 0x7fe6a40b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0003760 tx=0x7fe6b0005fb0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102bd0 msgr2=0x7fe6c01a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102bd0 0x7fe6c01a1e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00d950 tx=0x7fe6ac00dd10 comp rx=0 tx=0).stop 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 shutdown_connections 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe6a40b2900 0x7fe6a40b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c010fdd0 0x7fe6c019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c0102220 0x7fe6c01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 --2- 172.21.15.63:0/4203983632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c0102bd0 0x7fe6c01a1e20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.725+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 >> 172.21.15.63:0/4203983632 conn(0x7fe6c00fdee0 msgr2=0x7fe6c01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.726+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 shutdown_connections 2024-05-11T13:29:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:04.726+0000 7fe6bf200700 1 -- 172.21.15.63:0/4203983632 wait complete. 2024-05-11T13:29:05.047 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:04 smithi165 ceph-mon[33235]: pgmap v388: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:05.047 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:04 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4203983632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:05.112 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:04 smithi063 ceph-mon[29809]: pgmap v388: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:05.112 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:04 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4203983632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:05.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:04 smithi071 ceph-mon[33293]: pgmap v388: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:05.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:04 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4203983632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:05.521 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:29:06.002 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:06 smithi063 ceph-mon[29809]: pgmap v389: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:07.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:06 smithi165 ceph-mon[33235]: pgmap v389: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:07.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:06 smithi071 ceph-mon[33293]: pgmap v389: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:08.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 -- 172.21.15.63:0/2946474721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c103800 msgr2=0x7fc49c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2946474721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c103800 0x7fc49c103be0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0099c0 tx=0x7fc48c009cd0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 -- 172.21.15.63:0/2946474721 shutdown_connections 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2946474721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c108900 0x7fc49c110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2946474721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c103800 0x7fc49c103be0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2946474721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc49c1041b0 0x7fc49c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 -- 172.21.15.63:0/2946474721 >> 172.21.15.63:0/2946474721 conn(0x7fc49c07a6b0 msgr2=0x7fc49c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 -- 172.21.15.63:0/2946474721 shutdown_connections 2024-05-11T13:29:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.231+0000 7fc4a3400700 1 -- 172.21.15.63:0/2946474721 wait complete. 2024-05-11T13:29:08.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.232+0000 7fc4a3400700 1 Processor -- start 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.232+0000 7fc4a3400700 1 -- start start 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a3400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c103800 0x7fc49c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc49c1041b0 0x7fc49c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c108900 0x7fc49c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a3400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1135f0 con 0x7fc49c108900 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a3400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c113470 con 0x7fc49c1041b0 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a3400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c113770 con 0x7fc49c103800 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c108900 0x7fc49c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c108900 0x7fc49c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58396/0 (socket says 172.21.15.63:58396) 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.233+0000 7fc4a1400700 1 -- 172.21.15.63:0/2887662022 learned_addr learned my addr 172.21.15.63:0/2887662022 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:08.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc493e00700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc49c1041b0 0x7fc49c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:08.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a0a00700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c103800 0x7fc49c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:08.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a1400700 1 -- 172.21.15.63:0/2887662022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c103800 msgr2=0x7fc49c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a1400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c103800 0x7fc49c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a1400700 1 -- 172.21.15.63:0/2887662022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc49c1041b0 msgr2=0x7fc49c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a1400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc49c1041b0 0x7fc49c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a1400700 1 -- 172.21.15.63:0/2887662022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c19a090 con 0x7fc49c108900 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a0a00700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c103800 0x7fc49c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc4a1400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c108900 0x7fc49c199930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc494007ee0 tx=0x7fc49400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.234+0000 7fc49a000700 1 -- 172.21.15.63:0/2887662022 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49400dbd0 con 0x7fc49c108900 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.235+0000 7fc49a000700 1 -- 172.21.15.63:0/2887662022 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc49400dd70 con 0x7fc49c108900 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.235+0000 7fc49a000700 1 -- 172.21.15.63:0/2887662022 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940058d0 con 0x7fc49c108900 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.235+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c19a380 con 0x7fc49c108900 2024-05-11T13:29:08.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.235+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c1a6640 con 0x7fc49c108900 2024-05-11T13:29:08.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.236+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c10bb40 con 0x7fc49c108900 2024-05-11T13:29:08.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.238+0000 7fc49a000700 1 -- 172.21.15.63:0/2887662022 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc49401a020 con 0x7fc49c108900 2024-05-11T13:29:08.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.240+0000 7fc49a000700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc4800b2900 0x7fc4800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:08.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.240+0000 7fc49a000700 1 -- 172.21.15.63:0/2887662022 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc4940e1c80 con 0x7fc49c108900 2024-05-11T13:29:08.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.240+0000 7fc4a0a00700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc4800b2900 0x7fc4800b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:08.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.241+0000 7fc4a0a00700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc4800b2900 0x7fc4800b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc49c19aa70 tx=0x7fc48c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:08.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.245+0000 7fc49a000700 1 -- 172.21.15.63:0/2887662022 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc494010070 con 0x7fc49c108900 2024-05-11T13:29:08.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.495+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc49c10b520 con 0x7fc49c108900 2024-05-11T13:29:08.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.498+0000 7fc49a000700 1 -- 172.21.15.63:0/2887662022 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24092 (secure 0 0 0) 0x7fc4940a9700 con 0x7fc49c108900 2024-05-11T13:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":86,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:28:56.666760+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:29:08.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.501+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc4800b2900 msgr2=0x7fc4800b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:08.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.501+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc4800b2900 0x7fc4800b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc49c19aa70 tx=0x7fc48c01a040 comp rx=0 tx=0).stop 2024-05-11T13:29:08.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.501+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c108900 msgr2=0x7fc49c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:08.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.501+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c108900 0x7fc49c199930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc494007ee0 tx=0x7fc49400d3b0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 shutdown_connections 2024-05-11T13:29:08.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc4800b2900 0x7fc4800b4dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc49c108900 0x7fc49c199930 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c103800 0x7fc49c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 --2- 172.21.15.63:0/2887662022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc49c1041b0 0x7fc49c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:08.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 >> 172.21.15.63:0/2887662022 conn(0x7fc49c07a6b0 msgr2=0x7fc49c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:08.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 shutdown_connections 2024-05-11T13:29:08.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:08.502+0000 7fc4a3400700 1 -- 172.21.15.63:0/2887662022 wait complete. 2024-05-11T13:29:09.030 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:08 smithi063 ceph-mon[29809]: pgmap v390: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:09.031 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:08 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2887662022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:09.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:08 smithi165 ceph-mon[33235]: pgmap v390: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:09.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:08 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2887662022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:09.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:08 smithi071 ceph-mon[33293]: pgmap v390: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:09.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:08 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2887662022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:09.562 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:29:10.037 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:11.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:10 smithi165 ceph-mon[33235]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:11.176 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:10 smithi063 ceph-mon[29809]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:11.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:10 smithi071 ceph-mon[33293]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.565+0000 7efd99000700 1 -- 172.21.15.63:0/2278563238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 msgr2=0x7efd94105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.565+0000 7efd99000700 1 --2- 172.21.15.63:0/2278563238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd94105c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efd80009990 tx=0x7efd80009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.566+0000 7efd99000700 1 -- 172.21.15.63:0/2278563238 shutdown_connections 2024-05-11T13:29:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.566+0000 7efd99000700 1 --2- 172.21.15.63:0/2278563238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd94106150 0x7efd94112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.566+0000 7efd99000700 1 --2- 172.21.15.63:0/2278563238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd9410b790 0x7efd9410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.566+0000 7efd99000700 1 --2- 172.21.15.63:0/2278563238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd94105c10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.566+0000 7efd99000700 1 -- 172.21.15.63:0/2278563238 >> 172.21.15.63:0/2278563238 conn(0x7efd94101280 msgr2=0x7efd941036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.566+0000 7efd99000700 1 -- 172.21.15.63:0/2278563238 shutdown_connections 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.566+0000 7efd99000700 1 -- 172.21.15.63:0/2278563238 wait complete. 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 Processor -- start 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 -- start start 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd9419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd94106150 0x7efd9419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd9410b790 0x7efd941a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd94115870 con 0x7efd94105790 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd941156f0 con 0x7efd9410b790 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.567+0000 7efd99000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd941159f0 con 0x7efd94106150 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd9419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd9419bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48974/0 (socket says 172.21.15.63:48974) 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 -- 172.21.15.63:0/2318289214 learned_addr learned my addr 172.21.15.63:0/2318289214 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91400700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd94106150 0x7efd9419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd92800700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd9410b790 0x7efd941a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 -- 172.21.15.63:0/2318289214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd94106150 msgr2=0x7efd9419c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd94106150 0x7efd9419c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 -- 172.21.15.63:0/2318289214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd9410b790 msgr2=0x7efd941a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd9410b790 0x7efd941a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.568+0000 7efd91e00700 1 -- 172.21.15.63:0/2318289214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd941a0d80 con 0x7efd94105790 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.569+0000 7efd91e00700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd9419bdc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efd8400d950 tx=0x7efd8400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.569+0000 7efd8aa00700 1 -- 172.21.15.63:0/2318289214 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84016070 con 0x7efd94105790 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.569+0000 7efd8aa00700 1 -- 172.21.15.63:0/2318289214 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd84010460 con 0x7efd94105790 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.569+0000 7efd8aa00700 1 -- 172.21.15.63:0/2318289214 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8400f620 con 0x7efd94105790 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.569+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd941a1070 con 0x7efd94105790 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.569+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941a88f0 con 0x7efd94105790 2024-05-11T13:29:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.570+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9406a240 con 0x7efd94105790 2024-05-11T13:29:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.572+0000 7efd8aa00700 1 -- 172.21.15.63:0/2318289214 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efd84019070 con 0x7efd94105790 2024-05-11T13:29:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.574+0000 7efd8aa00700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efd780b2900 0x7efd780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.574+0000 7efd8aa00700 1 -- 172.21.15.63:0/2318289214 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7efd840e1fc0 con 0x7efd94105790 2024-05-11T13:29:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.574+0000 7efd91400700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efd780b2900 0x7efd780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.575+0000 7efd91400700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efd780b2900 0x7efd780b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd8000b5c0 tx=0x7efd8001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.579+0000 7efd8aa00700 1 -- 172.21.15.63:0/2318289214 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd840a9a40 con 0x7efd94105790 2024-05-11T13:29:12.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.869+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd9402e730 con 0x7efd94105790 2024-05-11T13:29:12.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.871+0000 7efd8aa00700 1 -- 172.21.15.63:0/2318289214 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7efd840a9340 con 0x7efd94105790 2024-05-11T13:29:12.871 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:12.872 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34403,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:28:56.667019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34472,"qdb_cluster":[34472]},"id":1}]} 2024-05-11T13:29:12.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efd780b2900 msgr2=0x7efd780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efd780b2900 0x7efd780b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd8000b5c0 tx=0x7efd8001a040 comp rx=0 tx=0).stop 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 msgr2=0x7efd9419bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd9419bdc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efd8400d950 tx=0x7efd8400dd10 comp rx=0 tx=0).stop 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 shutdown_connections 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efd780b2900 0x7efd780b4dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd9410b790 0x7efd941a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd94105790 0x7efd9419bdc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 --2- 172.21.15.63:0/2318289214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd94106150 0x7efd9419c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 >> 172.21.15.63:0/2318289214 conn(0x7efd94101280 msgr2=0x7efd94101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 shutdown_connections 2024-05-11T13:29:12.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:12.874+0000 7efd99000700 1 -- 172.21.15.63:0/2318289214 wait complete. 2024-05-11T13:29:12.876 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 17 2024-05-11T13:29:13.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:12 smithi165 ceph-mon[33235]: pgmap v392: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:13.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:12 smithi063 ceph-mon[29809]: pgmap v392: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:13.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:12 smithi071 ceph-mon[33293]: pgmap v392: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:13.876 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:29:14.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:13 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2318289214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:29:14.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:13 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2318289214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:29:14.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:13 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2318289214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:29:14.337 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:15.000 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:14 smithi063 ceph-mon[29809]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:15.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:14 smithi165 ceph-mon[33235]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:15.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:14 smithi071 ceph-mon[33293]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 -- 172.21.15.63:0/2035159868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84641095c0 msgr2=0x7f84641099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 --2- 172.21.15.63:0/2035159868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84641095c0 0x7f84641099a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8454009990 tx=0x7f8454009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 -- 172.21.15.63:0/2035159868 shutdown_connections 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 --2- 172.21.15.63:0/2035159868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846406bbf0 0x7f84641103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 --2- 172.21.15.63:0/2035159868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84641095c0 0x7f84641099a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 --2- 172.21.15.63:0/2035159868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846406b230 0x7f846406b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 -- 172.21.15.63:0/2035159868 >> 172.21.15.63:0/2035159868 conn(0x7f8464079400 msgr2=0x7f8464079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 -- 172.21.15.63:0/2035159868 shutdown_connections 2024-05-11T13:29:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.768+0000 7f8469600700 1 -- 172.21.15.63:0/2035159868 wait complete. 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.769+0000 7f8469600700 1 Processor -- start 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.769+0000 7f8469600700 1 -- start start 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8469600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846406b230 0x7f846419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8469600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846406bbf0 0x7f846419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8469600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84641095c0 0x7f84641a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8469600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8464113600 con 0x7f846406b230 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8469600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8464113480 con 0x7f846406bbf0 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8469600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8464113780 con 0x7f84641095c0 2024-05-11T13:29:16.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8462a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846406b230 0x7f846419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8462a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846406b230 0x7f846419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48992/0 (socket says 172.21.15.63:48992) 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.770+0000 7f8462a00700 1 -- 172.21.15.63:0/3150106722 learned_addr learned my addr 172.21.15.63:0/3150106722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f845be00700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846406bbf0 0x7f846419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8463400700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84641095c0 0x7f84641a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8462a00700 1 -- 172.21.15.63:0/3150106722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84641095c0 msgr2=0x7f84641a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8462a00700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84641095c0 0x7f84641a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8462a00700 1 -- 172.21.15.63:0/3150106722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846406bbf0 msgr2=0x7f846419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8462a00700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846406bbf0 0x7f846419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8462a00700 1 -- 172.21.15.63:0/3150106722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84641a0d90 con 0x7f846406b230 2024-05-11T13:29:16.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8463400700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84641095c0 0x7f84641a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:29:16.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.771+0000 7f8462a00700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846406b230 0x7f846419bd80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84540098e0 tx=0x7f84540057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:16.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.772+0000 7f845b400700 1 -- 172.21.15.63:0/3150106722 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454027070 con 0x7f846406b230 2024-05-11T13:29:16.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.772+0000 7f845b400700 1 -- 172.21.15.63:0/3150106722 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8454022470 con 0x7f846406b230 2024-05-11T13:29:16.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.772+0000 7f845b400700 1 -- 172.21.15.63:0/3150106722 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454017650 con 0x7f846406b230 2024-05-11T13:29:16.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.772+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84641a1020 con 0x7f846406b230 2024-05-11T13:29:16.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.772+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641a8900 con 0x7f846406b230 2024-05-11T13:29:16.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.773+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846406a240 con 0x7f846406b230 2024-05-11T13:29:16.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.776+0000 7f845b400700 1 -- 172.21.15.63:0/3150106722 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f845402a070 con 0x7f846406b230 2024-05-11T13:29:16.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.777+0000 7f845b400700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f84480b2880 0x7f84480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:16.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.777+0000 7f845b400700 1 -- 172.21.15.63:0/3150106722 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f84540e35b0 con 0x7f846406b230 2024-05-11T13:29:16.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.777+0000 7f845be00700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f84480b2880 0x7f84480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:16.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.778+0000 7f845be00700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f84480b2880 0x7f84480b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84641045f0 tx=0x7f844c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:16.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:16.780+0000 7f845b400700 1 -- 172.21.15.63:0/3150106722 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8454031080 con 0x7f846406b230 2024-05-11T13:29:16.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:16 smithi063 ceph-mon[29809]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:17.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.020+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84641a8ad0 con 0x7f846406b230 2024-05-11T13:29:17.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.022+0000 7f845b400700 1 -- 172.21.15.63:0/3150106722 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24092 (secure 0 0 0) 0x7f8454013070 con 0x7f846406b230 2024-05-11T13:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":86,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:28:56.666760+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:29:17.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.025+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f84480b2880 msgr2=0x7f84480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:17.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.025+0000 7f8469600700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f84480b2880 0x7f84480b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84641045f0 tx=0x7f844c009260 comp rx=0 tx=0).stop 2024-05-11T13:29:17.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.025+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846406b230 msgr2=0x7f846419bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:17.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.025+0000 7f8469600700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846406b230 0x7f846419bd80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84540098e0 tx=0x7f84540057c0 comp rx=0 tx=0).stop 2024-05-11T13:29:17.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 shutdown_connections 2024-05-11T13:29:17.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f84480b2880 0x7f84480b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:17.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84641095c0 0x7f84641a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:17.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846406b230 0x7f846419bd80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:17.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 --2- 172.21.15.63:0/3150106722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846406bbf0 0x7f846419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:17.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 >> 172.21.15.63:0/3150106722 conn(0x7f8464079400 msgr2=0x7f846410e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:17.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 shutdown_connections 2024-05-11T13:29:17.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:17.026+0000 7f8469600700 1 -- 172.21.15.63:0/3150106722 wait complete. 2024-05-11T13:29:17.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:16 smithi165 ceph-mon[33235]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:17.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:16 smithi071 ceph-mon[33293]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:17.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T13:28:38.143704+0000', 'modified': '2024-05-11T13:28:56.667019+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34472': {'gid': 34472, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.71:6839/1525017608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1525017608}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1525017608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34472, 'qdb_cluster': [34472]} 2024-05-11T13:29:17.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T13:29:17.845 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-11T13:29:18.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:17 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3150106722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:18.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:17 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3150106722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:18.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:17 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3150106722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:18.325 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:18 smithi165 ceph-mon[33235]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:19.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:18 smithi063 ceph-mon[29809]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:19.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:18 smithi071 ceph-mon[33293]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 -- 172.21.15.63:0/4110688548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 msgr2=0x7f7fa810faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 --2- 172.21.15.63:0/4110688548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa810faf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009990 tx=0x7f7f94009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 -- 172.21.15.63:0/4110688548 shutdown_connections 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 --2- 172.21.15.63:0/4110688548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa8110030 0x7f7fa8112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 --2- 172.21.15.63:0/4110688548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fa806bc80 0x7f7fa806c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 --2- 172.21.15.63:0/4110688548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa810faf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 -- 172.21.15.63:0/4110688548 >> 172.21.15.63:0/4110688548 conn(0x7f7fa8079400 msgr2=0x7f7fa8079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 -- 172.21.15.63:0/4110688548 shutdown_connections 2024-05-11T13:29:20.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.605+0000 7f7faca00700 1 -- 172.21.15.63:0/4110688548 wait complete. 2024-05-11T13:29:20.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.606+0000 7f7faca00700 1 Processor -- start 2024-05-11T13:29:20.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.607+0000 7f7faca00700 1 -- start start 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.607+0000 7f7faca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa806bc80 0x7f7fa819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.607+0000 7f7faca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.607+0000 7f7faca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fa8110030 0x7f7fa81a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.607+0000 7f7faca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8115840 con 0x7f7fa806c5a0 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.607+0000 7f7faca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81156c0 con 0x7f7fa806bc80 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.607+0000 7f7faca00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81159c0 con 0x7f7fa8110030 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59976/0 (socket says 172.21.15.63:59976) 2024-05-11T13:29:20.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 -- 172.21.15.63:0/1428695446 learned_addr learned my addr 172.21.15.63:0/1428695446 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:20.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa806bc80 0x7f7fa819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fa8110030 0x7f7fa81a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 -- 172.21.15.63:0/1428695446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fa8110030 msgr2=0x7f7fa81a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fa8110030 0x7f7fa81a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 -- 172.21.15.63:0/1428695446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa806bc80 msgr2=0x7f7fa819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa806bc80 0x7f7fa819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.608+0000 7f7fa5400700 1 -- 172.21.15.63:0/1428695446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa81a0ec0 con 0x7f7fa806c5a0 2024-05-11T13:29:20.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.609+0000 7f7fa5400700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa819c3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f94005910 tx=0x7f7f940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:20.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.609+0000 7f7f9ea00700 1 -- 172.21.15.63:0/1428695446 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94027070 con 0x7f7fa806c5a0 2024-05-11T13:29:20.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.609+0000 7f7f9ea00700 1 -- 172.21.15.63:0/1428695446 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f9400fc00 con 0x7f7fa806c5a0 2024-05-11T13:29:20.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.609+0000 7f7f9ea00700 1 -- 172.21.15.63:0/1428695446 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94022d40 con 0x7f7fa806c5a0 2024-05-11T13:29:20.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.609+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa81a1150 con 0x7f7fa806c5a0 2024-05-11T13:29:20.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.609+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa81a8a30 con 0x7f7fa806c5a0 2024-05-11T13:29:20.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.613+0000 7f7f9ea00700 1 -- 172.21.15.63:0/1428695446 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7f9402a070 con 0x7f7fa806c5a0 2024-05-11T13:29:20.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 Processor -- start 2024-05-11T13:29:20.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 -- start start 2024-05-11T13:29:20.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f6b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f6f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f6fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7f9ea00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f7f8c0b2900 0x7f7f8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3ebd9d0 con 0x55b3e3f6ba40 2024-05-11T13:29:20.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3e5b710 con 0x55b3e3f6b120 2024-05-11T13:29:20.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.615+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3eb9990 con 0x55b3e3f6fa90 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.616+0000 7f7f9ea00700 1 -- 172.21.15.63:0/1428695446 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f7f940e3180 con 0x7f7fa806c5a0 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.616+0000 7f7fa5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f6f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.616+0000 7f7fa5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f6f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59988/0 (socket says 172.21.15.63:59988) 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.616+0000 7f7fa5e00700 1 -- 172.21.15.63:0/1883277851 learned_addr learned my addr 172.21.15.63:0/1883277851 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.616+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f6b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.616+0000 7f7fa5400700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f7f8c0b2900 0x7f7f8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.616+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f6fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa5e00700 1 -- 172.21.15.63:0/1883277851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6fa90 msgr2=0x55b3e3f6fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f6fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa5e00700 1 -- 172.21.15.63:0/1883277851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 msgr2=0x55b3e3f6b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f6b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa5e00700 1 -- 172.21.15.63:0/1883277851 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e3e0a880 con 0x55b3e3f6ba40 2024-05-11T13:29:20.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f6b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:29:20.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa5400700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f7f8c0b2900 0x7f7f8c0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f940aa290 tx=0x7f7f94005ac0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:20.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f6f550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800bd30 tx=0x7f7f9800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483884e04c56596d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7f93400700 1 -- 172.21.15.63:0/1883277851 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98015070 con 0x55b3e3f6ba40 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7f93400700 1 -- 172.21.15.63:0/1883277851 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f9800fda0 con 0x55b3e3f6ba40 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.617+0000 7f7f93400700 1 -- 172.21.15.63:0/1883277851 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f980055b0 con 0x55b3e3f6ba40 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1883277851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6ba40 msgr2=0x55b3e3f6f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f6f550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800bd30 tx=0x7f7f9800f640 comp rx=0 tx=0).stop 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1883277851 shutdown_connections 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f6fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f6b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1883277851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f6f550 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1883277851 >> 172.21.15.63:0/1883277851 conn(0x55b3e3e99b10 msgr2=0x55b3e3f65860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:20.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1883277851 shutdown_connections 2024-05-11T13:29:20.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.618+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1883277851 wait complete. 2024-05-11T13:29:20.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.619+0000 7f7fb7470b80 1 -- start start 2024-05-11T13:29:20.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.619+0000 7f7fb7470b80 1 Processor -- start 2024-05-11T13:29:20.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6b120 0x55b3e3f76ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f74810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f77ca0 con 0x55b3e3f6b120 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3e23ad0 con 0x55b3e3f6fa90 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f77e20 con 0x55b3e3f6ba40 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fb7470b80 10 client.0 _ll_register_callbacks cb 0x55b3e3e64c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fa6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6b120 0x55b3e3f76ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fa6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6b120 0x55b3e3f76ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59996/0 (socket says 172.21.15.63:59996) 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fa6800700 1 -- 172.21.15.63:0/3885120624 learned_addr learned my addr 172.21.15.63:0/3885120624 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fa6800700 1 -- 172.21.15.63:0/3885120624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 msgr2=0x55b3e3f742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.620+0000 7f7fa6800700 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7fa6800700 1 -- 172.21.15.63:0/3885120624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6fa90 msgr2=0x55b3e3f74810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7fa6800700 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f74810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7fa6800700 1 -- 172.21.15.63:0/3885120624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e3e833d0 con 0x55b3e3f6b120 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7fa6800700 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6b120 0x55b3e3f76ec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009f10 tx=0x7f7fa0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a138f39ebf260c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7f87e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7f87e00700 1 -- 172.21.15.63:0/3885120624 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa001c020 con 0x55b3e3f6b120 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7f87e00700 1 -- 172.21.15.63:0/3885120624 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f7fa00169c0 con 0x55b3e3f6b120 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.621+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9800fda0 con 0x55b3e3f6b120 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7f87e00700 1 -- 172.21.15.63:0/3885120624 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7fa0016bc0 con 0x55b3e3f6b120 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 10 client.14685 fetch_fsmap learned FSMap version 17 2024-05-11T13:29:20.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55b3e3f825d0 con 0x55b3e3f6b120 2024-05-11T13:29:20.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7f87e00700 1 -- 172.21.15.63:0/3885120624 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f9800fda0 con 0x55b3e3f6b120 2024-05-11T13:29:20.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7f87e00700 1 -- 172.21.15.63:0/3885120624 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7fa00203a0 con 0x55b3e3f6b120 2024-05-11T13:29:20.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 10 client.14685 fetch_fsmap finished waiting for FSMap version 17 2024-05-11T13:29:20.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 10 client.14685 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:29:20.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 20 client.14685 populate_metadata read hostname 'smithi063' 2024-05-11T13:29:20.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3f84c10 0x55b3e3f86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 4 client.14685 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:29:20.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.622+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3e3f38160 con 0x55b3e3f84c10 2024-05-11T13:29:20.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.623+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3f84c10 0x55b3e3f86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.624+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3f84c10 0x55b3e3f86080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2550818efb9249e9 server_cookie=62c043a96e34b763 in_seq=0 out_seq=0 2024-05-11T13:29:20.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.624+0000 7f7f87e00700 10 client.14685 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:29:20.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7f87e00700 1 -- 172.21.15.63:0/3885120624 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b3e3f38160 con 0x55b3e3f84c10 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7f87e00700 10 client.14685 handle_command_reply: tid=0 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 client.14685 shutdown 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6b120 msgr2=0x55b3e3f76ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6b120 0x55b3e3f76ec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009f10 tx=0x7f7fa0007dd0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 shutdown_connections 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3f84c10 0x55b3e3f86080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f74810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6b120 0x55b3e3f76ec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3885120624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7f87e00700 0 client.14685 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 >> 172.21.15.63:0/3885120624 conn(0x55b3e3e99b10 msgr2=0x55b3e3f60370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.627+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 shutdown_connections 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.628+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3885120624 wait complete. 2024-05-11T13:29:20.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.628+0000 7f7fb7470b80 20 client.14685 trim_cache size 0 max 16384 2024-05-11T13:29:20.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.652+0000 7f7fb7470b80 1 Processor -- start 2024-05-11T13:29:20.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.652+0000 7f7fb7470b80 1 -- start start 2024-05-11T13:29:20.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f71ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f72bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f73100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f843c0 con 0x55b3e3f6fa90 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f84a90 con 0x55b3e3f6b120 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f85160 con 0x55b3e3f6ba40 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f73100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f73100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60006/0 (socket says 172.21.15.63:60006) 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 -- 172.21.15.63:0/3906800968 learned_addr learned my addr 172.21.15.63:0/3906800968 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f72bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 -- 172.21.15.63:0/3906800968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 msgr2=0x55b3e3f72bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f72bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 -- 172.21.15.63:0/3906800968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 msgr2=0x55b3e3f71ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f71ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.653+0000 7f7fa6800700 1 -- 172.21.15.63:0/3906800968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e3f72840 con 0x55b3e3f6fa90 2024-05-11T13:29:20.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.654+0000 7f7fa6800700 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f73100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0007fd0 tx=0x7f7fa0006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad38829dfd42d7e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:20.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.654+0000 7f7f92a00700 1 -- 172.21.15.63:0/3906800968 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa001c070 con 0x55b3e3f6fa90 2024-05-11T13:29:20.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.654+0000 7f7f92a00700 1 -- 172.21.15.63:0/3906800968 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fa0016d60 con 0x55b3e3f6fa90 2024-05-11T13:29:20.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.654+0000 7f7f92a00700 1 -- 172.21.15.63:0/3906800968 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0018650 con 0x55b3e3f6fa90 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3906800968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 msgr2=0x55b3e3f73100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f73100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0007fd0 tx=0x7f7fa0006040 comp rx=0 tx=0).stop 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3906800968 shutdown_connections 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f73100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f71ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/3906800968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f72bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3906800968 >> 172.21.15.63:0/3906800968 conn(0x55b3e3e99b10 msgr2=0x55b3e3f74290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3906800968 shutdown_connections 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 -- 172.21.15.63:0/3906800968 wait complete. 2024-05-11T13:29:20.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.655+0000 7f7fb7470b80 1 -- start start 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 1 Processor -- start 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f8f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f8fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f85f00 con 0x55b3e3f6fa90 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f85830 con 0x55b3e3f6b120 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3f865d0 con 0x55b3e3f6ba40 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.656+0000 7f7fb7470b80 10 client.0 _ll_register_callbacks cb 0x55b3e3e64c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:29:20.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f884a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60008/0 (socket says 172.21.15.63:60008) 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 -- 172.21.15.63:0/1859396902 learned_addr learned my addr 172.21.15.63:0/1859396902 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f8fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f8f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 -- 172.21.15.63:0/1859396902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 msgr2=0x55b3e3f8fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f8fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 -- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 msgr2=0x55b3e3f8f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f8f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.657+0000 7f7fa6800700 1 -- 172.21.15.63:0/1859396902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e3f88c40 con 0x55b3e3f6fa90 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7fa6800700 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f884a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0007840 tx=0x7f7fa0007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d2ab67709f52c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7f87e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7f87e00700 1 -- 172.21.15.63:0/1859396902 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa001c070 con 0x55b3e3f6fa90 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa0016d60 con 0x55b3e3f6fa90 2024-05-11T13:29:20.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7f87e00700 1 -- 172.21.15.63:0/1859396902 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f7fa0002030 con 0x55b3e3f6fa90 2024-05-11T13:29:20.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7f87e00700 1 -- 172.21.15.63:0/1859396902 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7fa0004b40 con 0x55b3e3f6fa90 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7fb7470b80 10 client.14691 fetch_fsmap learned FSMap version 17 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.658+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55b3e3ff92d0 con 0x55b3e3f6fa90 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7f87e00700 1 -- 172.21.15.63:0/1859396902 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fa0016d60 con 0x55b3e3f6fa90 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7f87e00700 1 -- 172.21.15.63:0/1859396902 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7fa0005020 con 0x55b3e3f6fa90 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7fb7470b80 10 client.14691 fetch_fsmap finished waiting for FSMap version 17 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7fb7470b80 10 client.14691 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7fb7470b80 20 client.14691 populate_metadata read hostname 'smithi063' 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3ffda10 0x55b3e3ffc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7fb7470b80 4 client.14691 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.659+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "status"}) -- 0x55b3e3ecb3b0 con 0x55b3e3ffda10 2024-05-11T13:29:20.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.660+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3ffda10 0x55b3e3ffc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:20.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.661+0000 7f7fa5e00700 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3ffda10 0x55b3e3ffc720 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26c7fb198b8889c server_cookie=9ca6fd563beaeabc in_seq=0 out_seq=0 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.661+0000 7f7f87e00700 10 client.14691 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.661+0000 7f7f87e00700 1 -- 172.21.15.63:0/1859396902 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b3e3ecb3b0 con 0x55b3e3ffda10 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.661+0000 7f7f87e00700 10 client.14691 handle_command_reply: tid=0 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.661+0000 7f7fb7470b80 1 client.14691 shutdown 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 msgr2=0x55b3e3f884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f884a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0007840 tx=0x7f7fa0007a80 comp rx=0 tx=0).stop 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 shutdown_connections 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b3e3f6fa90 0x55b3e3f884a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b3e3ffda10 0x55b3e3ffc720 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3e3f6b120 0x55b3e3f8f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 --2- 172.21.15.63:0/1859396902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3e3f6ba40 0x55b3e3f8fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 >> 172.21.15.63:0/1859396902 conn(0x55b3e3e99b10 msgr2=0x55b3e3f72b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7f87e00700 0 client.14691 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 shutdown_connections 2024-05-11T13:29:20.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 1 -- 172.21.15.63:0/1859396902 wait complete. 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.662+0000 7f7fb7470b80 20 client.14691 trim_cache size 0 max 16384 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 0, 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34472, 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 42.440330637000002, 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 17, 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 86, 2024-05-11T13:29:20.664 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 85, 2024-05-11T13:29:20.665 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 61.618306662000002 2024-05-11T13:29:20.665 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:29:20.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f7f8c0b2900 msgr2=0x7f7f8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f7f8c0b2900 0x7f7f8c0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f940aa290 tx=0x7f7f94005ac0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 msgr2=0x7f7fa819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:20.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa819c3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f94005910 tx=0x7f7f940047b0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 shutdown_connections 2024-05-11T13:29:20.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f7f8c0b2900 0x7f7f8c0b4dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fa8110030 0x7f7fa81a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa806bc80 0x7f7fa819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 --2- 172.21.15.63:0/1428695446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fa806c5a0 0x7f7fa819c3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:20.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 >> 172.21.15.63:0/1428695446 conn(0x7f7fa8079400 msgr2=0x7f7fa8111770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:20.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 shutdown_connections 2024-05-11T13:29:20.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:20.663+0000 7f7faca00700 1 -- 172.21.15.63:0/1428695446 wait complete. 2024-05-11T13:29:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:20 smithi063 ceph-mon[29809]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:21.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:20 smithi071 ceph-mon[33293]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:21.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:20 smithi165 ceph-mon[33235]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:21.628 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T13:29:21.629 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:21.629 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-11T13:29:22.100 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:22 smithi063 ceph-mon[29809]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:23.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:22 smithi165 ceph-mon[33235]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:23.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:22 smithi071 ceph-mon[33293]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 -- 172.21.15.63:0/34986203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc5810ff50 msgr2=0x7efc58112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 --2- 172.21.15.63:0/34986203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc5810ff50 0x7efc58112340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b0a0 tx=0x7efc5000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 -- 172.21.15.63:0/34986203 shutdown_connections 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 --2- 172.21.15.63:0/34986203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc5810ff50 0x7efc58112340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 --2- 172.21.15.63:0/34986203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc58075780 0x7efc58075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 --2- 172.21.15.63:0/34986203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc580760a0 0x7efc5810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 -- 172.21.15.63:0/34986203 >> 172.21.15.63:0/34986203 conn(0x7efc5807a6b0 msgr2=0x7efc5807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 -- 172.21.15.63:0/34986203 shutdown_connections 2024-05-11T13:29:24.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.389+0000 7efc5ca00700 1 -- 172.21.15.63:0/34986203 wait complete. 2024-05-11T13:29:24.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.391+0000 7efc5ca00700 1 Processor -- start 2024-05-11T13:29:24.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.391+0000 7efc5ca00700 1 -- start start 2024-05-11T13:29:24.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.392+0000 7efc5ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc58075780 0x7efc5819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:24.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.392+0000 7efc5ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc580760a0 0x7efc5819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:24.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.392+0000 7efc5ca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc5810ff50 0x7efc581a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:24.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.392+0000 7efc5ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc581157d0 con 0x7efc580760a0 2024-05-11T13:29:24.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.392+0000 7efc5ca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc58115650 con 0x7efc58075780 2024-05-11T13:29:24.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.392+0000 7efc5ca00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc58115950 con 0x7efc5810ff50 2024-05-11T13:29:24.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.392+0000 7efc4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc580760a0 0x7efc5819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:24.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc580760a0 0x7efc5819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60022/0 (socket says 172.21.15.63:60022) 2024-05-11T13:29:24.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 -- 172.21.15.63:0/2936311622 learned_addr learned my addr 172.21.15.63:0/2936311622 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:24.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc55e00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc58075780 0x7efc5819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:24.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc56800700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc5810ff50 0x7efc581a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:24.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 -- 172.21.15.63:0/2936311622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc5810ff50 msgr2=0x7efc581a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc5810ff50 0x7efc581a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 -- 172.21.15.63:0/2936311622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc58075780 msgr2=0x7efc5819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc58075780 0x7efc5819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 -- 172.21.15.63:0/2936311622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc581a0d30 con 0x7efc580760a0 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc55e00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc58075780 0x7efc5819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.393+0000 7efc4fe00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc580760a0 0x7efc5819c260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc4000eb30 tx=0x7efc4000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.394+0000 7efc4ea00700 1 -- 172.21.15.63:0/2936311622 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4000cd40 con 0x7efc580760a0 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.394+0000 7efc4ea00700 1 -- 172.21.15.63:0/2936311622 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc400044b0 con 0x7efc580760a0 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.394+0000 7efc4ea00700 1 -- 172.21.15.63:0/2936311622 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40010430 con 0x7efc580760a0 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.394+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc581a1020 con 0x7efc580760a0 2024-05-11T13:29:24.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.394+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc581a88a0 con 0x7efc580760a0 2024-05-11T13:29:24.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.395+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5806a240 con 0x7efc580760a0 2024-05-11T13:29:24.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.397+0000 7efc4ea00700 1 -- 172.21.15.63:0/2936311622 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efc40003f10 con 0x7efc580760a0 2024-05-11T13:29:24.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.399+0000 7efc4ea00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc3c0b2900 0x7efc3c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:24.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.399+0000 7efc4ea00700 1 -- 172.21.15.63:0/2936311622 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7efc400e2750 con 0x7efc580760a0 2024-05-11T13:29:24.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.399+0000 7efc55e00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc3c0b2900 0x7efc3c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:24.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.400+0000 7efc55e00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc3c0b2900 0x7efc3c0b4dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc48009cb0 tx=0x7efc48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:24.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.402+0000 7efc4ea00700 1 -- 172.21.15.63:0/2936311622 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc400aa260 con 0x7efc580760a0 2024-05-11T13:29:24.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.689+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efc5802e570 con 0x7efc580760a0 2024-05-11T13:29:24.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.693+0000 7efc4ea00700 1 -- 172.21.15.63:0/2936311622 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7efc400a9b60 con 0x7efc580760a0 2024-05-11T13:29:24.693 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.696+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc3c0b2900 msgr2=0x7efc3c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.696+0000 7efc5ca00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc3c0b2900 0x7efc3c0b4dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc48009cb0 tx=0x7efc48009260 comp rx=0 tx=0).stop 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.696+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc580760a0 msgr2=0x7efc5819c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.696+0000 7efc5ca00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc580760a0 0x7efc5819c260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc4000eb30 tx=0x7efc4000eef0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 shutdown_connections 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc3c0b2900 0x7efc3c0b4dc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc5810ff50 0x7efc581a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc58075780 0x7efc5819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 --2- 172.21.15.63:0/2936311622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc580760a0 0x7efc5819c260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:24.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 >> 172.21.15.63:0/2936311622 conn(0x7efc5807a6b0 msgr2=0x7efc58100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:24.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 shutdown_connections 2024-05-11T13:29:24.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:24.697+0000 7efc5ca00700 1 -- 172.21.15.63:0/2936311622 wait complete. 2024-05-11T13:29:25.075 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:24 smithi063 ceph-mon[29809]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:25.075 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:24 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2936311622' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T13:29:25.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:24 smithi071 ceph-mon[33293]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:25.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:24 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2936311622' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T13:29:25.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:24 smithi165 ceph-mon[33235]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:25.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:24 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2936311622' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T13:29:25.510 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-11T13:29:25.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:25.511 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-11T13:29:25.569 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:29:25.570 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T13:29:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:29:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T13:29:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 41884sec preferred_lft 41884sec 2024-05-11T13:29:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-11T13:29:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-11T13:29:25.571 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T13:29:25.572 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:29:25.572 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:29:25.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-11T13:29:25.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-11T13:29:25.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-11T13:29:25.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T13:29:25.572 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:29:25.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:29:26.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:29:26.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:26.054 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T13:29:26.127 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-11T13:29:26.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:26.130 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-11T13:29:26.190 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-11T13:29:26.190 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T13:29:26.190 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T13:29:26.190 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T13:29:26.190 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T13:29:26.191 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:29:26.191 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:29:26.191 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T13:29:26.191 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T13:29:26.191 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T13:29:26.191 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:29:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:29:26.918 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:26 smithi063 ceph-mon[29809]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:29:26.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:26.933 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-11T13:29:26.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:26.996 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-11T13:29:27.059 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:29:27.060 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:29:27.060 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:29:27.060 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-11T13:29:27.060 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:29:27.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:29:27.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:26 smithi071 ceph-mon[33293]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:29:27.263 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-11T13:29:27.264 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:29:27.264 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:29:27.264 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-11T13:29:27.264 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T13:29:27.264 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-11T13:29:27.264 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-11T13:29:27.265 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-11T13:29:27.265 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:29:27.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:26 smithi165 ceph-mon[33235]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:27.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:29:27.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:29:27.572 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:29:27.572 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:29:27.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-11T13:29:27.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T13:29:27.572 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:29:27.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:29:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:29:27.788 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-11T13:29:27.788 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T13:29:27.788 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:27.838 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-11T13:29:27.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T13:29:27.839 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:27.886 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-11T13:29:27.960 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-11T13:29:28.016 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T13:29:28.017 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/066ac03fb5ca3e1a8dadc84d5f3ab52a45c9209b1c80d02c848ede682c25ff0d/merged 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/72df2fc077412c4e4af695ef9f198bb31d7e3c052c9e160f9bd0d4dc6cd70b33/merged 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/40150c9075cef353a4533debcf913c7d6bd23da52bd9ce25a594c656f983a864/merged 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1719d935211d22b6bd790b1ddba7dfc6d15f8a773beb336cedbdecf76e6562a0/merged 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/4fe368d0473a68a1e52e321a508fa25ac11afa55517a21df5d6b356061efcbdd/merged 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/cf2a505ce6db6359598171dad10a47d6b558f1cb809c02159e9df5a8ee193c11/merged 2024-05-11T13:29:28.018 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8616b01baaa8532458654530a421e33810f567d1d8d4cbc136006fe087780e92/merged 2024-05-11T13:29:28.019 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a6fabb9e7c1965dee1e2844d69bec59f189c2c9c82b01d0ec04b6bbe043a49f/merged 2024-05-11T13:29:28.019 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/01b094eceb45788c05f1b4ff90dc262a42dc5269b472183a2f97b3b780d66a0f/merged 2024-05-11T13:29:28.019 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/03a8bd5cbd061b90d08d64b40852822224f9263e682bd0b20ca549b7b7123159/merged 2024-05-11T13:29:28.019 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-11T13:29:28.019 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:29:28.019 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:29:28.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:28.020 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-11T13:29:28.073 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T13:29:28.074 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-11T13:29:28.079 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-11T13:29:28.145 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T13:29:28.190 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-11T13:29:28.190 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-11T13:29:28.190 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-11T13:29:28.190 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-11T13:29:28.190 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-11T13:29:28.190 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T13:29:28.191 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/066ac03fb5ca3e1a8dadc84d5f3ab52a45c9209b1c80d02c848ede682c25ff0d/merged 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/72df2fc077412c4e4af695ef9f198bb31d7e3c052c9e160f9bd0d4dc6cd70b33/merged 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/40150c9075cef353a4533debcf913c7d6bd23da52bd9ce25a594c656f983a864/merged 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1719d935211d22b6bd790b1ddba7dfc6d15f8a773beb336cedbdecf76e6562a0/merged 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/4fe368d0473a68a1e52e321a508fa25ac11afa55517a21df5d6b356061efcbdd/merged 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/cf2a505ce6db6359598171dad10a47d6b558f1cb809c02159e9df5a8ee193c11/merged 2024-05-11T13:29:28.192 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8616b01baaa8532458654530a421e33810f567d1d8d4cbc136006fe087780e92/merged 2024-05-11T13:29:28.193 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a6fabb9e7c1965dee1e2844d69bec59f189c2c9c82b01d0ec04b6bbe043a49f/merged 2024-05-11T13:29:28.193 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/01b094eceb45788c05f1b4ff90dc262a42dc5269b472183a2f97b3b780d66a0f/merged 2024-05-11T13:29:28.193 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/03a8bd5cbd061b90d08d64b40852822224f9263e682bd0b20ca549b7b7123159/merged 2024-05-11T13:29:28.193 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-11T13:29:28.193 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:29:28.193 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:29:28.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:28.194 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-11T13:29:28.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:2024-05-11T13:29:28.281+0000 7f160dd29480 -1 init, newargv = 0x55e2b4793610 newargc=15 2024-05-11T13:29:28.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:ceph-fuse[73352]: starting ceph client 2024-05-11T13:29:28.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:ceph-fuse[73352]: starting fuse 2024-05-11T13:29:29.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:28 smithi063 ceph-mon[29809]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:29.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:28 smithi071 ceph-mon[33293]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:28 smithi165 ceph-mon[33235]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:29.385 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-11T13:29:29.475 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T13:29:29.528 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-11T13:29:29.528 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-11T13:29:29.528 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-11T13:29:29.528 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-11T13:29:29.528 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-11T13:29:29.528 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T13:29:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/066ac03fb5ca3e1a8dadc84d5f3ab52a45c9209b1c80d02c848ede682c25ff0d/merged 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/72df2fc077412c4e4af695ef9f198bb31d7e3c052c9e160f9bd0d4dc6cd70b33/merged 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/40150c9075cef353a4533debcf913c7d6bd23da52bd9ce25a594c656f983a864/merged 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1719d935211d22b6bd790b1ddba7dfc6d15f8a773beb336cedbdecf76e6562a0/merged 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/4fe368d0473a68a1e52e321a508fa25ac11afa55517a21df5d6b356061efcbdd/merged 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/cf2a505ce6db6359598171dad10a47d6b558f1cb809c02159e9df5a8ee193c11/merged 2024-05-11T13:29:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8616b01baaa8532458654530a421e33810f567d1d8d4cbc136006fe087780e92/merged 2024-05-11T13:29:29.531 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a6fabb9e7c1965dee1e2844d69bec59f189c2c9c82b01d0ec04b6bbe043a49f/merged 2024-05-11T13:29:29.531 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/01b094eceb45788c05f1b4ff90dc262a42dc5269b472183a2f97b3b780d66a0f/merged 2024-05-11T13:29:29.531 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/03a8bd5cbd061b90d08d64b40852822224f9263e682bd0b20ca549b7b7123159/merged 2024-05-11T13:29:29.531 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-11T13:29:29.531 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:29:29.531 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:29:29.531 INFO:teuthology.orchestra.run.smithi063.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:29.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:29.532 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-11T13:29:29.584 INFO:teuthology.orchestra.run.smithi063.stdout:114 2024-05-11T13:29:29.585 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-05-11T13:29:29.585 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-05-11T13:29:29.586 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-05-11T13:29:29.587 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-05-11T13:29:29.587 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T13:29:29.587 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:29:29.587 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.admin")) 2024-05-11T13:29:29.587 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-05-11T13:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.admin.73352.asok 2024-05-11T13:29:29.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:29:29.784 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73352.asok 2024-05-11T13:29:29.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:29.784 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73352.asok status 2024-05-11T13:29:30.023 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "308506802b0bdea4df7050cfc1fcbdc04fea6a6e", 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "admin", 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "73352", 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/" 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:29:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 14703, 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 14703 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1745213651 2024-05-11T13:29:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1745213651 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.14703 192.168.0.1:0/1745213651", 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.1:0/1745213651", 2024-05-11T13:29:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 1, 2024-05-11T13:29:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 17, 2024-05-11T13:29:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 86, 2024-05-11T13:29:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-05-11T13:29:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-05-11T13:29:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-05-11T13:29:30.027 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:29:30.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:30.040 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-05-11T13:29:30.040 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:30.087 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-05-11T13:29:30.087 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:30.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:30.088 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:30.226 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-11T13:29:30.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T13:29:30.380 INFO:teuthology.orchestra.run.smithi063.stdout:# file: . 2024-05-11T13:29:30.380 INFO:teuthology.orchestra.run.smithi063.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-11T13:29:30.380 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:30.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:29:30.392 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:29:30.392 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T13:29:30.449 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-11T13:29:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:31.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:30 smithi063 ceph-mon[29809]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:31.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:30 smithi071 ceph-mon[33293]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:31.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:30 smithi165 ceph-mon[33235]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:33.122 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:32 smithi063 ceph-mon[29809]: pgmap v402: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:33.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:32 smithi071 ceph-mon[33293]: pgmap v402: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 -- 172.21.15.63:0/4174635681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 msgr2=0x7f28ec1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 --2- 172.21.15.63:0/4174635681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec1099a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b0a0 tx=0x7f28e000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 -- 172.21.15.63:0/4174635681 shutdown_connections 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 --2- 172.21.15.63:0/4174635681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28ec06bbf0 0x7f28ec1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 --2- 172.21.15.63:0/4174635681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec1099a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 --2- 172.21.15.63:0/4174635681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28ec06b230 0x7f28ec06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 -- 172.21.15.63:0/4174635681 >> 172.21.15.63:0/4174635681 conn(0x7f28ec079400 msgr2=0x7f28ec079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 -- 172.21.15.63:0/4174635681 shutdown_connections 2024-05-11T13:29:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.299+0000 7f28f3e00700 1 -- 172.21.15.63:0/4174635681 wait complete. 2024-05-11T13:29:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.300+0000 7f28f3e00700 1 Processor -- start 2024-05-11T13:29:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f3e00700 1 -- start start 2024-05-11T13:29:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28ec06b230 0x7f28ec19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28ec06bbf0 0x7f28ec19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1135e0 con 0x7f28ec1095c0 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f3e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec113460 con 0x7f28ec06b230 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f3e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec113760 con 0x7f28ec06bbf0 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35560/0 (socket says 172.21.15.63:35560) 2024-05-11T13:29:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.301+0000 7f28f1e00700 1 -- 172.21.15.63:0/1072989552 learned_addr learned my addr 172.21.15.63:0/1072989552 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1400700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28ec06b230 0x7f28ec19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f0a00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28ec06bbf0 0x7f28ec19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1e00700 1 -- 172.21.15.63:0/1072989552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28ec06bbf0 msgr2=0x7f28ec19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28ec06bbf0 0x7f28ec19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1e00700 1 -- 172.21.15.63:0/1072989552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28ec06b230 msgr2=0x7f28ec19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28ec06b230 0x7f28ec19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1e00700 1 -- 172.21.15.63:0/1072989552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec19a0e0 con 0x7f28ec1095c0 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1400700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28ec06b230 0x7f28ec19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.302+0000 7f28f1e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec199980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f28e400d980 tx=0x7f28e400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.303+0000 7f28ea000700 1 -- 172.21.15.63:0/1072989552 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4017070 con 0x7f28ec1095c0 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.303+0000 7f28ea000700 1 -- 172.21.15.63:0/1072989552 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28e4009dd0 con 0x7f28ec1095c0 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.303+0000 7f28ea000700 1 -- 172.21.15.63:0/1072989552 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4005260 con 0x7f28ec1095c0 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.303+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec19a3d0 con 0x7f28ec1095c0 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.303+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a6690 con 0x7f28ec1095c0 2024-05-11T13:29:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.304+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec10baa0 con 0x7f28ec1095c0 2024-05-11T13:29:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.307+0000 7f28ea000700 1 -- 172.21.15.63:0/1072989552 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f28e4009940 con 0x7f28ec1095c0 2024-05-11T13:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.308+0000 7f28ea000700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f28d40b2900 0x7f28d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.308+0000 7f28ea000700 1 -- 172.21.15.63:0/1072989552 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f28e40e27d0 con 0x7f28ec1095c0 2024-05-11T13:29:33.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.308+0000 7f28f1400700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f28d40b2900 0x7f28d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:33.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.309+0000 7f28f1400700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f28d40b2900 0x7f28d40b4dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009e60 tx=0x7f28e000b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:33.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.311+0000 7f28ea000700 1 -- 172.21.15.63:0/1072989552 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28e4020020 con 0x7f28ec1095c0 2024-05-11T13:29:33.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:32 smithi165 ceph-mon[33235]: pgmap v402: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:33.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.565+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28ec06a240 con 0x7f28ec1095c0 2024-05-11T13:29:33.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.567+0000 7f28ea000700 1 -- 172.21.15.63:0/1072989552 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+590 (secure 0 0 0) 0x7f28e40aa2d0 con 0x7f28ec1095c0 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3897275629 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2606154945 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2618629485 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2351804499 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4232658320 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:33.568 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:33.569 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:33.569 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1483448731 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f28d40b2900 msgr2=0x7f28d40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f28d40b2900 0x7f28d40b4dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009e60 tx=0x7f28e000b7f0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 msgr2=0x7f28ec199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec199980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f28e400d980 tx=0x7f28e400dd40 comp rx=0 tx=0).stop 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 shutdown_connections 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f28d40b2900 0x7f28d40b4dc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28ec1095c0 0x7f28ec199980 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28ec06b230 0x7f28ec19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 --2- 172.21.15.63:0/1072989552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28ec06bbf0 0x7f28ec19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.570+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 >> 172.21.15.63:0/1072989552 conn(0x7f28ec079400 msgr2=0x7f28ec1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.571+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 shutdown_connections 2024-05-11T13:29:33.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:33.571+0000 7f28f3e00700 1 -- 172.21.15.63:0/1072989552 wait complete. 2024-05-11T13:29:33.572 INFO:teuthology.orchestra.run.smithi063.stderr:listed 10 entries 2024-05-11T13:29:34.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:33 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1072989552' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:29:34.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:33 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:29:34.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:33 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1072989552' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:29:34.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:33 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:29:34.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:33 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1072989552' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:29:34.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:33 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:29:34.421 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:29:34.421 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T13:29:34.466 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-11T13:29:35.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:34 smithi063 ceph-mon[29809]: pgmap v403: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:35.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:34 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:34 smithi071 ceph-mon[33293]: pgmap v403: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:35.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:34 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:34 smithi165 ceph-mon[33235]: pgmap v403: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:34 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:37.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:36 smithi063 ceph-mon[29809]: pgmap v404: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:37.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:36 smithi071 ceph-mon[33293]: pgmap v404: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:37.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:36 smithi165 ceph-mon[33235]: pgmap v404: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.708+0000 7fbce8200700 1 -- 172.21.15.63:0/2437560411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce010fff0 msgr2=0x7fbce01123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.708+0000 7fbce8200700 1 --2- 172.21.15.63:0/2437560411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce010fff0 0x7fbce01123e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800b0a0 tx=0x7fbcd800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.709+0000 7fbce8200700 1 -- 172.21.15.63:0/2437560411 shutdown_connections 2024-05-11T13:29:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.709+0000 7fbce8200700 1 --2- 172.21.15.63:0/2437560411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce010fff0 0x7fbce01123e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.709+0000 7fbce8200700 1 --2- 172.21.15.63:0/2437560411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce006baf0 0x7fbce006bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.709+0000 7fbce8200700 1 --2- 172.21.15.63:0/2437560411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbce006c410 0x7fbce010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.709+0000 7fbce8200700 1 -- 172.21.15.63:0/2437560411 >> 172.21.15.63:0/2437560411 conn(0x7fbce00793c0 msgr2=0x7fbce00797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:37.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.709+0000 7fbce8200700 1 -- 172.21.15.63:0/2437560411 shutdown_connections 2024-05-11T13:29:37.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.709+0000 7fbce8200700 1 -- 172.21.15.63:0/2437560411 wait complete. 2024-05-11T13:29:37.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.710+0000 7fbce8200700 1 Processor -- start 2024-05-11T13:29:37.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.711+0000 7fbce8200700 1 -- start start 2024-05-11T13:29:37.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.711+0000 7fbce8200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce006baf0 0x7fbce01a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:37.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.711+0000 7fbce8200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce006c410 0x7fbce01a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:37.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.711+0000 7fbce8200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbce010fff0 0x7fbce019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:37.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.711+0000 7fbce8200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01157a0 con 0x7fbce006c410 2024-05-11T13:29:37.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.711+0000 7fbce8200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0115620 con 0x7fbce010fff0 2024-05-11T13:29:37.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.711+0000 7fbce8200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0115920 con 0x7fbce006baf0 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce006c410 0x7fbce01a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce006c410 0x7fbce01a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35564/0 (socket says 172.21.15.63:35564) 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 -- 172.21.15.63:0/407765661 learned_addr learned my addr 172.21.15.63:0/407765661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce5800700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce006baf0 0x7fbce01a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce6200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbce010fff0 0x7fbce019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 -- 172.21.15.63:0/407765661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce006baf0 msgr2=0x7fbce01a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce006baf0 0x7fbce01a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 -- 172.21.15.63:0/407765661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbce010fff0 msgr2=0x7fbce019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbce010fff0 0x7fbce019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce4e00700 1 -- 172.21.15.63:0/407765661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce019c280 con 0x7fbce006c410 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.712+0000 7fbce6200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbce010fff0 0x7fbce019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:29:37.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.713+0000 7fbce4e00700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce006c410 0x7fbce01a1e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00d950 tx=0x7fbccc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:37.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.713+0000 7fbcde000700 1 -- 172.21.15.63:0/407765661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc016070 con 0x7fbce006c410 2024-05-11T13:29:37.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.713+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce019c570 con 0x7fbce006c410 2024-05-11T13:29:37.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.713+0000 7fbcde000700 1 -- 172.21.15.63:0/407765661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbccc009dd0 con 0x7fbce006c410 2024-05-11T13:29:37.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.713+0000 7fbcde000700 1 -- 172.21.15.63:0/407765661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00f3c0 con 0x7fbce006c410 2024-05-11T13:29:37.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.713+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01a87e0 con 0x7fbce006c410 2024-05-11T13:29:37.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.714+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce006a240 con 0x7fbce006c410 2024-05-11T13:29:37.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.717+0000 7fbcde000700 1 -- 172.21.15.63:0/407765661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbccc019070 con 0x7fbce006c410 2024-05-11T13:29:37.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.718+0000 7fbcde000700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbcc80b2900 0x7fbcc80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:37.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.719+0000 7fbcde000700 1 -- 172.21.15.63:0/407765661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fbccc0e1ff0 con 0x7fbce006c410 2024-05-11T13:29:37.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.719+0000 7fbce5800700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbcc80b2900 0x7fbcc80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:37.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.719+0000 7fbce5800700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbcc80b2900 0x7fbcc80b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4004630 tx=0x7fbcd400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:37.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.721+0000 7fbcde000700 1 -- 172.21.15.63:0/407765661 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbccc0a9a70 con 0x7fbce006c410 2024-05-11T13:29:37.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.992+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbce002e570 con 0x7fbce006c410 2024-05-11T13:29:37.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.993+0000 7fbcde000700 1 -- 172.21.15.63:0/407765661 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+590 (secure 0 0 0) 0x7fbccc0a9370 con 0x7fbce006c410 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3897275629 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2606154945 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2618629485 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2351804499 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4232658320 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:29:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1483448731 2024-05-12T13:18:39.354713+0000 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.996+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbcc80b2900 msgr2=0x7fbcc80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.996+0000 7fbce8200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbcc80b2900 0x7fbcc80b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4004630 tx=0x7fbcd400b3c0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.996+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce006c410 msgr2=0x7fbce01a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.996+0000 7fbce8200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce006c410 0x7fbce01a1e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00d950 tx=0x7fbccc00dc60 comp rx=0 tx=0).stop 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 shutdown_connections 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbcc80b2900 0x7fbcc80b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbce010fff0 0x7fbce019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce006baf0 0x7fbce01a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 --2- 172.21.15.63:0/407765661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce006c410 0x7fbce01a1e50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:37.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 >> 172.21.15.63:0/407765661 conn(0x7fbce00793c0 msgr2=0x7fbce0111540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:37.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 shutdown_connections 2024-05-11T13:29:37.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:37.997+0000 7fbce8200700 1 -- 172.21.15.63:0/407765661 wait complete. 2024-05-11T13:29:37.999 INFO:teuthology.orchestra.run.smithi063.stderr:listed 10 entries 2024-05-11T13:29:38.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: pgmap v405: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:38.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.567 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:38 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/407765661' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:29:38.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: pgmap v405: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.685 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:38 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/407765661' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: pgmap v405: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:38.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:38 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/407765661' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:29:38.938 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-05-11T13:29:38.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:38.938 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-11T13:29:39.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:ceph-fuse[73352]: fuse finished with error 0 and tester_r 0 2024-05-11T13:29:39.033 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T13:29:39.066 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:29:39.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:29:39.066 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-11T13:29:39.167 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:29:39.625 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:39.797 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-11T13:29:39.797 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-11T13:29:39.797 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-11T13:29:39.797 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-11T13:29:39.797 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-05-11T13:29:39.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-11T13:29:39.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-11T13:29:39.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-11T13:29:39.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-05-11T13:29:39.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:39 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-11T13:29:39.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-11T13:29:39.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-11T13:29:39.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-11T13:29:39.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-11T13:29:39.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-05-11T13:29:39.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-11T13:29:39.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-11T13:29:39.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-11T13:29:39.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-05-11T13:29:39.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:39 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-11T13:29:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-11T13:29:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-11T13:29:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-11T13:29:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-11T13:29:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-05-11T13:29:39.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-11T13:29:39.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-11T13:29:39.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-11T13:29:39.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-05-11T13:29:39.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:39 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-11T13:29:40.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-11T13:29:40.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: osdmap e87: 12 total, 12 up, 12 in 2024-05-11T13:29:40.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:29:40.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:29:40.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:40.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: pgmap v407: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2024-05-11T13:29:40.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:40 smithi063 ceph-mon[29809]: osdmap e88: 12 total, 12 up, 12 in 2024-05-11T13:29:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-11T13:29:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-11T13:29:40.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: osdmap e87: 12 total, 12 up, 12 in 2024-05-11T13:29:40.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:29:40.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:29:40.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:40.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: pgmap v407: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2024-05-11T13:29:40.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:40 smithi165 ceph-mon[33235]: osdmap e88: 12 total, 12 up, 12 in 2024-05-11T13:29:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-11T13:29:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-11T13:29:40.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: osdmap e87: 12 total, 12 up, 12 in 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: pgmap v407: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2024-05-11T13:29:40.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:40 smithi071 ceph-mon[33293]: osdmap e88: 12 total, 12 up, 12 in 2024-05-11T13:29:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 -- 172.21.15.63:0/4038994029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc34105660 msgr2=0x7efc34105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 --2- 172.21.15.63:0/4038994029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc34105660 0x7efc34105ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc20009990 tx=0x7efc20009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 -- 172.21.15.63:0/4038994029 shutdown_connections 2024-05-11T13:29:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 --2- 172.21.15.63:0/4038994029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc34106020 0x7efc34112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 --2- 172.21.15.63:0/4038994029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc3410b660 0x7efc3410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 --2- 172.21.15.63:0/4038994029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc34105660 0x7efc34105ae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 -- 172.21.15.63:0/4038994029 >> 172.21.15.63:0/4038994029 conn(0x7efc34101130 msgr2=0x7efc34103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.912+0000 7efc33200700 1 -- 172.21.15.63:0/4038994029 shutdown_connections 2024-05-11T13:29:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.913+0000 7efc33200700 1 -- 172.21.15.63:0/4038994029 wait complete. 2024-05-11T13:29:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.914+0000 7efc33200700 1 Processor -- start 2024-05-11T13:29:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.915+0000 7efc33200700 1 -- start start 2024-05-11T13:29:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.915+0000 7efc33200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc34105660 0x7efc341a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:41.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.915+0000 7efc33200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc34106020 0x7efc341a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:41.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.915+0000 7efc33200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc3410b660 0x7efc3419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:41.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.915+0000 7efc33200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc34115740 con 0x7efc3410b660 2024-05-11T13:29:41.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.915+0000 7efc33200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc341155c0 con 0x7efc34105660 2024-05-11T13:29:41.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.915+0000 7efc33200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc341158c0 con 0x7efc34106020 2024-05-11T13:29:41.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc3410b660 0x7efc3419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:41.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc3410b660 0x7efc3419baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32854/0 (socket says 172.21.15.63:32854) 2024-05-11T13:29:41.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 -- 172.21.15.63:0/3348134271 learned_addr learned my addr 172.21.15.63:0/3348134271 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:41.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc31e00700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc34105660 0x7efc341a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:41.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc31400700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc34106020 0x7efc341a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 -- 172.21.15.63:0/3348134271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc34106020 msgr2=0x7efc341a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc34106020 0x7efc341a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 -- 172.21.15.63:0/3348134271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc34105660 msgr2=0x7efc341a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc34105660 0x7efc341a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.916+0000 7efc32800700 1 -- 172.21.15.63:0/3348134271 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3419c200 con 0x7efc3410b660 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.917+0000 7efc32800700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc3410b660 0x7efc3419baa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ada0 tx=0x7efc2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.917+0000 7efc2aa00700 1 -- 172.21.15.63:0/3348134271 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c017020 con 0x7efc3410b660 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.917+0000 7efc2aa00700 1 -- 172.21.15.63:0/3348134271 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc2c00ce10 con 0x7efc3410b660 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.917+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3419c4f0 con 0x7efc3410b660 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.917+0000 7efc2aa00700 1 -- 172.21.15.63:0/3348134271 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c005280 con 0x7efc3410b660 2024-05-11T13:29:41.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.917+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a87b0 con 0x7efc3410b660 2024-05-11T13:29:41.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.918+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3406a240 con 0x7efc3410b660 2024-05-11T13:29:41.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.920+0000 7efc2aa00700 1 -- 172.21.15.63:0/3348134271 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efc2c00c980 con 0x7efc3410b660 2024-05-11T13:29:41.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.922+0000 7efc2aa00700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc180b2900 0x7efc180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:41.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.922+0000 7efc2aa00700 1 -- 172.21.15.63:0/3348134271 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7efc2c0e1c90 con 0x7efc3410b660 2024-05-11T13:29:41.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.922+0000 7efc31e00700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc180b2900 0x7efc180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:41.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.923+0000 7efc31e00700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc180b2900 0x7efc180b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc24005950 tx=0x7efc2400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:41.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:41.927+0000 7efc2aa00700 1 -- 172.21.15.63:0/3348134271 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc2c021020 con 0x7efc3410b660 2024-05-11T13:29:42.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.160+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc3402e570 con 0x7efc3410b660 2024-05-11T13:29:42.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.165+0000 7efc2aa00700 1 -- 172.21.15.63:0/3348134271 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24619 (secure 0 0 0) 0x7efc2c0a9680 con 0x7efc3410b660 2024-05-11T13:29:42.167 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:42.168 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":88,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:29:40.520411+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:29:42.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc180b2900 msgr2=0x7efc180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:42.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc180b2900 0x7efc180b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc24005950 tx=0x7efc2400a300 comp rx=0 tx=0).stop 2024-05-11T13:29:42.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc3410b660 msgr2=0x7efc3419baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:42.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc3410b660 0x7efc3419baa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ada0 tx=0x7efc2c00c330 comp rx=0 tx=0).stop 2024-05-11T13:29:42.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 shutdown_connections 2024-05-11T13:29:42.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7efc180b2900 0x7efc180b4dc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:42.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc3410b660 0x7efc3419baa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:42.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc34105660 0x7efc341a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:42.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 --2- 172.21.15.63:0/3348134271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc34106020 0x7efc341a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:42.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.168+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 >> 172.21.15.63:0/3348134271 conn(0x7efc34101130 msgr2=0x7efc34101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:42.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.169+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 shutdown_connections 2024-05-11T13:29:42.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:42.169+0000 7efc33200700 1 -- 172.21.15.63:0/3348134271 wait complete. 2024-05-11T13:29:42.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:42 smithi063 ceph-mon[29809]: pgmap v409: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-05-11T13:29:42.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:42 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3348134271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:42.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:42 smithi071 ceph-mon[33293]: pgmap v409: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-05-11T13:29:42.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:42 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3348134271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:43.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:42 smithi165 ceph-mon[33235]: pgmap v409: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-05-11T13:29:43.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3348134271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:43.077 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:29:43.583 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:44.929 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:44 smithi063 ceph-mon[29809]: pgmap v410: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:44.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:44 smithi071 ceph-mon[33293]: pgmap v410: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:44 smithi165 ceph-mon[33235]: pgmap v410: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:45.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 -- 172.21.15.63:0/3513266694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae810b770 msgr2=0x7f1ae810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:45.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 --2- 172.21.15.63:0/3513266694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae810b770 0x7f1ae810bb50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009990 tx=0x7f1ad8009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:45.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 -- 172.21.15.63:0/3513266694 shutdown_connections 2024-05-11T13:29:45.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 --2- 172.21.15.63:0/3513266694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ae8106130 0x7f1ae8112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:45.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 --2- 172.21.15.63:0/3513266694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae810b770 0x7f1ae810bb50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:45.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 --2- 172.21.15.63:0/3513266694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae8105770 0x7f1ae8105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:45.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 -- 172.21.15.63:0/3513266694 >> 172.21.15.63:0/3513266694 conn(0x7f1ae8101260 msgr2=0x7f1ae8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:45.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.776+0000 7f1aed000700 1 -- 172.21.15.63:0/3513266694 shutdown_connections 2024-05-11T13:29:45.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.777+0000 7f1aed000700 1 -- 172.21.15.63:0/3513266694 wait complete. 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 Processor -- start 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 -- start start 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ae8105770 0x7f1ae819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae8106130 0x7f1ae819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae810b770 0x7f1ae81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8115850 con 0x7f1ae8106130 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81156d0 con 0x7f1ae810b770 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.778+0000 7f1aed000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81159d0 con 0x7f1ae8105770 2024-05-11T13:29:45.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.779+0000 7f1ae5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae8106130 0x7f1ae819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:45.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.779+0000 7f1ae5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae8106130 0x7f1ae819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32884/0 (socket says 172.21.15.63:32884) 2024-05-11T13:29:45.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.779+0000 7f1ae5400700 1 -- 172.21.15.63:0/2597779507 learned_addr learned my addr 172.21.15.63:0/2597779507 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:45.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.779+0000 7f1ae5e00700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ae8105770 0x7f1ae819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:45.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.779+0000 7f1ae6800700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae810b770 0x7f1ae81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.780+0000 7f1ae5400700 1 -- 172.21.15.63:0/2597779507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ae8105770 msgr2=0x7f1ae819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.780+0000 7f1ae5400700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ae8105770 0x7f1ae819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.780+0000 7f1ae5400700 1 -- 172.21.15.63:0/2597779507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae810b770 msgr2=0x7f1ae81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.780+0000 7f1ae5400700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae810b770 0x7f1ae81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.780+0000 7f1ae5400700 1 -- 172.21.15.63:0/2597779507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae81a0d90 con 0x7f1ae8106130 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.780+0000 7f1ae5400700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae8106130 0x7f1ae819c2c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400d950 tx=0x7f1ad400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.781+0000 7f1adea00700 1 -- 172.21.15.63:0/2597779507 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4016070 con 0x7f1ae8106130 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.781+0000 7f1adea00700 1 -- 172.21.15.63:0/2597779507 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ad400fd20 con 0x7f1ae8106130 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.781+0000 7f1adea00700 1 -- 172.21.15.63:0/2597779507 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400e3b0 con 0x7f1ae8106130 2024-05-11T13:29:45.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.781+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae81a1080 con 0x7f1ae8106130 2024-05-11T13:29:45.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.781+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae81a8900 con 0x7f1ae8106130 2024-05-11T13:29:45.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.782+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae806a240 con 0x7f1ae8106130 2024-05-11T13:29:45.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.784+0000 7f1adea00700 1 -- 172.21.15.63:0/2597779507 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1ad4019070 con 0x7f1ae8106130 2024-05-11T13:29:45.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.786+0000 7f1adea00700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1acc0b2900 0x7f1acc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:45.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.786+0000 7f1adea00700 1 -- 172.21.15.63:0/2597779507 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1ad40e1fe0 con 0x7f1ae8106130 2024-05-11T13:29:45.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.786+0000 7f1ae5e00700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1acc0b2900 0x7f1acc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:45.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.787+0000 7f1ae5e00700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1acc0b2900 0x7f1acc0b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8003760 tx=0x7f1ad8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:45.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:45.791+0000 7f1adea00700 1 -- 172.21.15.63:0/2597779507 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ad40a9940 con 0x7f1ae8106130 2024-05-11T13:29:46.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.027+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ae802e570 con 0x7f1ae8106130 2024-05-11T13:29:46.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.031+0000 7f1adea00700 1 -- 172.21.15.63:0/2597779507 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24619 (secure 0 0 0) 0x7f1ad40a9240 con 0x7f1ae8106130 2024-05-11T13:29:46.033 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:29:46.033 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":88,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:29:40.520411+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:29:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1acc0b2900 msgr2=0x7f1acc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1acc0b2900 0x7f1acc0b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8003760 tx=0x7f1ad8005fd0 comp rx=0 tx=0).stop 2024-05-11T13:29:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae8106130 msgr2=0x7f1ae819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae8106130 0x7f1ae819c2c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400d950 tx=0x7f1ad400dc60 comp rx=0 tx=0).stop 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 shutdown_connections 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1acc0b2900 0x7f1acc0b4dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae810b770 0x7f1ae81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ae8105770 0x7f1ae819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 --2- 172.21.15.63:0/2597779507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ae8106130 0x7f1ae819c2c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 >> 172.21.15.63:0/2597779507 conn(0x7f1ae8101260 msgr2=0x7f1ae8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 shutdown_connections 2024-05-11T13:29:46.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:46.035+0000 7f1aed000700 1 -- 172.21.15.63:0/2597779507 wait complete. 2024-05-11T13:29:46.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:46 smithi063 ceph-mon[29809]: pgmap v411: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:46.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:46 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2597779507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:46.878 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-11T13:29:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:46 smithi071 ceph-mon[33293]: pgmap v411: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:46.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:46 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2597779507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:46 smithi165 ceph-mon[33235]: pgmap v411: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:46 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2597779507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:29:47.349 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:48.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:48 smithi063 ceph-mon[29809]: pgmap v412: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:48.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:48 smithi071 ceph-mon[33293]: pgmap v412: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:48 smithi165 ceph-mon[33235]: pgmap v412: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 -- 172.21.15.63:0/774872863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f202806b260 msgr2=0x7f202806b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 --2- 172.21.15.63:0/774872863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f202806b260 0x7f202806b640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f201c009990 tx=0x7f201c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 -- 172.21.15.63:0/774872863 shutdown_connections 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 --2- 172.21.15.63:0/774872863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2028110000 0x7f20281123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 --2- 172.21.15.63:0/774872863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f202806b260 0x7f202806b640 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 --2- 172.21.15.63:0/774872863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f202806bb80 0x7f202810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 -- 172.21.15.63:0/774872863 >> 172.21.15.63:0/774872863 conn(0x7f2028079400 msgr2=0x7f2028079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 -- 172.21.15.63:0/774872863 shutdown_connections 2024-05-11T13:29:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.661+0000 7f2030200700 1 -- 172.21.15.63:0/774872863 wait complete. 2024-05-11T13:29:49.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.662+0000 7f2030200700 1 Processor -- start 2024-05-11T13:29:49.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f2030200700 1 -- start start 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f2030200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202806b260 0x7f202819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f2030200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f202806bb80 0x7f202819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f2030200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2028110000 0x7f20281a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f2030200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2028115810 con 0x7f2028110000 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f2030200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2028115690 con 0x7f202806b260 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f2030200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2028115990 con 0x7f202806bb80 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f202e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2028110000 0x7f20281a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f202e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2028110000 0x7f20281a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32902/0 (socket says 172.21.15.63:32902) 2024-05-11T13:29:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.663+0000 7f202e200700 1 -- 172.21.15.63:0/38795564 learned_addr learned my addr 172.21.15.63:0/38795564 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:49.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202ce00700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f202806bb80 0x7f202819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:49.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202d800700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202806b260 0x7f202819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:49.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202e200700 1 -- 172.21.15.63:0/38795564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f202806bb80 msgr2=0x7f202819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:49.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202e200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f202806bb80 0x7f202819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202e200700 1 -- 172.21.15.63:0/38795564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202806b260 msgr2=0x7f202819bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202e200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202806b260 0x7f202819bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202e200700 1 -- 172.21.15.63:0/38795564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20281a0db0 con 0x7f2028110000 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.664+0000 7f202e200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2028110000 0x7f20281a0670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f202000ef90 tx=0x7f202000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.665+0000 7f2026000700 1 -- 172.21.15.63:0/38795564 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20200092e0 con 0x7f2028110000 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.665+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20281a1040 con 0x7f2028110000 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.665+0000 7f2026000700 1 -- 172.21.15.63:0/38795564 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2020004bb0 con 0x7f2028110000 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.665+0000 7f2026000700 1 -- 172.21.15.63:0/38795564 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020004280 con 0x7f2028110000 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.665+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20281a8920 con 0x7f2028110000 2024-05-11T13:29:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.666+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202806a240 con 0x7f2028110000 2024-05-11T13:29:49.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.668+0000 7f2026000700 1 -- 172.21.15.63:0/38795564 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f202001b030 con 0x7f2028110000 2024-05-11T13:29:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.670+0000 7f2026000700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f20100b2880 0x7f20100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.670+0000 7f2026000700 1 -- 172.21.15.63:0/38795564 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f20200e2210 con 0x7f2028110000 2024-05-11T13:29:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.670+0000 7f202d800700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f20100b2880 0x7f20100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.671+0000 7f202d800700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f20100b2880 0x7f20100b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b2e0 tx=0x7f201c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:49.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.674+0000 7f2026000700 1 -- 172.21.15.63:0/38795564 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2020022020 con 0x7f2028110000 2024-05-11T13:29:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:49.963+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f202802e570 con 0x7f2028110000 2024-05-11T13:29:50.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.664+0000 7f2026000700 1 -- 172.21.15.63:0/38795564 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f2020010040 con 0x7f2028110000 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.668+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f20100b2880 msgr2=0x7f20100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.668+0000 7f2030200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f20100b2880 0x7f20100b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b2e0 tx=0x7f201c01a040 comp rx=0 tx=0).stop 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.668+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2028110000 msgr2=0x7f20281a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.668+0000 7f2030200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2028110000 0x7f20281a0670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f202000ef90 tx=0x7f202000c450 comp rx=0 tx=0).stop 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 shutdown_connections 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f20100b2880 0x7f20100b4d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2028110000 0x7f20281a0670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202806b260 0x7f202819bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 --2- 172.21.15.63:0/38795564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f202806bb80 0x7f202819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 >> 172.21.15.63:0/38795564 conn(0x7f2028079400 msgr2=0x7f20281115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 shutdown_connections 2024-05-11T13:29:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:50.669+0000 7f2030200700 1 -- 172.21.15.63:0/38795564 wait complete. 2024-05-11T13:29:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:50 smithi063 ceph-mon[29809]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:50 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/38795564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-11T13:29:50.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:50 smithi071 ceph-mon[33293]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:50.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:50 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/38795564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-11T13:29:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:50 smithi165 ceph-mon[33235]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-11T13:29:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:50 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/38795564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-11T13:29:51.512 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-11T13:29:51.512 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-11T13:29:51.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:51 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/38795564' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-11T13:29:51.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:51 smithi063 ceph-mon[29809]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:29:51.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:51 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/38795564' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-11T13:29:51.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:51 smithi071 ceph-mon[33293]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:29:52.008 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:52.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:51 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/38795564' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-11T13:29:52.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:51 smithi165 ceph-mon[33235]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-11T13:29:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:52 smithi063 ceph-mon[29809]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 92 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-11T13:29:52.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:52 smithi071 ceph-mon[33293]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 92 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-11T13:29:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:52 smithi165 ceph-mon[33235]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 92 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-11T13:29:54.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.245+0000 7f8a8e600700 1 -- 172.21.15.63:0/1186649623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8810b770 msgr2=0x7f8a8810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:54.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.245+0000 7f8a8e600700 1 --2- 172.21.15.63:0/1186649623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8810b770 0x7f8a8810bb50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009990 tx=0x7f8a78009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.246+0000 7f8a8e600700 1 -- 172.21.15.63:0/1186649623 shutdown_connections 2024-05-11T13:29:54.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.246+0000 7f8a8e600700 1 --2- 172.21.15.63:0/1186649623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a88106130 0x7f8a88112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.246+0000 7f8a8e600700 1 --2- 172.21.15.63:0/1186649623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8810b770 0x7f8a8810bb50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.246+0000 7f8a8e600700 1 --2- 172.21.15.63:0/1186649623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a88105770 0x7f8a88105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.246+0000 7f8a8e600700 1 -- 172.21.15.63:0/1186649623 >> 172.21.15.63:0/1186649623 conn(0x7f8a88101260 msgr2=0x7f8a88103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:54.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.246+0000 7f8a8e600700 1 -- 172.21.15.63:0/1186649623 shutdown_connections 2024-05-11T13:29:54.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.246+0000 7f8a8e600700 1 -- 172.21.15.63:0/1186649623 wait complete. 2024-05-11T13:29:54.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.247+0000 7f8a8e600700 1 Processor -- start 2024-05-11T13:29:54.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.247+0000 7f8a8e600700 1 -- start start 2024-05-11T13:29:54.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.247+0000 7f8a8e600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a88105770 0x7f8a8819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.247+0000 7f8a8e600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a88106130 0x7f8a8819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.247+0000 7f8a8e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8810b770 0x7f8a881a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a8e600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a88115820 con 0x7f8a88106130 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a8e600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881156a0 con 0x7f8a8810b770 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a8e600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881159a0 con 0x7f8a88105770 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a88106130 0x7f8a8819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a88106130 0x7f8a8819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55094/0 (socket says 172.21.15.63:55094) 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 -- 172.21.15.63:0/2546696100 learned_addr learned my addr 172.21.15.63:0/2546696100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a87400700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a88105770 0x7f8a8819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a87e00700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8810b770 0x7f8a881a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 -- 172.21.15.63:0/2546696100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a88105770 msgr2=0x7f8a8819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a88105770 0x7f8a8819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 -- 172.21.15.63:0/2546696100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8810b770 msgr2=0x7f8a881a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:54.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8810b770 0x7f8a881a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.248+0000 7f8a86a00700 1 -- 172.21.15.63:0/2546696100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a881a0d60 con 0x7f8a88106130 2024-05-11T13:29:54.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.249+0000 7f8a86a00700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a88106130 0x7f8a8819c290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400d980 tx=0x7f8a7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:54.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.249+0000 7f8a7fe00700 1 -- 172.21.15.63:0/2546696100 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74017070 con 0x7f8a88106130 2024-05-11T13:29:54.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.249+0000 7f8a7fe00700 1 -- 172.21.15.63:0/2546696100 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a74005020 con 0x7f8a88106130 2024-05-11T13:29:54.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.249+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a881a1050 con 0x7f8a88106130 2024-05-11T13:29:54.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.249+0000 7f8a7fe00700 1 -- 172.21.15.63:0/2546696100 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74009ab0 con 0x7f8a88106130 2024-05-11T13:29:54.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.249+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a881a88d0 con 0x7f8a88106130 2024-05-11T13:29:54.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.251+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8810dd00 con 0x7f8a88106130 2024-05-11T13:29:54.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.253+0000 7f8a7fe00700 1 -- 172.21.15.63:0/2546696100 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8a740051d0 con 0x7f8a88106130 2024-05-11T13:29:54.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.254+0000 7f8a7fe00700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:54.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.255+0000 7f8a7fe00700 1 -- 172.21.15.63:0/2546696100 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f8a740e24d0 con 0x7f8a88106130 2024-05-11T13:29:54.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.255+0000 7f8a87400700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:54.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.255+0000 7f8a87400700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098e0 tx=0x7f8a78005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:54.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.259+0000 7f8a7fe00700 1 -- 172.21.15.63:0/2546696100 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a74020020 con 0x7f8a88106130 2024-05-11T13:29:54.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.550+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f8a8810d6e0 con 0x7f8a88106130 2024-05-11T13:29:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.697+0000 7f8a7fe00700 1 -- 172.21.15.63:0/2546696100 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f8a740a9e30 con 0x7f8a88106130 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8a6c0b2880 msgr2=0x7f8a6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098e0 tx=0x7f8a78005960 comp rx=0 tx=0).stop 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a88106130 msgr2=0x7f8a8819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a88106130 0x7f8a8819c290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400d980 tx=0x7f8a7400dc90 comp rx=0 tx=0).stop 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 shutdown_connections 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8810b770 0x7f8a881a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a88105770 0x7f8a8819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 --2- 172.21.15.63:0/2546696100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a88106130 0x7f8a8819c290 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 >> 172.21.15.63:0/2546696100 conn(0x7f8a88101260 msgr2=0x7f8a88101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.701+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 shutdown_connections 2024-05-11T13:29:54.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:54.702+0000 7f8a8e600700 1 -- 172.21.15.63:0/2546696100 wait complete. 2024-05-11T13:29:54.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:54 smithi063 ceph-mon[29809]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-05-11T13:29:54.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:54 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2546696100' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-11T13:29:55.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:54 smithi165 ceph-mon[33235]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-05-11T13:29:55.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:54 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2546696100' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-11T13:29:55.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:54 smithi071 ceph-mon[33293]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-05-11T13:29:55.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:54 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2546696100' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-11T13:29:55.566 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-11T13:29:55.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:55 smithi063 ceph-mon[29809]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-11T13:29:55.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:55 smithi063 ceph-mon[29809]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-11T13:29:55.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:55 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2546696100' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-11T13:29:55.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:55 smithi063 ceph-mon[29809]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-11T13:29:55.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:55 smithi063 ceph-mon[29809]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-11T13:29:56.045 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:29:56.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:55 smithi165 ceph-mon[33235]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-11T13:29:56.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:55 smithi165 ceph-mon[33235]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-11T13:29:56.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:55 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2546696100' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-11T13:29:56.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:55 smithi165 ceph-mon[33235]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-11T13:29:56.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:55 smithi165 ceph-mon[33235]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-11T13:29:56.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:55 smithi071 ceph-mon[33293]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-11T13:29:56.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:55 smithi071 ceph-mon[33293]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-11T13:29:56.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:55 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2546696100' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-11T13:29:56.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:55 smithi071 ceph-mon[33293]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-11T13:29:56.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:55 smithi071 ceph-mon[33293]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-11T13:29:56.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:56 smithi063 ceph-mon[29809]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:56.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:56 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] up:active 2024-05-11T13:29:56.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:56 smithi063 ceph-mon[29809]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-11T13:29:56.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:56 smithi063 ceph-mon[29809]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-11T13:29:56.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:56 smithi063 ceph-mon[29809]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-11T13:29:56.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:56 smithi063 ceph-mon[29809]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-11T13:29:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:56 smithi165 ceph-mon[33235]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:56 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] up:active 2024-05-11T13:29:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:56 smithi165 ceph-mon[33235]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-11T13:29:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:56 smithi165 ceph-mon[33235]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-11T13:29:57.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:56 smithi165 ceph-mon[33235]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-11T13:29:57.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:56 smithi165 ceph-mon[33235]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-11T13:29:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:56 smithi071 ceph-mon[33293]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:29:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:56 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] up:active 2024-05-11T13:29:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:56 smithi071 ceph-mon[33293]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-11T13:29:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:56 smithi071 ceph-mon[33293]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-11T13:29:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:56 smithi071 ceph-mon[33293]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-11T13:29:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:56 smithi071 ceph-mon[33293]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-11T13:29:58.010 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:57 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] up:active 2024-05-11T13:29:58.010 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:57 smithi063 ceph-mon[29809]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-11T13:29:58.010 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:57 smithi063 ceph-mon[29809]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-11T13:29:58.010 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:57 smithi063 ceph-mon[29809]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-11T13:29:58.010 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:57 smithi063 ceph-mon[29809]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-11T13:29:58.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:57 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] up:active 2024-05-11T13:29:58.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:57 smithi165 ceph-mon[33235]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-11T13:29:58.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:57 smithi165 ceph-mon[33235]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-11T13:29:58.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:57 smithi165 ceph-mon[33235]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-11T13:29:58.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:57 smithi165 ceph-mon[33235]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-11T13:29:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:57 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] up:active 2024-05-11T13:29:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:57 smithi071 ceph-mon[33293]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-11T13:29:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:57 smithi071 ceph-mon[33293]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-11T13:29:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:57 smithi071 ceph-mon[33293]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-11T13:29:58.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:57 smithi071 ceph-mon[33293]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-11T13:29:58.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 -- 172.21.15.63:0/885001411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64103600 msgr2=0x7f9c64103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 --2- 172.21.15.63:0/885001411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64103600 0x7f9c64103a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009990 tx=0x7f9c50009ca0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 -- 172.21.15.63:0/885001411 shutdown_connections 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 --2- 172.21.15.63:0/885001411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c64103fa0 0x7f9c64110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 --2- 172.21.15.63:0/885001411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c64108510 0x7f9c641088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 --2- 172.21.15.63:0/885001411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64103600 0x7f9c64103a60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 -- 172.21.15.63:0/885001411 >> 172.21.15.63:0/885001411 conn(0x7f9c6407a6b0 msgr2=0x7f9c6407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 -- 172.21.15.63:0/885001411 shutdown_connections 2024-05-11T13:29:58.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.311+0000 7f9c6b800700 1 -- 172.21.15.63:0/885001411 wait complete. 2024-05-11T13:29:58.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.312+0000 7f9c6b800700 1 Processor -- start 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.312+0000 7f9c6b800700 1 -- start start 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c6b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c64103600 0x7f9c64075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c6b800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c64103fa0 0x7f9c64075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c6b800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64108510 0x7f9c640762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c6b800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641136a0 con 0x7f9c64108510 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c6b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9c64113520 con 0x7f9c64103600 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c6b800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c64113820 con 0x7f9c64103fa0 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c69800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64108510 0x7f9c640762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c69800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64108510 0x7f9c640762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55118/0 (socket says 172.21.15.63:55118) 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c69800700 1 -- 172.21.15.63:0/4119663199 learned_addr learned my addr 172.21.15.63:0/4119663199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.313+0000 7f9c68e00700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c64103600 0x7f9c64075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:58.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.314+0000 7f9c5be00700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c64103fa0 0x7f9c64075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:58.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.314+0000 7f9c69800700 1 -- 172.21.15.63:0/4119663199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c64103fa0 msgr2=0x7f9c64075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:58.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.314+0000 7f9c69800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c64103fa0 0x7f9c64075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.314+0000 7f9c69800700 1 -- 172.21.15.63:0/4119663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c64103600 msgr2=0x7f9c64075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.314+0000 7f9c69800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c64103600 0x7f9c64075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.314+0000 7f9c69800700 1 -- 172.21.15.63:0/4119663199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c641a8040 con 0x7f9c64108510 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.314+0000 7f9c69800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64108510 0x7f9c640762d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00cca0 tx=0x7f9c5c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.315+0000 7f9c62000700 1 -- 172.21.15.63:0/4119663199 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c016020 con 0x7f9c64108510 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.315+0000 7f9c62000700 1 -- 172.21.15.63:0/4119663199 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c5c012420 con 0x7f9c64108510 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.315+0000 7f9c62000700 1 -- 172.21.15.63:0/4119663199 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0115e0 con 0x7f9c64108510 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.315+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c641a8330 con 0x7f9c64108510 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.315+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c641a8840 con 0x7f9c64108510 2024-05-11T13:29:58.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.316+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6410bb40 con 0x7f9c64108510 2024-05-11T13:29:58.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.318+0000 7f9c62000700 1 -- 172.21.15.63:0/4119663199 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9c5c019020 con 0x7f9c64108510 2024-05-11T13:29:58.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.320+0000 7f9c62000700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c400b2900 0x7f9c400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:29:58.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.320+0000 7f9c62000700 1 -- 172.21.15.63:0/4119663199 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9c5c0e2380 con 0x7f9c64108510 2024-05-11T13:29:58.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.320+0000 7f9c68e00700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c400b2900 0x7f9c400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:29:58.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.321+0000 7f9c68e00700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c400b2900 0x7f9c400b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c48005950 tx=0x7f9c4800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:29:58.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.325+0000 7f9c62000700 1 -- 172.21.15.63:0/4119663199 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c5c020020 con 0x7f9c64108510 2024-05-11T13:29:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.620+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f9c64076d80 con 0x7f9c64108510 2024-05-11T13:29:58.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.766+0000 7f9c62000700 1 -- 172.21.15.63:0/4119663199 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f9c5c0a9ce0 con 0x7f9c64108510 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.770+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c400b2900 msgr2=0x7f9c400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c400b2900 0x7f9c400b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c48005950 tx=0x7f9c4800a300 comp rx=0 tx=0).stop 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64108510 msgr2=0x7f9c640762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64108510 0x7f9c640762d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00cca0 tx=0x7f9c5c00a060 comp rx=0 tx=0).stop 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 shutdown_connections 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c400b2900 0x7f9c400b4dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c64108510 0x7f9c640762d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c64103600 0x7f9c64075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 --2- 172.21.15.63:0/4119663199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c64103fa0 0x7f9c64075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 >> 172.21.15.63:0/4119663199 conn(0x7f9c6407a6b0 msgr2=0x7f9c641016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 shutdown_connections 2024-05-11T13:29:58.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:29:58.771+0000 7f9c6b800700 1 -- 172.21.15.63:0/4119663199 wait complete. 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: pgmap v417: 129 pgs: 129 active+clean; 583 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] up:active 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: Cluster is now healthy 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-11T13:29:58.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:58 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4119663199' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-11T13:29:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: pgmap v417: 129 pgs: 129 active+clean; 583 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-05-11T13:29:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] up:active 2024-05-11T13:29:59.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-11T13:29:59.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-11T13:29:59.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: Cluster is now healthy 2024-05-11T13:29:59.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-11T13:29:59.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-11T13:29:59.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-11T13:29:59.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:58 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4119663199' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-11T13:29:59.105 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: pgmap v417: 129 pgs: 129 active+clean; 583 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-05-11T13:29:59.105 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] up:active 2024-05-11T13:29:59.105 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-11T13:29:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-11T13:29:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: Cluster is now healthy 2024-05-11T13:29:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-11T13:29:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-11T13:29:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-11T13:29:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:58 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4119663199' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-11T13:29:59.619 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-11T13:30:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:59 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4119663199' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-11T13:30:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:59 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] up:active 2024-05-11T13:30:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:59 smithi165 ceph-mon[33235]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-11T13:30:00.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:29:59 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:30:00.087 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:00.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:59 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4119663199' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-11T13:30:00.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:59 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] up:active 2024-05-11T13:30:00.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:59 smithi063 ceph-mon[29809]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-11T13:30:00.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:29:59 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:30:00.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:59 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4119663199' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-11T13:30:00.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:59 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] up:active 2024-05-11T13:30:00.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:59 smithi071 ceph-mon[33293]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-11T13:30:00.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:29:59 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:30:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:00 smithi063 ceph-mon[29809]: pgmap v418: 129 pgs: 129 active+clean; 586 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-11T13:30:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:00 smithi063 ceph-mon[29809]: overall HEALTH_OK 2024-05-11T13:30:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:00 smithi165 ceph-mon[33235]: pgmap v418: 129 pgs: 129 active+clean; 586 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-11T13:30:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:00 smithi165 ceph-mon[33235]: overall HEALTH_OK 2024-05-11T13:30:01.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:00 smithi071 ceph-mon[33293]: pgmap v418: 129 pgs: 129 active+clean; 586 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-11T13:30:01.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:00 smithi071 ceph-mon[33293]: overall HEALTH_OK 2024-05-11T13:30:02.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.421+0000 7f06a7e00700 1 -- 172.21.15.63:0/2840572873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 msgr2=0x7f06a8109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.421+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2840572873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a8109930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f06980099c0 tx=0x7f0698009cd0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.421+0000 7f06a7e00700 1 -- 172.21.15.63:0/2840572873 shutdown_connections 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.421+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2840572873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a8103e60 0x7f06a8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.421+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2840572873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a8109930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.421+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2840572873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a81034a0 0x7f06a8103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.421+0000 7f06a7e00700 1 -- 172.21.15.63:0/2840572873 >> 172.21.15.63:0/2840572873 conn(0x7f06a80fefd0 msgr2=0x7f06a81013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.422+0000 7f06a7e00700 1 -- 172.21.15.63:0/2840572873 shutdown_connections 2024-05-11T13:30:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.422+0000 7f06a7e00700 1 -- 172.21.15.63:0/2840572873 wait complete. 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.423+0000 7f06a7e00700 1 Processor -- start 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.423+0000 7f06a7e00700 1 -- start start 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.423+0000 7f06a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a81034a0 0x7f06a81a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a8103e60 0x7f06a81a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8113560 con 0x7f06a8109550 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81133e0 con 0x7f06a81034a0 2024-05-11T13:30:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81136e0 con 0x7f06a8103e60 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a819ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50718/0 (socket says 172.21.15.63:50718) 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a7400700 1 -- 172.21.15.63:0/2932567126 learned_addr learned my addr 172.21.15.63:0/2932567126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a6a00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a81034a0 0x7f06a81a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.424+0000 7f06a6000700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a8103e60 0x7f06a81a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f06a7400700 1 -- 172.21.15.63:0/2932567126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a8103e60 msgr2=0x7f06a81a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f06a7400700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a8103e60 0x7f06a81a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f06a7400700 1 -- 172.21.15.63:0/2932567126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a81034a0 msgr2=0x7f06a81a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f06a7400700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a81034a0 0x7f06a81a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f06a7400700 1 -- 172.21.15.63:0/2932567126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a819c1b0 con 0x7f06a8109550 2024-05-11T13:30:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f06a7400700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a819ba50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f06a000ef30 tx=0x7f06a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f069f400700 1 -- 172.21.15.63:0/2932567126 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a00092e0 con 0x7f06a8109550 2024-05-11T13:30:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.425+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a819c440 con 0x7f06a8109550 2024-05-11T13:30:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.426+0000 7f069f400700 1 -- 172.21.15.63:0/2932567126 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06a0004bb0 con 0x7f06a8109550 2024-05-11T13:30:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.426+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81a8760 con 0x7f06a8109550 2024-05-11T13:30:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.426+0000 7f069f400700 1 -- 172.21.15.63:0/2932567126 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0004020 con 0x7f06a8109550 2024-05-11T13:30:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.427+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a806a240 con 0x7f06a8109550 2024-05-11T13:30:02.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.429+0000 7f069f400700 1 -- 172.21.15.63:0/2932567126 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f06a001b030 con 0x7f06a8109550 2024-05-11T13:30:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.431+0000 7f069f400700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f068c0b2900 0x7f068c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.431+0000 7f069f400700 1 -- 172.21.15.63:0/2932567126 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f06a00e2a30 con 0x7f06a8109550 2024-05-11T13:30:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.431+0000 7f06a6a00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f068c0b2900 0x7f068c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.432+0000 7f06a6a00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f068c0b2900 0x7f068c0b4dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0698000c00 tx=0x7f0698005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.436+0000 7f069f400700 1 -- 172.21.15.63:0/2932567126 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06a0022020 con 0x7f06a8109550 2024-05-11T13:30:02.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.728+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f06a81a8a50 con 0x7f06a8109550 2024-05-11T13:30:02.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.800+0000 7f069f400700 1 -- 172.21.15.63:0/2932567126 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f06a0010040 con 0x7f06a8109550 2024-05-11T13:30:02.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.805+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f068c0b2900 msgr2=0x7f068c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:02.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.805+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f068c0b2900 0x7f068c0b4dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0698000c00 tx=0x7f0698005fb0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.805+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 msgr2=0x7f06a819ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:02.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.805+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a819ba50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f06a000ef30 tx=0x7f06a000c3a0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.806+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 shutdown_connections 2024-05-11T13:30:02.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.806+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f068c0b2900 0x7f068c0b4dc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.806+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a8109550 0x7f06a819ba50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.806+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a81034a0 0x7f06a81a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.806+0000 7f06a7e00700 1 --2- 172.21.15.63:0/2932567126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a8103e60 0x7f06a81a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:02.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.806+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 >> 172.21.15.63:0/2932567126 conn(0x7f06a80fefd0 msgr2=0x7f06a8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:02.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.806+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 shutdown_connections 2024-05-11T13:30:02.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:02.807+0000 7f06a7e00700 1 -- 172.21.15.63:0/2932567126 wait complete. 2024-05-11T13:30:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:02 smithi165 ceph-mon[33235]: pgmap v419: 129 pgs: 129 active+clean; 586 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-05-11T13:30:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:02 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2932567126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-11T13:30:03.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:02 smithi063 ceph-mon[29809]: pgmap v419: 129 pgs: 129 active+clean; 586 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-05-11T13:30:03.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:02 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2932567126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-11T13:30:03.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:02 smithi071 ceph-mon[33293]: pgmap v419: 129 pgs: 129 active+clean; 586 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-05-11T13:30:03.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:02 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2932567126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-11T13:30:03.635 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-11T13:30:03.635 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-11T13:30:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:03 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2932567126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-11T13:30:04.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:03 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:30:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:04.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:03 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2932567126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-11T13:30:04.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:03 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:30:04.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:03 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2932567126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-11T13:30:04.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:03 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:30:04.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:04 smithi063 ceph-mon[29809]: pgmap v420: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-11T13:30:05.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:04 smithi071 ceph-mon[33293]: pgmap v420: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-11T13:30:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:04 smithi165 ceph-mon[33235]: pgmap v420: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-11T13:30:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 -- 172.21.15.63:0/3028823752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 msgr2=0x7fcbb006c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 --2- 172.21.15.63:0/3028823752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb006c0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009990 tx=0x7fcb9c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 -- 172.21.15.63:0/3028823752 shutdown_connections 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 --2- 172.21.15.63:0/3028823752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbb006c5e0 0x7fcbb0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 --2- 172.21.15.63:0/3028823752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb01095e0 0x7fcbb01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 --2- 172.21.15.63:0/3028823752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb006c0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 -- 172.21.15.63:0/3028823752 >> 172.21.15.63:0/3028823752 conn(0x7fcbb0079400 msgr2=0x7fcbb0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 -- 172.21.15.63:0/3028823752 shutdown_connections 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.369+0000 7fcbb8200700 1 -- 172.21.15.63:0/3028823752 wait complete. 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.370+0000 7fcbb8200700 1 Processor -- start 2024-05-11T13:30:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.370+0000 7fcbb8200700 1 -- start start 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb8200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb8200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb006c5e0 0x7fcbb019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb8200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbb01095e0 0x7fcbb0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb8200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01135a0 con 0x7fcbb006bc20 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb8200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113420 con 0x7fcbb01095e0 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb8200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113720 con 0x7fcbb006c5e0 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb5800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb5800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50734/0 (socket says 172.21.15.63:50734) 2024-05-11T13:30:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.371+0000 7fcbb5800700 1 -- 172.21.15.63:0/4154662631 learned_addr learned my addr 172.21.15.63:0/4154662631 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb4e00700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb006c5e0 0x7fcbb019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb6200700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbb01095e0 0x7fcbb0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb5800700 1 -- 172.21.15.63:0/4154662631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb006c5e0 msgr2=0x7fcbb019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb5800700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb006c5e0 0x7fcbb019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb5800700 1 -- 172.21.15.63:0/4154662631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbb01095e0 msgr2=0x7fcbb0199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb5800700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbb01095e0 0x7fcbb0199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb5800700 1 -- 172.21.15.63:0/4154662631 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb019a080 con 0x7fcbb006bc20 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.372+0000 7fcbb5800700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb019f760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcba400d950 tx=0x7fcba400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.373+0000 7fcbae000700 1 -- 172.21.15.63:0/4154662631 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4016070 con 0x7fcbb006bc20 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.373+0000 7fcbae000700 1 -- 172.21.15.63:0/4154662631 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcba4010460 con 0x7fcbb006bc20 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.373+0000 7fcbae000700 1 -- 172.21.15.63:0/4154662631 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba400f620 con 0x7fcbb006bc20 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.373+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb019a370 con 0x7fcbb006bc20 2024-05-11T13:30:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.373+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a6630 con 0x7fcbb006bc20 2024-05-11T13:30:06.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.374+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0104ca0 con 0x7fcbb006bc20 2024-05-11T13:30:06.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.376+0000 7fcbae000700 1 -- 172.21.15.63:0/4154662631 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcba4019070 con 0x7fcbb006bc20 2024-05-11T13:30:06.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.378+0000 7fcbae000700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcb980b2880 0x7fcb980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:06.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.378+0000 7fcbae000700 1 -- 172.21.15.63:0/4154662631 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fcba40e2190 con 0x7fcbb006bc20 2024-05-11T13:30:06.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.378+0000 7fcbb4e00700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcb980b2880 0x7fcb980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:06.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.379+0000 7fcbb4e00700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcb980b2880 0x7fcb980b4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005850 tx=0x7fcb9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:06.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.383+0000 7fcbae000700 1 -- 172.21.15.63:0/4154662631 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcba40a9af0 con 0x7fcbb006bc20 2024-05-11T13:30:06.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.666+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcbb0063990 con 0x7fcb980b2880 2024-05-11T13:30:06.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.735+0000 7fcbae000700 1 -- 172.21.15.63:0/4154662631 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcbb0063990 con 0x7fcb980b2880 2024-05-11T13:30:06.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcb980b2880 msgr2=0x7fcb980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcb980b2880 0x7fcb980b4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005850 tx=0x7fcb9c01a040 comp rx=0 tx=0).stop 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 msgr2=0x7fcbb019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb019f760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcba400d950 tx=0x7fcba400dd10 comp rx=0 tx=0).stop 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 shutdown_connections 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcb980b2880 0x7fcb980b4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbb01095e0 0x7fcbb0199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbb006bc20 0x7fcbb019f760 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 --2- 172.21.15.63:0/4154662631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb006c5e0 0x7fcbb019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 >> 172.21.15.63:0/4154662631 conn(0x7fcbb0079400 msgr2=0x7fcbb0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 shutdown_connections 2024-05-11T13:30:06.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:06.739+0000 7fcbb8200700 1 -- 172.21.15.63:0/4154662631 wait complete. 2024-05-11T13:30:06.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:06 smithi063 ceph-mon[29809]: pgmap v421: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-11T13:30:06.935 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:30:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:30:06.677+0000 7fa5faa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:30:06.935 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:30:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:30:06.677+0000 7fa5faa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:30:06.935 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:30:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:30:06.677+0000 7fa5faa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:30:06.935 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:30:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:30:06.677+0000 7fa5faa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:30:06.935 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:30:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:30:06.677+0000 7fa5faa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:30:07.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:06 smithi071 ceph-mon[33293]: pgmap v421: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-11T13:30:07.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:06 smithi165 ceph-mon[33235]: pgmap v421: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-11T13:30:07.621 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-11T13:30:08.116 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:08.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:07 smithi063 ceph-mon[29809]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:08.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:07 smithi071 ceph-mon[33293]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:08.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:07 smithi165 ceph-mon[33235]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:08 smithi063 ceph-mon[29809]: pgmap v422: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-11T13:30:08.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:08 smithi063 ceph-mon[29809]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-11T13:30:09.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:08 smithi071 ceph-mon[33293]: pgmap v422: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-11T13:30:09.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:08 smithi071 ceph-mon[33293]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-11T13:30:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:08 smithi165 ceph-mon[33235]: pgmap v422: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-11T13:30:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:08 smithi165 ceph-mon[33235]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-11T13:30:10.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 -- 172.21.15.63:0/2270698267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 msgr2=0x7fa8c8075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2270698267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c8075b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009990 tx=0x7fa8bc009ca0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 -- 172.21.15.63:0/2270698267 shutdown_connections 2024-05-11T13:30:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2270698267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c810ff50 0x7fa8c8112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2270698267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c8075b60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2270698267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80760a0 0x7fa8c810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 -- 172.21.15.63:0/2270698267 >> 172.21.15.63:0/2270698267 conn(0x7fa8c807a700 msgr2=0x7fa8c807ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 -- 172.21.15.63:0/2270698267 shutdown_connections 2024-05-11T13:30:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.501+0000 7fa8d0200700 1 -- 172.21.15.63:0/2270698267 wait complete. 2024-05-11T13:30:10.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.503+0000 7fa8d0200700 1 Processor -- start 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.503+0000 7fa8d0200700 1 -- start start 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.503+0000 7fa8d0200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8d0200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c80760a0 0x7fa8c819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8d0200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c810ff50 0x7fa8c81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8d0200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8115840 con 0x7fa8c8075780 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8d0200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81156c0 con 0x7fa8c810ff50 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8d0200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81159c0 con 0x7fa8c80760a0 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8cd800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8cd800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c819be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47974/0 (socket says 172.21.15.63:47974) 2024-05-11T13:30:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8cd800700 1 -- 172.21.15.63:0/2678876108 learned_addr learned my addr 172.21.15.63:0/2678876108 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8ce200700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c810ff50 0x7fa8c81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.504+0000 7fa8cce00700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c80760a0 0x7fa8c819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8cd800700 1 -- 172.21.15.63:0/2678876108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c80760a0 msgr2=0x7fa8c819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8cd800700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c80760a0 0x7fa8c819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8cd800700 1 -- 172.21.15.63:0/2678876108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c810ff50 msgr2=0x7fa8c81a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8cd800700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c810ff50 0x7fa8c81a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8cd800700 1 -- 172.21.15.63:0/2678876108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c81a0e20 con 0x7fa8c8075780 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8cd800700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c819be10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc006010 tx=0x7fa8bc00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8c6000700 1 -- 172.21.15.63:0/2678876108 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc027070 con 0x7fa8c8075780 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8c6000700 1 -- 172.21.15.63:0/2678876108 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8bc022470 con 0x7fa8c8075780 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8c6000700 1 -- 172.21.15.63:0/2678876108 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc017650 con 0x7fa8c8075780 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.505+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c81a10b0 con 0x7fa8c8075780 2024-05-11T13:30:10.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.506+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81a8990 con 0x7fa8c8075780 2024-05-11T13:30:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.507+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c806a240 con 0x7fa8c8075780 2024-05-11T13:30:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.509+0000 7fa8c6000700 1 -- 172.21.15.63:0/2678876108 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa8bc009d10 con 0x7fa8c8075780 2024-05-11T13:30:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.511+0000 7fa8c6000700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8b00b2910 0x7fa8b00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.511+0000 7fa8c6000700 1 -- 172.21.15.63:0/2678876108 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fa8bc0e33e0 con 0x7fa8c8075780 2024-05-11T13:30:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.511+0000 7fa8cce00700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8b00b2910 0x7fa8b00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.512+0000 7fa8cce00700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8b00b2910 0x7fa8b00b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8111cc0 tx=0x7fa8b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:10.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.520+0000 7fa8c6000700 1 -- 172.21.15.63:0/2678876108 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8bc013070 con 0x7fa8c8075780 2024-05-11T13:30:10.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.795+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa8c819d170 con 0x7fa8b00b2910 2024-05-11T13:30:10.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.875+0000 7fa8c6000700 1 -- 172.21.15.63:0/2678876108 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa8c819d170 con 0x7fa8b00b2910 2024-05-11T13:30:10.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.877+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8b00b2910 msgr2=0x7fa8b00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:10.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.877+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8b00b2910 0x7fa8b00b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8111cc0 tx=0x7fa8b4009260 comp rx=0 tx=0).stop 2024-05-11T13:30:10.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.877+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 msgr2=0x7fa8c819be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:10.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.877+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c819be10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc006010 tx=0x7fa8bc00bfc0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 shutdown_connections 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8b00b2910 0x7fa8b00b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c810ff50 0x7fa8c81a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c8075780 0x7fa8c819be10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 --2- 172.21.15.63:0/2678876108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c80760a0 0x7fa8c819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 >> 172.21.15.63:0/2678876108 conn(0x7fa8c807a700 msgr2=0x7fa8c8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 shutdown_connections 2024-05-11T13:30:10.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:10.878+0000 7fa8d0200700 1 -- 172.21.15.63:0/2678876108 wait complete. 2024-05-11T13:30:11.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:10 smithi063 ceph-mon[29809]: pgmap v423: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 21 op/s 2024-05-11T13:30:11.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:10 smithi071 ceph-mon[33293]: pgmap v423: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 21 op/s 2024-05-11T13:30:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:10 smithi165 ceph-mon[33235]: pgmap v423: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 21 op/s 2024-05-11T13:30:11.739 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-11T13:30:11.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:11 smithi063 ceph-mon[29809]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:12.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:11 smithi071 ceph-mon[33293]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:12.204 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:12.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:11 smithi165 ceph-mon[33235]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:13.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:12 smithi063 ceph-mon[29809]: pgmap v424: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-11T13:30:13.219 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:12 smithi071 ceph-mon[33293]: pgmap v424: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-11T13:30:13.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:12 smithi165 ceph-mon[33235]: pgmap v424: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-11T13:30:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.576+0000 7fa15f200700 1 -- 172.21.15.63:0/4215276158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa158103800 msgr2=0x7fa158103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.576+0000 7fa15f200700 1 --2- 172.21.15.63:0/4215276158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa158103800 0x7fa158103be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa1480099c0 tx=0x7fa148009cd0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.577+0000 7fa15f200700 1 -- 172.21.15.63:0/4215276158 shutdown_connections 2024-05-11T13:30:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.577+0000 7fa15f200700 1 --2- 172.21.15.63:0/4215276158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa158108900 0x7fa158110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.577+0000 7fa15f200700 1 --2- 172.21.15.63:0/4215276158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa158103800 0x7fa158103be0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.577+0000 7fa15f200700 1 --2- 172.21.15.63:0/4215276158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1581041b0 0x7fa1581081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.577+0000 7fa15f200700 1 -- 172.21.15.63:0/4215276158 >> 172.21.15.63:0/4215276158 conn(0x7fa15807a6b0 msgr2=0x7fa15807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.577+0000 7fa15f200700 1 -- 172.21.15.63:0/4215276158 shutdown_connections 2024-05-11T13:30:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.577+0000 7fa15f200700 1 -- 172.21.15.63:0/4215276158 wait complete. 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.578+0000 7fa15f200700 1 Processor -- start 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.578+0000 7fa15f200700 1 -- start start 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa15f200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa158103800 0x7fa1581a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa15f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1581041b0 0x7fa1581a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa15f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa158108900 0x7fa15819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa15f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa158113590 con 0x7fa1581041b0 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa15f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa158113410 con 0x7fa158108900 2024-05-11T13:30:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa15f200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa158113710 con 0x7fa158103800 2024-05-11T13:30:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa157400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1581041b0 0x7fa1581a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa157400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1581041b0 0x7fa1581a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47996/0 (socket says 172.21.15.63:47996) 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa157400700 1 -- 172.21.15.63:0/943535430 learned_addr learned my addr 172.21.15.63:0/943535430 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.579+0000 7fa157e00700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa158103800 0x7fa1581a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa15d200700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa158108900 0x7fa15819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa157400700 1 -- 172.21.15.63:0/943535430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa158103800 msgr2=0x7fa1581a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa157400700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa158103800 0x7fa1581a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa157400700 1 -- 172.21.15.63:0/943535430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa158108900 msgr2=0x7fa15819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa157400700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa158108900 0x7fa15819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa157400700 1 -- 172.21.15.63:0/943535430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15819c250 con 0x7fa1581041b0 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa157400700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1581041b0 0x7fa1581a1e70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d950 tx=0x7fa14400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa154c00700 1 -- 172.21.15.63:0/943535430 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144016070 con 0x7fa1581041b0 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa154c00700 1 -- 172.21.15.63:0/943535430 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa144010460 con 0x7fa1581041b0 2024-05-11T13:30:14.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa154c00700 1 -- 172.21.15.63:0/943535430 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f620 con 0x7fa1581041b0 2024-05-11T13:30:14.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15819c540 con 0x7fa1581041b0 2024-05-11T13:30:14.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.580+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1581a8800 con 0x7fa1581041b0 2024-05-11T13:30:14.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.582+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15806a240 con 0x7fa1581041b0 2024-05-11T13:30:14.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.584+0000 7fa154c00700 1 -- 172.21.15.63:0/943535430 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa144019070 con 0x7fa1581041b0 2024-05-11T13:30:14.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.586+0000 7fa154c00700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa13c0b2990 0x7fa13c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:14.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.586+0000 7fa154c00700 1 -- 172.21.15.63:0/943535430 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fa1440e2320 con 0x7fa1581041b0 2024-05-11T13:30:14.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.586+0000 7fa157e00700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa13c0b2990 0x7fa13c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:14.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.587+0000 7fa157e00700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa13c0b2990 0x7fa13c0b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1480058e0 tx=0x7fa14801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:14.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.588+0000 7fa154c00700 1 -- 172.21.15.63:0/943535430 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1440a9b20 con 0x7fa1581041b0 2024-05-11T13:30:14.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.872+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fa15810ea00 con 0x7fa13c0b2990 2024-05-11T13:30:14.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.885+0000 7fa154c00700 1 -- 172.21.15.63:0/943535430 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa15810ea00 con 0x7fa13c0b2990 2024-05-11T13:30:14.887 INFO:teuthology.orchestra.run.smithi063.stdout:/volumes/qa/sv_0/de410bc9-fc99-432f-8537-cb4b491e8ce2 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.888+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa13c0b2990 msgr2=0x7fa13c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.888+0000 7fa15f200700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa13c0b2990 0x7fa13c0b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1480058e0 tx=0x7fa14801a040 comp rx=0 tx=0).stop 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.888+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1581041b0 msgr2=0x7fa1581a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.888+0000 7fa15f200700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1581041b0 0x7fa1581a1e70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d950 tx=0x7fa14400dd10 comp rx=0 tx=0).stop 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 shutdown_connections 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa13c0b2990 0x7fa13c0b4e50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa158108900 0x7fa15819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa158103800 0x7fa1581a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 --2- 172.21.15.63:0/943535430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1581041b0 0x7fa1581a1e70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:14.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 >> 172.21.15.63:0/943535430 conn(0x7fa15807a6b0 msgr2=0x7fa15810ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:14.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 shutdown_connections 2024-05-11T13:30:14.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:14.889+0000 7fa15f200700 1 -- 172.21.15.63:0/943535430 wait complete. 2024-05-11T13:30:15.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:14 smithi063 ceph-mon[29809]: pgmap v425: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-11T13:30:15.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:14 smithi071 ceph-mon[33293]: pgmap v425: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-11T13:30:15.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:14 smithi165 ceph-mon[33235]: pgmap v425: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-11T13:30:15.755 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-11T13:30:15.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:15 smithi063 ceph-mon[29809]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:16.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:15 smithi071 ceph-mon[33293]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:16.234 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:16.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:15 smithi165 ceph-mon[33235]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:17.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:16 smithi063 ceph-mon[29809]: pgmap v426: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:17.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:16 smithi071 ceph-mon[33293]: pgmap v426: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:17.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:16 smithi165 ceph-mon[33235]: pgmap v426: 129 pgs: 129 active+clean; 594 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:18.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.520+0000 7f777c200700 1 -- 172.21.15.63:0/3355299847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f777406c480 msgr2=0x7f77741103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:18.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.520+0000 7f777c200700 1 --2- 172.21.15.63:0/3355299847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f777406c480 0x7f77741103d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f776c00b0a0 tx=0x7f776c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.520+0000 7f777c200700 1 -- 172.21.15.63:0/3355299847 shutdown_connections 2024-05-11T13:30:18.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.520+0000 7f777c200700 1 --2- 172.21.15.63:0/3355299847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f777406c480 0x7f77741103d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.520+0000 7f777c200700 1 --2- 172.21.15.63:0/3355299847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77741095b0 0x7f7774109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.520+0000 7f777c200700 1 --2- 172.21.15.63:0/3355299847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f777406bac0 0x7f777406bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.520+0000 7f777c200700 1 -- 172.21.15.63:0/3355299847 >> 172.21.15.63:0/3355299847 conn(0x7f77740793c0 msgr2=0x7f77740797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:18.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.521+0000 7f777c200700 1 -- 172.21.15.63:0/3355299847 shutdown_connections 2024-05-11T13:30:18.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.521+0000 7f777c200700 1 -- 172.21.15.63:0/3355299847 wait complete. 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 Processor -- start 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 -- start start 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f777406bac0 0x7f7774102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f777406c480 0x7f7774102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77741095b0 0x7f7774102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77741135e0 con 0x7f77741095b0 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7774113460 con 0x7f777406c480 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.522+0000 7f777c200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7774113760 con 0x7f777406bac0 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77741095b0 0x7f7774102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:18.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77741095b0 0x7f7774102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48014/0 (socket says 172.21.15.63:48014) 2024-05-11T13:30:18.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 -- 172.21.15.63:0/1426237181 learned_addr learned my addr 172.21.15.63:0/1426237181 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:18.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f7778e00700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f777406c480 0x7f7774102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:18.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f7779800700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f777406bac0 0x7f7774102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:18.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 -- 172.21.15.63:0/1426237181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f777406bac0 msgr2=0x7f7774102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f777406bac0 0x7f7774102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 -- 172.21.15.63:0/1426237181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f777406c480 msgr2=0x7f7774102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f777406c480 0x7f7774102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.523+0000 7f777a200700 1 -- 172.21.15.63:0/1426237181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77741a8000 con 0x7f77741095b0 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.524+0000 7f777a200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77741095b0 0x7f7774102f90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f776c00b070 tx=0x7f776c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.524+0000 7f7772000700 1 -- 172.21.15.63:0/1426237181 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c021070 con 0x7f77741095b0 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.524+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77741a8290 con 0x7f77741095b0 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.524+0000 7f7772000700 1 -- 172.21.15.63:0/1426237181 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f776c007b80 con 0x7f77741095b0 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.524+0000 7f7772000700 1 -- 172.21.15.63:0/1426237181 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c004640 con 0x7f77741095b0 2024-05-11T13:30:18.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.524+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741a8740 con 0x7f77741095b0 2024-05-11T13:30:18.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.526+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777406a240 con 0x7f77741095b0 2024-05-11T13:30:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.528+0000 7f7772000700 1 -- 172.21.15.63:0/1426237181 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f776c003a00 con 0x7f77741095b0 2024-05-11T13:30:18.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.529+0000 7f7772000700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f775c0b2990 0x7f775c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.530+0000 7f7772000700 1 -- 172.21.15.63:0/1426237181 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f776c00f020 con 0x7f77741095b0 2024-05-11T13:30:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.530+0000 7f7779800700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f775c0b2990 0x7f775c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:18.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.531+0000 7f7779800700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f775c0b2990 0x7f775c0b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f776800abd0 tx=0x7f776800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.535+0000 7f7772000700 1 -- 172.21.15.63:0/1426237181 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f776c02a020 con 0x7f77741095b0 2024-05-11T13:30:18.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.813+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7774063990 con 0x7f775c0b2990 2024-05-11T13:30:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.878+0000 7f7772000700 1 -- 172.21.15.63:0/1426237181 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7774063990 con 0x7f775c0b2990 2024-05-11T13:30:18.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f775c0b2990 msgr2=0x7f775c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:18.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f775c0b2990 0x7f775c0b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f776800abd0 tx=0x7f776800a380 comp rx=0 tx=0).stop 2024-05-11T13:30:18.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77741095b0 msgr2=0x7f7774102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77741095b0 0x7f7774102f90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f776c00b070 tx=0x7f776c004d50 comp rx=0 tx=0).stop 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 shutdown_connections 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f775c0b2990 0x7f775c0b4e50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77741095b0 0x7f7774102f90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f777406bac0 0x7f7774102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 --2- 172.21.15.63:0/1426237181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f777406c480 0x7f7774102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 >> 172.21.15.63:0/1426237181 conn(0x7f77740793c0 msgr2=0x7f77741010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.883+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 shutdown_connections 2024-05-11T13:30:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:18.884+0000 7f777c200700 1 -- 172.21.15.63:0/1426237181 wait complete. 2024-05-11T13:30:19.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:18 smithi063 ceph-mon[29809]: pgmap v427: 129 pgs: 129 active+clean; 613 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-05-11T13:30:19.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:18 smithi071 ceph-mon[33293]: pgmap v427: 129 pgs: 129 active+clean; 613 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-05-11T13:30:19.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:18 smithi165 ceph-mon[33235]: pgmap v427: 129 pgs: 129 active+clean; 613 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-05-11T13:30:19.760 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-11T13:30:19.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:19 smithi063 ceph-mon[29809]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:20.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:19 smithi071 ceph-mon[33293]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:20.240 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:20.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:19 smithi165 ceph-mon[33235]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:20.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:20 smithi063 ceph-mon[29809]: pgmap v428: 129 pgs: 129 active+clean; 613 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-11T13:30:21.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:20 smithi071 ceph-mon[33293]: pgmap v428: 129 pgs: 129 active+clean; 613 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-11T13:30:21.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:20 smithi165 ceph-mon[33235]: pgmap v428: 129 pgs: 129 active+clean; 613 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-11T13:30:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 -- 172.21.15.63:0/2618654239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390106020 msgr2=0x7f0390112530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 --2- 172.21.15.63:0/2618654239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390106020 0x7f0390112530 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f038800b0a0 tx=0x7f038800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 -- 172.21.15.63:0/2618654239 shutdown_connections 2024-05-11T13:30:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 --2- 172.21.15.63:0/2618654239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390106020 0x7f0390112530 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 --2- 172.21.15.63:0/2618654239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039010b660 0x7f039010ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 --2- 172.21.15.63:0/2618654239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0390105660 0x7f0390105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 -- 172.21.15.63:0/2618654239 >> 172.21.15.63:0/2618654239 conn(0x7f0390101130 msgr2=0x7f0390103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:22.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 -- 172.21.15.63:0/2618654239 shutdown_connections 2024-05-11T13:30:22.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.617+0000 7f0396600700 1 -- 172.21.15.63:0/2618654239 wait complete. 2024-05-11T13:30:22.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.618+0000 7f0396600700 1 Processor -- start 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.618+0000 7f0396600700 1 -- start start 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0396600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390105660 0x7f03901a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0396600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390106020 0x7f03901a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0396600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f039010b660 0x7f039019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0396600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03901157a0 con 0x7f0390105660 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0396600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0390115620 con 0x7f0390106020 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0396600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0390115920 con 0x7f039010b660 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390105660 0x7f03901a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390105660 0x7f03901a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52726/0 (socket says 172.21.15.63:52726) 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395200700 1 -- 172.21.15.63:0/952092485 learned_addr learned my addr 172.21.15.63:0/952092485 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f038fe00700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390106020 0x7f03901a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:22.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395c00700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f039010b660 0x7f039019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395200700 1 -- 172.21.15.63:0/952092485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f039010b660 msgr2=0x7f039019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395200700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f039010b660 0x7f039019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395200700 1 -- 172.21.15.63:0/952092485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390106020 msgr2=0x7f03901a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.619+0000 7f0395200700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390106020 0x7f03901a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.620+0000 7f0395200700 1 -- 172.21.15.63:0/952092485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039019c1a0 con 0x7f0390105660 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.620+0000 7f0395200700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390105660 0x7f03901a1830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f038400d950 tx=0x7f038400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.620+0000 7f038d600700 1 -- 172.21.15.63:0/952092485 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384016070 con 0x7f0390105660 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.620+0000 7f038d600700 1 -- 172.21.15.63:0/952092485 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0384010460 con 0x7f0390105660 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.620+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039019c490 con 0x7f0390105660 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.620+0000 7f038d600700 1 -- 172.21.15.63:0/952092485 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038400f620 con 0x7f0390105660 2024-05-11T13:30:22.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.620+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03901a8700 con 0x7f0390105660 2024-05-11T13:30:22.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.622+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039006a240 con 0x7f0390105660 2024-05-11T13:30:22.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.624+0000 7f038d600700 1 -- 172.21.15.63:0/952092485 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0384019070 con 0x7f0390105660 2024-05-11T13:30:22.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.625+0000 7f038d600700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f03780b2910 0x7f03780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:22.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.626+0000 7f038d600700 1 -- 172.21.15.63:0/952092485 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f03840e2300 con 0x7f0390105660 2024-05-11T13:30:22.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.626+0000 7f038fe00700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f03780b2910 0x7f03780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:22.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.627+0000 7f038fe00700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f03780b2910 0x7f03780b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f037c005ea0 tx=0x7f037c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.628+0000 7f038d600700 1 -- 172.21.15.63:0/952092485 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03840a9b00 con 0x7f0390105660 2024-05-11T13:30:22.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.908+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0390063990 con 0x7f03780b2910 2024-05-11T13:30:22.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.918+0000 7f038d600700 1 -- 172.21.15.63:0/952092485 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0390063990 con 0x7f03780b2910 2024-05-11T13:30:22.919 INFO:teuthology.orchestra.run.smithi063.stdout:/volumes/qa/sv_1/d8cec438-7bc2-4be7-ac34-bb9f7884541b 2024-05-11T13:30:22.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f03780b2910 msgr2=0x7f03780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:22.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f03780b2910 0x7f03780b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f037c005ea0 tx=0x7f037c005e30 comp rx=0 tx=0).stop 2024-05-11T13:30:22.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390105660 msgr2=0x7f03901a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390105660 0x7f03901a1830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f038400d950 tx=0x7f038400dd10 comp rx=0 tx=0).stop 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 shutdown_connections 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f03780b2910 0x7f03780b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f039010b660 0x7f039019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0390105660 0x7f03901a1830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.921+0000 7f0396600700 1 --2- 172.21.15.63:0/952092485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390106020 0x7f03901a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.922+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 >> 172.21.15.63:0/952092485 conn(0x7f0390101130 msgr2=0x7f03901016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:22.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.922+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 shutdown_connections 2024-05-11T13:30:22.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:22.922+0000 7f0396600700 1 -- 172.21.15.63:0/952092485 wait complete. 2024-05-11T13:30:23.186 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:22 smithi063 ceph-mon[29809]: pgmap v429: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-05-11T13:30:23.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:22 smithi071 ceph-mon[33293]: pgmap v429: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-05-11T13:30:23.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:22 smithi165 ceph-mon[33235]: pgmap v429: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-05-11T13:30:23.831 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:30:24.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:23 smithi063 ceph-mon[29809]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:24.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:23 smithi071 ceph-mon[33293]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:24.218 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:23 smithi165 ceph-mon[33235]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:30:24.305 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:25.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:24 smithi063 ceph-mon[29809]: pgmap v430: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-05-11T13:30:25.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:24 smithi071 ceph-mon[33293]: pgmap v430: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-05-11T13:30:25.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:24 smithi165 ceph-mon[33235]: pgmap v430: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 -- 172.21.15.63:0/930178224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 msgr2=0x7f1d24075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 --2- 172.21.15.63:0/930178224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d24075b60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009990 tx=0x7f1d14009ca0 comp rx=0 tx=0).stop 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 -- 172.21.15.63:0/930178224 shutdown_connections 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 --2- 172.21.15.63:0/930178224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2410ff50 0x7f1d24112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 --2- 172.21.15.63:0/930178224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d24075b60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 --2- 172.21.15.63:0/930178224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d240760a0 0x7f1d2410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 -- 172.21.15.63:0/930178224 >> 172.21.15.63:0/930178224 conn(0x7f1d2407a6b0 msgr2=0x7f1d2407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 -- 172.21.15.63:0/930178224 shutdown_connections 2024-05-11T13:30:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.690+0000 7f1d2a000700 1 -- 172.21.15.63:0/930178224 wait complete. 2024-05-11T13:30:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 Processor -- start 2024-05-11T13:30:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 -- start start 2024-05-11T13:30:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d2419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d240760a0 0x7f1d2419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2410ff50 0x7f1d241a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241157d0 con 0x7f1d24075780 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24115650 con 0x7f1d2410ff50 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.692+0000 7f1d2a000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24115950 con 0x7f1d240760a0 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d2419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d2419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52750/0 (socket says 172.21.15.63:52750) 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 -- 172.21.15.63:0/714393186 learned_addr learned my addr 172.21.15.63:0/714393186 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d22a00700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d240760a0 0x7f1d2419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23e00700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2410ff50 0x7f1d241a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 -- 172.21.15.63:0/714393186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d240760a0 msgr2=0x7f1d2419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d240760a0 0x7f1d2419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 -- 172.21.15.63:0/714393186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2410ff50 msgr2=0x7f1d241a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2410ff50 0x7f1d241a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23400700 1 -- 172.21.15.63:0/714393186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d241a0db0 con 0x7f1d24075780 2024-05-11T13:30:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.693+0000 7f1d23e00700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2410ff50 0x7f1d241a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:30:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.694+0000 7f1d23400700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d2419bda0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d140057c0 tx=0x7f1d14009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.694+0000 7f1d1be00700 1 -- 172.21.15.63:0/714393186 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14027070 con 0x7f1d24075780 2024-05-11T13:30:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.694+0000 7f1d1be00700 1 -- 172.21.15.63:0/714393186 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d1400fda0 con 0x7f1d24075780 2024-05-11T13:30:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.694+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d241a1040 con 0x7f1d24075780 2024-05-11T13:30:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.694+0000 7f1d1be00700 1 -- 172.21.15.63:0/714393186 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14022ca0 con 0x7f1d24075780 2024-05-11T13:30:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.694+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a8920 con 0x7f1d24075780 2024-05-11T13:30:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.695+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2406a240 con 0x7f1d24075780 2024-05-11T13:30:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.699+0000 7f1d1be00700 1 -- 172.21.15.63:0/714393186 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1d1402a070 con 0x7f1d24075780 2024-05-11T13:30:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.701+0000 7f1d1be00700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1d080b2a60 0x7f1d080b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.701+0000 7f1d1be00700 1 -- 172.21.15.63:0/714393186 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1d140e36c0 con 0x7f1d24075780 2024-05-11T13:30:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.701+0000 7f1d22a00700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1d080b2a60 0x7f1d080b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.702+0000 7f1d22a00700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1d080b2a60 0x7f1d080b4f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005950 tx=0x7f1d100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.704+0000 7f1d1be00700 1 -- 172.21.15.63:0/714393186 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d14031080 con 0x7f1d24075780 2024-05-11T13:30:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.995+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d2402e570 con 0x7f1d24075780 2024-05-11T13:30:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:26.997+0000 7f1d1be00700 1 -- 172.21.15.63:0/714393186 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f1d14013070 con 0x7f1d24075780 2024-05-11T13:30:26.999 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:30:26.999 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:30:02.730662+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1d080b2a60 msgr2=0x7f1d080b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1d080b2a60 0x7f1d080b4f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005950 tx=0x7f1d100058e0 comp rx=0 tx=0).stop 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 msgr2=0x7f1d2419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d2419bda0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d140057c0 tx=0x7f1d14009f90 comp rx=0 tx=0).stop 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 shutdown_connections 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1d080b2a60 0x7f1d080b4f20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2410ff50 0x7f1d241a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:27.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d24075780 0x7f1d2419bda0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:27.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 --2- 172.21.15.63:0/714393186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d240760a0 0x7f1d2419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:27.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.000+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 >> 172.21.15.63:0/714393186 conn(0x7f1d2407a6b0 msgr2=0x7f1d24101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:27.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.001+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 shutdown_connections 2024-05-11T13:30:27.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:27.001+0000 7f1d2a000700 1 -- 172.21.15.63:0/714393186 wait complete. 2024-05-11T13:30:27.004 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-11T13:30:27.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:26 smithi063 ceph-mon[29809]: pgmap v431: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-05-11T13:30:27.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:26 smithi071 ceph-mon[33293]: pgmap v431: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-05-11T13:30:27.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:26 smithi165 ceph-mon[33235]: pgmap v431: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-05-11T13:30:27.937 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:30:28.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:27 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/714393186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:30:28.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:27 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/714393186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:30:28.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:27 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/714393186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:30:28.387 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:29.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:28 smithi063 ceph-mon[29809]: pgmap v432: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-05-11T13:30:29.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:28 smithi071 ceph-mon[33293]: pgmap v432: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-05-11T13:30:29.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:28 smithi165 ceph-mon[33235]: pgmap v432: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-05-11T13:30:30.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.711+0000 7faf6d000700 1 -- 172.21.15.63:0/4232445873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf681095b0 msgr2=0x7faf68109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:30.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.711+0000 7faf6d000700 1 --2- 172.21.15.63:0/4232445873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf681095b0 0x7faf68109990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faf58009990 tx=0x7faf58009ca0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.711+0000 7faf6d000700 1 -- 172.21.15.63:0/4232445873 shutdown_connections 2024-05-11T13:30:30.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.711+0000 7faf6d000700 1 --2- 172.21.15.63:0/4232445873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf6806c480 0x7faf681103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.711+0000 7faf6d000700 1 --2- 172.21.15.63:0/4232445873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf681095b0 0x7faf68109990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.711+0000 7faf6d000700 1 --2- 172.21.15.63:0/4232445873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6806bac0 0x7faf6806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.711+0000 7faf6d000700 1 -- 172.21.15.63:0/4232445873 >> 172.21.15.63:0/4232445873 conn(0x7faf680793c0 msgr2=0x7faf680797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:30.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.712+0000 7faf6d000700 1 -- 172.21.15.63:0/4232445873 shutdown_connections 2024-05-11T13:30:30.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.712+0000 7faf6d000700 1 -- 172.21.15.63:0/4232445873 wait complete. 2024-05-11T13:30:30.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.714+0000 7faf6d000700 1 Processor -- start 2024-05-11T13:30:30.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.714+0000 7faf6d000700 1 -- start start 2024-05-11T13:30:30.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.714+0000 7faf6d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6806bac0 0x7faf6819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:30.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.714+0000 7faf6d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6806c480 0x7faf6819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:30.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.714+0000 7faf6d000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf681095b0 0x7faf681a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:30.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf6d000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf68113590 con 0x7faf6806bac0 2024-05-11T13:30:30.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf6d000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf68113410 con 0x7faf6806c480 2024-05-11T13:30:30.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf6d000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faf68113710 con 0x7faf681095b0 2024-05-11T13:30:30.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf65e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6806bac0 0x7faf6819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:30.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf65e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6806bac0 0x7faf6819bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54100/0 (socket says 172.21.15.63:54100) 2024-05-11T13:30:30.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf65e00700 1 -- 172.21.15.63:0/3117112309 learned_addr learned my addr 172.21.15.63:0/3117112309 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:30.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf66800700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf681095b0 0x7faf681a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:30.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.715+0000 7faf65400700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6806c480 0x7faf6819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf65e00700 1 -- 172.21.15.63:0/3117112309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf681095b0 msgr2=0x7faf681a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf65e00700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf681095b0 0x7faf681a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf65e00700 1 -- 172.21.15.63:0/3117112309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6806c480 msgr2=0x7faf6819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf65e00700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6806c480 0x7faf6819c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf65e00700 1 -- 172.21.15.63:0/3117112309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf681a0d70 con 0x7faf6806bac0 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf65e00700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6806bac0 0x7faf6819bd60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf580057c0 tx=0x7faf58009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf5ea00700 1 -- 172.21.15.63:0/3117112309 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58027070 con 0x7faf6806bac0 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf5ea00700 1 -- 172.21.15.63:0/3117112309 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf5800fda0 con 0x7faf6806bac0 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.716+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf681a1000 con 0x7faf6806bac0 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.717+0000 7faf5ea00700 1 -- 172.21.15.63:0/3117112309 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58022ca0 con 0x7faf6806bac0 2024-05-11T13:30:30.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.717+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf681a88e0 con 0x7faf6806bac0 2024-05-11T13:30:30.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.718+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6806a240 con 0x7faf6806bac0 2024-05-11T13:30:30.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.720+0000 7faf5ea00700 1 -- 172.21.15.63:0/3117112309 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7faf5802a070 con 0x7faf6806bac0 2024-05-11T13:30:30.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.722+0000 7faf5ea00700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7faf4c0b2990 0x7faf4c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:30.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.722+0000 7faf5ea00700 1 -- 172.21.15.63:0/3117112309 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7faf580e3740 con 0x7faf6806bac0 2024-05-11T13:30:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.722+0000 7faf65400700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7faf4c0b2990 0x7faf4c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.723+0000 7faf65400700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7faf4c0b2990 0x7faf4c0b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf54005950 tx=0x7faf540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.729+0000 7faf5ea00700 1 -- 172.21.15.63:0/3117112309 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf58013070 con 0x7faf6806bac0 2024-05-11T13:30:30.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.974+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf681a8ab0 con 0x7faf6806bac0 2024-05-11T13:30:30.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.978+0000 7faf5ea00700 1 -- 172.21.15.63:0/3117112309 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24619 (secure 0 0 0) 0x7faf580095a0 con 0x7faf6806bac0 2024-05-11T13:30:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:30:30.979 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":88,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:29:40.520411+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:30:30.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7faf4c0b2990 msgr2=0x7faf4c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:30.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7faf4c0b2990 0x7faf4c0b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf54005950 tx=0x7faf540058e0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6806bac0 msgr2=0x7faf6819bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:30.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6806bac0 0x7faf6819bd60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf580057c0 tx=0x7faf58009f90 comp rx=0 tx=0).stop 2024-05-11T13:30:30.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 shutdown_connections 2024-05-11T13:30:30.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7faf4c0b2990 0x7faf4c0b4e50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf681095b0 0x7faf681a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6806bac0 0x7faf6819bd60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 --2- 172.21.15.63:0/3117112309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6806c480 0x7faf6819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:30.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.981+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 >> 172.21.15.63:0/3117112309 conn(0x7faf680793c0 msgr2=0x7faf6810e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:30.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.982+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 shutdown_connections 2024-05-11T13:30:30.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:30.982+0000 7faf6d000700 1 -- 172.21.15.63:0/3117112309 wait complete. 2024-05-11T13:30:31.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:30 smithi071 ceph-mon[33293]: pgmap v433: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-11T13:30:31.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:30 smithi165 ceph-mon[33235]: pgmap v433: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-11T13:30:31.391 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:30 smithi063 ceph-mon[29809]: pgmap v433: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-11T13:30:31.950 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-11T13:30:31.950 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T13:30:32.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:31 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3117112309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:32.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:31 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3117112309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:32.438 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:32.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:31 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3117112309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:33.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:32 smithi165 ceph-mon[33235]: pgmap v434: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-11T13:30:33.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:32 smithi063 ceph-mon[29809]: pgmap v434: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-11T13:30:33.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:32 smithi071 ceph-mon[33293]: pgmap v434: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-11T13:30:34.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 -- 172.21.15.63:0/1315354133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 msgr2=0x7f80ec101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:34.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 --2- 172.21.15.63:0/1315354133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec101fc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f80d80099c0 tx=0x7f80d8009cd0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 -- 172.21.15.63:0/1315354133 shutdown_connections 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 --2- 172.21.15.63:0/1315354133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ec102500 0x7f80ec1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 --2- 172.21.15.63:0/1315354133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80ec109530 0x7f80ec109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 --2- 172.21.15.63:0/1315354133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec101fc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 -- 172.21.15.63:0/1315354133 >> 172.21.15.63:0/1315354133 conn(0x7f80ec078ac0 msgr2=0x7f80ec0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 -- 172.21.15.63:0/1315354133 shutdown_connections 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.661+0000 7f80f2800700 1 -- 172.21.15.63:0/1315354133 wait complete. 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.662+0000 7f80f2800700 1 Processor -- start 2024-05-11T13:30:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.662+0000 7f80f2800700 1 -- start start 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80f2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80f2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80ec102500 0x7f80ec19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80f2800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ec109530 0x7f80ec1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80f2800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec113610 con 0x7f80ec101b40 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80f2800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec113490 con 0x7f80ec102500 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80f2800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec113790 con 0x7f80ec109530 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80eb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80eb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54120/0 (socket says 172.21.15.63:54120) 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80eb400700 1 -- 172.21.15.63:0/1954167965 learned_addr learned my addr 172.21.15.63:0/1954167965 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80eaa00700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80ec102500 0x7f80ec19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80ebe00700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ec109530 0x7f80ec1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80eb400700 1 -- 172.21.15.63:0/1954167965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ec109530 msgr2=0x7f80ec1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80eb400700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ec109530 0x7f80ec1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.663+0000 7f80eb400700 1 -- 172.21.15.63:0/1954167965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80ec102500 msgr2=0x7f80ec19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80eb400700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80ec102500 0x7f80ec19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80eb400700 1 -- 172.21.15.63:0/1954167965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec1a0d50 con 0x7f80ec101b40 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80eaa00700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80ec102500 0x7f80ec19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80eb400700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec19bd40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00d950 tx=0x7f80dc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80e3e00700 1 -- 172.21.15.63:0/1954167965 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc016070 con 0x7f80ec101b40 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80e3e00700 1 -- 172.21.15.63:0/1954167965 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80dc010460 con 0x7f80ec101b40 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec1a1040 con 0x7f80ec101b40 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80e3e00700 1 -- 172.21.15.63:0/1954167965 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00f620 con 0x7f80ec101b40 2024-05-11T13:30:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.664+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec1a88c0 con 0x7f80ec101b40 2024-05-11T13:30:34.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.666+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec06a240 con 0x7f80ec101b40 2024-05-11T13:30:34.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.668+0000 7f80e3e00700 1 -- 172.21.15.63:0/1954167965 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f80dc019070 con 0x7f80ec101b40 2024-05-11T13:30:34.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.669+0000 7f80e3e00700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f80d00b2910 0x7f80d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:34.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.670+0000 7f80e3e00700 1 -- 172.21.15.63:0/1954167965 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f80dc0e2320 con 0x7f80ec101b40 2024-05-11T13:30:34.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.670+0000 7f80eaa00700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f80d00b2910 0x7f80d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:34.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.671+0000 7f80eaa00700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f80d00b2910 0x7f80d00b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f80d8000c00 tx=0x7f80d801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:34.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.674+0000 7f80e3e00700 1 -- 172.21.15.63:0/1954167965 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80dc0a9b20 con 0x7f80ec101b40 2024-05-11T13:30:34.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.908+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80ec100700 con 0x7f80d00b2910 2024-05-11T13:30:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.927+0000 7f80e3e00700 1 -- 172.21.15.63:0/1954167965 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340433 (secure 0 0 0) 0x7f80ec100700 con 0x7f80d00b2910 2024-05-11T13:30:34.935 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.934+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f80d00b2910 msgr2=0x7f80d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.934+0000 7f80f2800700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f80d00b2910 0x7f80d00b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f80d8000c00 tx=0x7f80d801a040 comp rx=0 tx=0).stop 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 msgr2=0x7f80ec19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec19bd40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00d950 tx=0x7f80dc00dd10 comp rx=0 tx=0).stop 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 shutdown_connections 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f80d00b2910 0x7f80d00b4dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ec109530 0x7f80ec1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80ec101b40 0x7f80ec19bd40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 --2- 172.21.15.63:0/1954167965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80ec102500 0x7f80ec19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 >> 172.21.15.63:0/1954167965 conn(0x7f80ec078ac0 msgr2=0x7f80ec0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 shutdown_connections 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:34.935+0000 7f80f2800700 1 -- 172.21.15.63:0/1954167965 wait complete. 2024-05-11T13:30:34.938 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-11T13:30:35.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:34 smithi063 ceph-mon[29809]: pgmap v435: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:35.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:34 smithi165 ceph-mon[33235]: pgmap v435: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:35.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:34 smithi071 ceph-mon[33293]: pgmap v435: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:35.807 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-05-11T13:30:33.595615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660463,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":448,"num_read_kb":329,"num_write":373,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375580,"kb_used_data":13948,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566244,"statfs":{"total":1151940427776,"available":1151555833856,"internally_reserved":0,"allocated":14282752,"data_stored":7944546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9668,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":66,"num_write":10,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006510"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537148+0000","last_change":"2024-05-11T13:28:35.223417+0000","last_active":"2024-05-11T13:29:40.537148+0000","last_peered":"2024-05-11T13:29:40.537148+0000","last_clean":"2024-05-11T13:29:40.537148+0000","last_became_active":"2024-05-11T13:28:35.222862+0000","last_became_peered":"2024-05-11T13:28:35.222862+0000","last_unstale":"2024-05-11T13:29:40.537148+0000","last_undegraded":"2024-05-11T13:29:40.537148+0000","last_fullsized":"2024-05-11T13:29:40.537148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:20.259043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844676+0000","last_change":"2024-05-11T13:28:35.222289+0000","last_active":"2024-05-11T13:29:40.844676+0000","last_peered":"2024-05-11T13:29:40.844676+0000","last_clean":"2024-05-11T13:29:40.844676+0000","last_became_active":"2024-05-11T13:28:35.221366+0000","last_became_peered":"2024-05-11T13:28:35.221366+0000","last_unstale":"2024-05-11T13:29:40.844676+0000","last_undegraded":"2024-05-11T13:29:40.844676+0000","last_fullsized":"2024-05-11T13:29:40.844676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:02.648140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843736+0000","last_change":"2024-05-11T13:28:35.221271+0000","last_active":"2024-05-11T13:29:40.843736+0000","last_peered":"2024-05-11T13:29:40.843736+0000","last_clean":"2024-05-11T13:29:40.843736+0000","last_became_active":"2024-05-11T13:28:35.220294+0000","last_became_peered":"2024-05-11T13:28:35.220294+0000","last_unstale":"2024-05-11T13:29:40.843736+0000","last_undegraded":"2024-05-11T13:29:40.843736+0000","last_fullsized":"2024-05-11T13:29:40.843736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:04.608042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550631+0000","last_change":"2024-05-11T13:28:35.197643+0000","last_active":"2024-05-11T13:29:40.550631+0000","last_peered":"2024-05-11T13:29:40.550631+0000","last_clean":"2024-05-11T13:29:40.550631+0000","last_became_active":"2024-05-11T13:28:35.196528+0000","last_became_peered":"2024-05-11T13:28:35.196528+0000","last_unstale":"2024-05-11T13:29:40.550631+0000","last_undegraded":"2024-05-11T13:29:40.550631+0000","last_fullsized":"2024-05-11T13:29:40.550631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:02:03.088456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550105+0000","last_change":"2024-05-11T13:28:35.206833+0000","last_active":"2024-05-11T13:29:40.550105+0000","last_peered":"2024-05-11T13:29:40.550105+0000","last_clean":"2024-05-11T13:29:40.550105+0000","last_became_active":"2024-05-11T13:28:35.206287+0000","last_became_peered":"2024-05-11T13:28:35.206287+0000","last_unstale":"2024-05-11T13:29:40.550105+0000","last_undegraded":"2024-05-11T13:29:40.550105+0000","last_fullsized":"2024-05-11T13:29:40.550105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:25:20.576464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.538289+0000","last_change":"2024-05-11T13:28:35.218950+0000","last_active":"2024-05-11T13:29:40.538289+0000","last_peered":"2024-05-11T13:29:40.538289+0000","last_clean":"2024-05-11T13:29:40.538289+0000","last_became_active":"2024-05-11T13:28:35.218597+0000","last_became_peered":"2024-05-11T13:28:35.218597+0000","last_unstale":"2024-05-11T13:29:40.538289+0000","last_undegraded":"2024-05-11T13:29:40.538289+0000","last_fullsized":"2024-05-11T13:29:40.538289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:08:42.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.541106+0000","last_change":"2024-05-11T13:29:40.541106+0000","last_active":"2024-05-11T13:29:40.541106+0000","last_peered":"2024-05-11T13:29:40.541106+0000","last_clean":"2024-05-11T13:29:40.541106+0000","last_became_active":"2024-05-11T13:29:40.540007+0000","last_became_peered":"2024-05-11T13:29:40.540007+0000","last_unstale":"2024-05-11T13:29:40.541106+0000","last_undegraded":"2024-05-11T13:29:40.541106+0000","last_fullsized":"2024-05-11T13:29:40.541106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:00.596580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545783+0000","last_change":"2024-05-11T13:28:35.205483+0000","last_active":"2024-05-11T13:29:40.545783+0000","last_peered":"2024-05-11T13:29:40.545783+0000","last_clean":"2024-05-11T13:29:40.545783+0000","last_became_active":"2024-05-11T13:28:35.205101+0000","last_became_peered":"2024-05-11T13:28:35.205101+0000","last_unstale":"2024-05-11T13:29:40.545783+0000","last_undegraded":"2024-05-11T13:29:40.545783+0000","last_fullsized":"2024-05-11T13:29:40.545783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:37.917464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.552796+0000","last_change":"2024-05-11T13:28:35.217372+0000","last_active":"2024-05-11T13:29:40.552796+0000","last_peered":"2024-05-11T13:29:40.552796+0000","last_clean":"2024-05-11T13:29:40.552796+0000","last_became_active":"2024-05-11T13:28:35.217021+0000","last_became_peered":"2024-05-11T13:28:35.217021+0000","last_unstale":"2024-05-11T13:29:40.552796+0000","last_undegraded":"2024-05-11T13:29:40.552796+0000","last_fullsized":"2024-05-11T13:29:40.552796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:24.273041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548634+0000","last_change":"2024-05-11T13:29:40.548634+0000","last_active":"2024-05-11T13:29:40.548634+0000","last_peered":"2024-05-11T13:29:40.548634+0000","last_clean":"2024-05-11T13:29:40.548634+0000","last_became_active":"2024-05-11T13:29:40.547536+0000","last_became_peered":"2024-05-11T13:29:40.547536+0000","last_unstale":"2024-05-11T13:29:40.548634+0000","last_undegraded":"2024-05-11T13:29:40.548634+0000","last_fullsized":"2024-05-11T13:29:40.548634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:15.362317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531686+0000","last_change":"2024-05-11T13:28:35.214091+0000","last_active":"2024-05-11T13:29:40.531686+0000","last_peered":"2024-05-11T13:29:40.531686+0000","last_clean":"2024-05-11T13:29:40.531686+0000","last_became_active":"2024-05-11T13:28:35.213668+0000","last_became_peered":"2024-05-11T13:28:35.213668+0000","last_unstale":"2024-05-11T13:29:40.531686+0000","last_undegraded":"2024-05-11T13:29:40.531686+0000","last_fullsized":"2024-05-11T13:29:40.531686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:19:55.768203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550431+0000","last_change":"2024-05-11T13:28:35.211874+0000","last_active":"2024-05-11T13:29:40.550431+0000","last_peered":"2024-05-11T13:29:40.550431+0000","last_clean":"2024-05-11T13:29:40.550431+0000","last_became_active":"2024-05-11T13:28:35.211503+0000","last_became_peered":"2024-05-11T13:28:35.211503+0000","last_unstale":"2024-05-11T13:29:40.550431+0000","last_undegraded":"2024-05-11T13:29:40.550431+0000","last_fullsized":"2024-05-11T13:29:40.550431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:46.884110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537167+0000","last_change":"2024-05-11T13:28:35.209308+0000","last_active":"2024-05-11T13:29:40.537167+0000","last_peered":"2024-05-11T13:29:40.537167+0000","last_clean":"2024-05-11T13:29:40.537167+0000","last_became_active":"2024-05-11T13:28:35.207927+0000","last_became_peered":"2024-05-11T13:28:35.207927+0000","last_unstale":"2024-05-11T13:29:40.537167+0000","last_undegraded":"2024-05-11T13:29:40.537167+0000","last_fullsized":"2024-05-11T13:29:40.537167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:04.541849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843334+0000","last_change":"2024-05-11T13:28:35.208524+0000","last_active":"2024-05-11T13:29:40.843334+0000","last_peered":"2024-05-11T13:29:40.843334+0000","last_clean":"2024-05-11T13:29:40.843334+0000","last_became_active":"2024-05-11T13:28:35.207875+0000","last_became_peered":"2024-05-11T13:28:35.207875+0000","last_unstale":"2024-05-11T13:29:40.843334+0000","last_undegraded":"2024-05-11T13:29:40.843334+0000","last_fullsized":"2024-05-11T13:29:40.843334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:23:38.896916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843412+0000","last_change":"2024-05-11T13:28:35.216418+0000","last_active":"2024-05-11T13:29:40.843412+0000","last_peered":"2024-05-11T13:29:40.843412+0000","last_clean":"2024-05-11T13:29:40.843412+0000","last_became_active":"2024-05-11T13:28:35.215984+0000","last_became_peered":"2024-05-11T13:28:35.215984+0000","last_unstale":"2024-05-11T13:29:40.843412+0000","last_undegraded":"2024-05-11T13:29:40.843412+0000","last_fullsized":"2024-05-11T13:29:40.843412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:15:13.650821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548506+0000","last_change":"2024-05-11T13:28:35.217701+0000","last_active":"2024-05-11T13:29:40.548506+0000","last_peered":"2024-05-11T13:29:40.548506+0000","last_clean":"2024-05-11T13:29:40.548506+0000","last_became_active":"2024-05-11T13:28:35.217050+0000","last_became_peered":"2024-05-11T13:28:35.217050+0000","last_unstale":"2024-05-11T13:29:40.548506+0000","last_undegraded":"2024-05-11T13:29:40.548506+0000","last_fullsized":"2024-05-11T13:29:40.548506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:29:38.274540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536406+0000","last_change":"2024-05-11T13:28:35.223794+0000","last_active":"2024-05-11T13:29:40.536406+0000","last_peered":"2024-05-11T13:29:40.536406+0000","last_clean":"2024-05-11T13:29:40.536406+0000","last_became_active":"2024-05-11T13:28:35.223388+0000","last_became_peered":"2024-05-11T13:28:35.223388+0000","last_unstale":"2024-05-11T13:29:40.536406+0000","last_undegraded":"2024-05-11T13:29:40.536406+0000","last_fullsized":"2024-05-11T13:29:40.536406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:56:13.272357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.528208+0000","last_change":"2024-05-11T13:28:35.216452+0000","last_active":"2024-05-11T13:29:40.528208+0000","last_peered":"2024-05-11T13:29:40.528208+0000","last_clean":"2024-05-11T13:29:40.528208+0000","last_became_active":"2024-05-11T13:28:35.216075+0000","last_became_peered":"2024-05-11T13:28:35.216075+0000","last_unstale":"2024-05-11T13:29:40.528208+0000","last_undegraded":"2024-05-11T13:29:40.528208+0000","last_fullsized":"2024-05-11T13:29:40.528208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:23.012302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535020+0000","last_change":"2024-05-11T13:28:35.206528+0000","last_active":"2024-05-11T13:29:40.535020+0000","last_peered":"2024-05-11T13:29:40.535020+0000","last_clean":"2024-05-11T13:29:40.535020+0000","last_became_active":"2024-05-11T13:28:35.206102+0000","last_became_peered":"2024-05-11T13:28:35.206102+0000","last_unstale":"2024-05-11T13:29:40.535020+0000","last_undegraded":"2024-05-11T13:29:40.535020+0000","last_fullsized":"2024-05-11T13:29:40.535020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.685477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533743+0000","last_change":"2024-05-11T13:28:49.683087+0000","last_active":"2024-05-11T13:29:40.533743+0000","last_peered":"2024-05-11T13:29:40.533743+0000","last_clean":"2024-05-11T13:29:40.533743+0000","last_became_active":"2024-05-11T13:28:49.681968+0000","last_became_peered":"2024-05-11T13:28:49.681968+0000","last_unstale":"2024-05-11T13:29:40.533743+0000","last_undegraded":"2024-05-11T13:29:40.533743+0000","last_fullsized":"2024-05-11T13:29:40.533743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:08:04.594962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":85,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:28.879850+0000","last_change":"2024-05-11T13:28:30.428076+0000","last_active":"2024-05-11T13:30:28.879850+0000","last_peered":"2024-05-11T13:30:28.879850+0000","last_clean":"2024-05-11T13:30:28.879850+0000","last_became_active":"2024-05-11T13:28:30.427519+0000","last_became_peered":"2024-05-11T13:28:30.427519+0000","last_unstale":"2024-05-11T13:30:28.879850+0000","last_undegraded":"2024-05-11T13:30:28.879850+0000","last_fullsized":"2024-05-11T13:30:28.879850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:52.808051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546766+0000","last_change":"2024-05-11T13:28:35.207587+0000","last_active":"2024-05-11T13:29:40.546766+0000","last_peered":"2024-05-11T13:29:40.546766+0000","last_clean":"2024-05-11T13:29:40.546766+0000","last_became_active":"2024-05-11T13:28:35.206692+0000","last_became_peered":"2024-05-11T13:28:35.206692+0000","last_unstale":"2024-05-11T13:29:40.546766+0000","last_undegraded":"2024-05-11T13:29:40.546766+0000","last_fullsized":"2024-05-11T13:29:40.546766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:25.171342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545088+0000","last_change":"2024-05-11T13:28:49.682649+0000","last_active":"2024-05-11T13:29:40.545088+0000","last_peered":"2024-05-11T13:29:40.545088+0000","last_clean":"2024-05-11T13:29:40.545088+0000","last_became_active":"2024-05-11T13:28:49.682192+0000","last_became_peered":"2024-05-11T13:28:49.682192+0000","last_unstale":"2024-05-11T13:29:40.545088+0000","last_undegraded":"2024-05-11T13:29:40.545088+0000","last_fullsized":"2024-05-11T13:29:40.545088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:01.728705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548440+0000","last_change":"2024-05-11T13:28:30.707050+0000","last_active":"2024-05-11T13:29:40.548440+0000","last_peered":"2024-05-11T13:29:40.548440+0000","last_clean":"2024-05-11T13:29:40.548440+0000","last_became_active":"2024-05-11T13:28:30.706279+0000","last_became_peered":"2024-05-11T13:28:30.706279+0000","last_unstale":"2024-05-11T13:29:40.548440+0000","last_undegraded":"2024-05-11T13:29:40.548440+0000","last_fullsized":"2024-05-11T13:29:40.548440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:14.135734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844412+0000","last_change":"2024-05-11T13:28:35.219622+0000","last_active":"2024-05-11T13:29:40.844412+0000","last_peered":"2024-05-11T13:29:40.844412+0000","last_clean":"2024-05-11T13:29:40.844412+0000","last_became_active":"2024-05-11T13:28:35.218687+0000","last_became_peered":"2024-05-11T13:28:35.218687+0000","last_unstale":"2024-05-11T13:29:40.844412+0000","last_undegraded":"2024-05-11T13:29:40.844412+0000","last_fullsized":"2024-05-11T13:29:40.844412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:53.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545586+0000","last_change":"2024-05-11T13:28:49.685770+0000","last_active":"2024-05-11T13:29:40.545586+0000","last_peered":"2024-05-11T13:29:40.545586+0000","last_clean":"2024-05-11T13:29:40.545586+0000","last_became_active":"2024-05-11T13:28:49.685091+0000","last_became_peered":"2024-05-11T13:28:49.685091+0000","last_unstale":"2024-05-11T13:29:40.545586+0000","last_undegraded":"2024-05-11T13:29:40.545586+0000","last_fullsized":"2024-05-11T13:29:40.545586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:42.458310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:58.785640+0000","last_change":"2024-05-11T13:28:30.435304+0000","last_active":"2024-05-11T13:29:58.785640+0000","last_peered":"2024-05-11T13:29:58.785640+0000","last_clean":"2024-05-11T13:29:58.785640+0000","last_became_active":"2024-05-11T13:28:30.434648+0000","last_became_peered":"2024-05-11T13:28:30.434648+0000","last_unstale":"2024-05-11T13:29:58.785640+0000","last_undegraded":"2024-05-11T13:29:58.785640+0000","last_fullsized":"2024-05-11T13:29:58.785640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:21.553376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548611+0000","last_change":"2024-05-11T13:28:35.210877+0000","last_active":"2024-05-11T13:29:40.548611+0000","last_peered":"2024-05-11T13:29:40.548611+0000","last_clean":"2024-05-11T13:29:40.548611+0000","last_became_active":"2024-05-11T13:28:35.207069+0000","last_became_peered":"2024-05-11T13:28:35.207069+0000","last_unstale":"2024-05-11T13:29:40.548611+0000","last_undegraded":"2024-05-11T13:29:40.548611+0000","last_fullsized":"2024-05-11T13:29:40.548611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:59.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545234+0000","last_change":"2024-05-11T13:28:49.689070+0000","last_active":"2024-05-11T13:29:40.545234+0000","last_peered":"2024-05-11T13:29:40.545234+0000","last_clean":"2024-05-11T13:29:40.545234+0000","last_became_active":"2024-05-11T13:28:49.688502+0000","last_became_peered":"2024-05-11T13:28:49.688502+0000","last_unstale":"2024-05-11T13:29:40.545234+0000","last_undegraded":"2024-05-11T13:29:40.545234+0000","last_fullsized":"2024-05-11T13:29:40.545234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:21.516074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778649+0000","last_change":"2024-05-11T13:29:40.564494+0000","last_active":"2024-05-11T13:29:57.778649+0000","last_peered":"2024-05-11T13:29:57.778649+0000","last_clean":"2024-05-11T13:29:57.778649+0000","last_became_active":"2024-05-11T13:29:40.555386+0000","last_became_peered":"2024-05-11T13:29:40.555386+0000","last_unstale":"2024-05-11T13:29:57.778649+0000","last_undegraded":"2024-05-11T13:29:57.778649+0000","last_fullsized":"2024-05-11T13:29:57.778649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:32:50.641372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.542113+0000","last_change":"2024-05-11T13:28:35.217950+0000","last_active":"2024-05-11T13:29:40.542113+0000","last_peered":"2024-05-11T13:29:40.542113+0000","last_clean":"2024-05-11T13:29:40.542113+0000","last_became_active":"2024-05-11T13:28:35.216698+0000","last_became_peered":"2024-05-11T13:28:35.216698+0000","last_unstale":"2024-05-11T13:29:40.542113+0000","last_undegraded":"2024-05-11T13:29:40.542113+0000","last_fullsized":"2024-05-11T13:29:40.542113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:01.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.534360+0000","last_change":"2024-05-11T13:28:49.687889+0000","last_active":"2024-05-11T13:29:40.534360+0000","last_peered":"2024-05-11T13:29:40.534360+0000","last_clean":"2024-05-11T13:29:40.534360+0000","last_became_active":"2024-05-11T13:28:49.684980+0000","last_became_peered":"2024-05-11T13:28:49.684980+0000","last_unstale":"2024-05-11T13:29:40.534360+0000","last_undegraded":"2024-05-11T13:29:40.534360+0000","last_fullsized":"2024-05-11T13:29:40.534360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:02:59.751371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:30.890498+0000","last_change":"2024-05-11T13:29:40.555238+0000","last_active":"2024-05-11T13:30:30.890498+0000","last_peered":"2024-05-11T13:30:30.890498+0000","last_clean":"2024-05-11T13:30:30.890498+0000","last_became_active":"2024-05-11T13:29:40.554661+0000","last_became_peered":"2024-05-11T13:29:40.554661+0000","last_unstale":"2024-05-11T13:30:30.890498+0000","last_undegraded":"2024-05-11T13:30:30.890498+0000","last_fullsized":"2024-05-11T13:30:30.890498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:22.983649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549668+0000","last_change":"2024-05-11T13:28:35.209163+0000","last_active":"2024-05-11T13:29:40.549668+0000","last_peered":"2024-05-11T13:29:40.549668+0000","last_clean":"2024-05-11T13:29:40.549668+0000","last_became_active":"2024-05-11T13:28:35.204014+0000","last_became_peered":"2024-05-11T13:28:35.204014+0000","last_unstale":"2024-05-11T13:29:40.549668+0000","last_undegraded":"2024-05-11T13:29:40.549668+0000","last_fullsized":"2024-05-11T13:29:40.549668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:17.570863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.555520+0000","last_change":"2024-05-11T13:29:40.555520+0000","last_active":"2024-05-11T13:29:40.555520+0000","last_peered":"2024-05-11T13:29:40.555520+0000","last_clean":"2024-05-11T13:29:40.555520+0000","last_became_active":"2024-05-11T13:29:40.553952+0000","last_became_peered":"2024-05-11T13:29:40.553952+0000","last_unstale":"2024-05-11T13:29:40.555520+0000","last_undegraded":"2024-05-11T13:29:40.555520+0000","last_fullsized":"2024-05-11T13:29:40.555520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.035691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.761820+0000","last_change":"2024-05-11T13:28:30.429829+0000","last_active":"2024-05-11T13:29:56.761820+0000","last_peered":"2024-05-11T13:29:56.761820+0000","last_clean":"2024-05-11T13:29:56.761820+0000","last_became_active":"2024-05-11T13:28:30.428818+0000","last_became_peered":"2024-05-11T13:28:30.428818+0000","last_unstale":"2024-05-11T13:29:56.761820+0000","last_undegraded":"2024-05-11T13:29:56.761820+0000","last_fullsized":"2024-05-11T13:29:56.761820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:59:25.653181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543583+0000","last_change":"2024-05-11T13:28:35.214153+0000","last_active":"2024-05-11T13:29:40.543583+0000","last_peered":"2024-05-11T13:29:40.543583+0000","last_clean":"2024-05-11T13:29:40.543583+0000","last_became_active":"2024-05-11T13:28:35.213800+0000","last_became_peered":"2024-05-11T13:28:35.213800+0000","last_unstale":"2024-05-11T13:29:40.543583+0000","last_undegraded":"2024-05-11T13:29:40.543583+0000","last_fullsized":"2024-05-11T13:29:40.543583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:12.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536071+0000","last_change":"2024-05-11T13:28:49.697320+0000","last_active":"2024-05-11T13:29:40.536071+0000","last_peered":"2024-05-11T13:29:40.536071+0000","last_clean":"2024-05-11T13:29:40.536071+0000","last_became_active":"2024-05-11T13:28:49.696751+0000","last_became_peered":"2024-05-11T13:28:49.696751+0000","last_unstale":"2024-05-11T13:29:40.536071+0000","last_undegraded":"2024-05-11T13:29:40.536071+0000","last_fullsized":"2024-05-11T13:29:40.536071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:24.617241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:55.754505+0000","last_change":"2024-05-11T13:28:30.423943+0000","last_active":"2024-05-11T13:29:55.754505+0000","last_peered":"2024-05-11T13:29:55.754505+0000","last_clean":"2024-05-11T13:29:55.754505+0000","last_became_active":"2024-05-11T13:28:30.423454+0000","last_became_peered":"2024-05-11T13:28:30.423454+0000","last_unstale":"2024-05-11T13:29:55.754505+0000","last_undegraded":"2024-05-11T13:29:55.754505+0000","last_fullsized":"2024-05-11T13:29:55.754505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:23:10.967537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545683+0000","last_change":"2024-05-11T13:28:35.220479+0000","last_active":"2024-05-11T13:29:40.545683+0000","last_peered":"2024-05-11T13:29:40.545683+0000","last_clean":"2024-05-11T13:29:40.545683+0000","last_became_active":"2024-05-11T13:28:35.214541+0000","last_became_peered":"2024-05-11T13:28:35.214541+0000","last_unstale":"2024-05-11T13:29:40.545683+0000","last_undegraded":"2024-05-11T13:29:40.545683+0000","last_fullsized":"2024-05-11T13:29:40.545683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:07.746301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844335+0000","last_change":"2024-05-11T13:28:49.691867+0000","last_active":"2024-05-11T13:29:40.844335+0000","last_peered":"2024-05-11T13:29:40.844335+0000","last_clean":"2024-05-11T13:29:40.844335+0000","last_became_active":"2024-05-11T13:28:49.690045+0000","last_became_peered":"2024-05-11T13:28:49.690045+0000","last_unstale":"2024-05-11T13:29:40.844335+0000","last_undegraded":"2024-05-11T13:29:40.844335+0000","last_fullsized":"2024-05-11T13:29:40.844335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:16.220334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.776314+0000","last_change":"2024-05-11T13:28:30.436354+0000","last_active":"2024-05-11T13:29:57.776314+0000","last_peered":"2024-05-11T13:29:57.776314+0000","last_clean":"2024-05-11T13:29:57.776314+0000","last_became_active":"2024-05-11T13:28:30.435983+0000","last_became_peered":"2024-05-11T13:28:30.435983+0000","last_unstale":"2024-05-11T13:29:57.776314+0000","last_undegraded":"2024-05-11T13:29:57.776314+0000","last_fullsized":"2024-05-11T13:29:57.776314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:42.442829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.527843+0000","last_change":"2024-05-11T13:28:35.207284+0000","last_active":"2024-05-11T13:29:40.527843+0000","last_peered":"2024-05-11T13:29:40.527843+0000","last_clean":"2024-05-11T13:29:40.527843+0000","last_became_active":"2024-05-11T13:28:35.206945+0000","last_became_peered":"2024-05-11T13:28:35.206945+0000","last_unstale":"2024-05-11T13:29:40.527843+0000","last_undegraded":"2024-05-11T13:29:40.527843+0000","last_fullsized":"2024-05-11T13:29:40.527843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:18.493659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546335+0000","last_change":"2024-05-11T13:28:49.689558+0000","last_active":"2024-05-11T13:29:40.546335+0000","last_peered":"2024-05-11T13:29:40.546335+0000","last_clean":"2024-05-11T13:29:40.546335+0000","last_became_active":"2024-05-11T13:28:49.689156+0000","last_became_peered":"2024-05-11T13:28:49.689156+0000","last_unstale":"2024-05-11T13:29:40.546335+0000","last_undegraded":"2024-05-11T13:29:40.546335+0000","last_fullsized":"2024-05-11T13:29:40.546335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:21.530111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:29.884950+0000","last_change":"2024-05-11T13:28:30.434534+0000","last_active":"2024-05-11T13:30:29.884950+0000","last_peered":"2024-05-11T13:30:29.884950+0000","last_clean":"2024-05-11T13:30:29.884950+0000","last_became_active":"2024-05-11T13:28:30.433983+0000","last_became_peered":"2024-05-11T13:28:30.433983+0000","last_unstale":"2024-05-11T13:30:29.884950+0000","last_undegraded":"2024-05-11T13:30:29.884950+0000","last_fullsized":"2024-05-11T13:30:29.884950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:16.056662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.547958+0000","last_change":"2024-05-11T13:28:35.205021+0000","last_active":"2024-05-11T13:29:40.547958+0000","last_peered":"2024-05-11T13:29:40.547958+0000","last_clean":"2024-05-11T13:29:40.547958+0000","last_became_active":"2024-05-11T13:28:35.204243+0000","last_became_peered":"2024-05-11T13:28:35.204243+0000","last_unstale":"2024-05-11T13:29:40.547958+0000","last_undegraded":"2024-05-11T13:29:40.547958+0000","last_fullsized":"2024-05-11T13:29:40.547958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:12.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.527578+0000","last_change":"2024-05-11T13:28:49.677392+0000","last_active":"2024-05-11T13:29:40.527578+0000","last_peered":"2024-05-11T13:29:40.527578+0000","last_clean":"2024-05-11T13:29:40.527578+0000","last_became_active":"2024-05-11T13:28:49.674008+0000","last_became_peered":"2024-05-11T13:28:49.674008+0000","last_unstale":"2024-05-11T13:29:40.527578+0000","last_undegraded":"2024-05-11T13:29:40.527578+0000","last_fullsized":"2024-05-11T13:29:40.527578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:47:14.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778632+0000","last_change":"2024-05-11T13:28:30.429500+0000","last_active":"2024-05-11T13:29:57.778632+0000","last_peered":"2024-05-11T13:29:57.778632+0000","last_clean":"2024-05-11T13:29:57.778632+0000","last_became_active":"2024-05-11T13:28:30.429049+0000","last_became_peered":"2024-05-11T13:28:30.429049+0000","last_unstale":"2024-05-11T13:29:57.778632+0000","last_undegraded":"2024-05-11T13:29:57.778632+0000","last_fullsized":"2024-05-11T13:29:57.778632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:00.533603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844631+0000","last_change":"2024-05-11T13:28:35.223764+0000","last_active":"2024-05-11T13:29:40.844631+0000","last_peered":"2024-05-11T13:29:40.844631+0000","last_clean":"2024-05-11T13:29:40.844631+0000","last_became_active":"2024-05-11T13:28:35.223016+0000","last_became_peered":"2024-05-11T13:28:35.223016+0000","last_unstale":"2024-05-11T13:29:40.844631+0000","last_undegraded":"2024-05-11T13:29:40.844631+0000","last_fullsized":"2024-05-11T13:29:40.844631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:42.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546160+0000","last_change":"2024-05-11T13:28:49.683733+0000","last_active":"2024-05-11T13:29:40.546160+0000","last_peered":"2024-05-11T13:29:40.546160+0000","last_clean":"2024-05-11T13:29:40.546160+0000","last_became_active":"2024-05-11T13:28:49.681245+0000","last_became_peered":"2024-05-11T13:28:49.681245+0000","last_unstale":"2024-05-11T13:29:40.546160+0000","last_undegraded":"2024-05-11T13:29:40.546160+0000","last_fullsized":"2024-05-11T13:29:40.546160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:03.386943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.763658+0000","last_change":"2024-05-11T13:28:30.707394+0000","last_active":"2024-05-11T13:29:56.763658+0000","last_peered":"2024-05-11T13:29:56.763658+0000","last_clean":"2024-05-11T13:29:56.763658+0000","last_became_active":"2024-05-11T13:28:30.706114+0000","last_became_peered":"2024-05-11T13:28:30.706114+0000","last_unstale":"2024-05-11T13:29:56.763658+0000","last_undegraded":"2024-05-11T13:29:56.763658+0000","last_fullsized":"2024-05-11T13:29:56.763658+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:53:04.599800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549373+0000","last_change":"2024-05-11T13:28:35.219836+0000","last_active":"2024-05-11T13:29:40.549373+0000","last_peered":"2024-05-11T13:29:40.549373+0000","last_clean":"2024-05-11T13:29:40.549373+0000","last_became_active":"2024-05-11T13:28:35.214474+0000","last_became_peered":"2024-05-11T13:28:35.214474+0000","last_unstale":"2024-05-11T13:29:40.549373+0000","last_undegraded":"2024-05-11T13:29:40.549373+0000","last_fullsized":"2024-05-11T13:29:40.549373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:02.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.528108+0000","last_change":"2024-05-11T13:28:49.694677+0000","last_active":"2024-05-11T13:29:40.528108+0000","last_peered":"2024-05-11T13:29:40.528108+0000","last_clean":"2024-05-11T13:29:40.528108+0000","last_became_active":"2024-05-11T13:28:49.694176+0000","last_became_peered":"2024-05-11T13:28:49.694176+0000","last_unstale":"2024-05-11T13:29:40.528108+0000","last_undegraded":"2024-05-11T13:29:40.528108+0000","last_fullsized":"2024-05-11T13:29:40.528108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:35.461937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":149,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:29.907281+0000","last_change":"2024-05-11T13:28:30.433202+0000","last_active":"2024-05-11T13:30:29.907281+0000","last_peered":"2024-05-11T13:30:29.907281+0000","last_clean":"2024-05-11T13:30:29.907281+0000","last_became_active":"2024-05-11T13:28:30.429269+0000","last_became_peered":"2024-05-11T13:28:30.429269+0000","last_unstale":"2024-05-11T13:30:29.907281+0000","last_undegraded":"2024-05-11T13:30:29.907281+0000","last_fullsized":"2024-05-11T13:30:29.907281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:27:15.620717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62657,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":98,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.551139+0000","last_change":"2024-05-11T13:28:35.210882+0000","last_active":"2024-05-11T13:29:40.551139+0000","last_peered":"2024-05-11T13:29:40.551139+0000","last_clean":"2024-05-11T13:29:40.551139+0000","last_became_active":"2024-05-11T13:28:35.210506+0000","last_became_peered":"2024-05-11T13:28:35.210506+0000","last_unstale":"2024-05-11T13:29:40.551139+0000","last_undegraded":"2024-05-11T13:29:40.551139+0000","last_fullsized":"2024-05-11T13:29:40.551139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:45.752666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543867+0000","last_change":"2024-05-11T13:28:49.693254+0000","last_active":"2024-05-11T13:29:40.543867+0000","last_peered":"2024-05-11T13:29:40.543867+0000","last_clean":"2024-05-11T13:29:40.543867+0000","last_became_active":"2024-05-11T13:28:49.690307+0000","last_became_peered":"2024-05-11T13:28:49.690307+0000","last_unstale":"2024-05-11T13:29:40.543867+0000","last_undegraded":"2024-05-11T13:29:40.543867+0000","last_fullsized":"2024-05-11T13:29:40.543867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:41:31.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:58.785687+0000","last_change":"2024-05-11T13:28:30.429421+0000","last_active":"2024-05-11T13:29:58.785687+0000","last_peered":"2024-05-11T13:29:58.785687+0000","last_clean":"2024-05-11T13:29:58.785687+0000","last_became_active":"2024-05-11T13:28:30.429032+0000","last_became_peered":"2024-05-11T13:28:30.429032+0000","last_unstale":"2024-05-11T13:29:58.785687+0000","last_undegraded":"2024-05-11T13:29:58.785687+0000","last_fullsized":"2024-05-11T13:29:58.785687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:06.234650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535909+0000","last_change":"2024-05-11T13:28:35.207709+0000","last_active":"2024-05-11T13:29:40.535909+0000","last_peered":"2024-05-11T13:29:40.535909+0000","last_clean":"2024-05-11T13:29:40.535909+0000","last_became_active":"2024-05-11T13:28:35.207222+0000","last_became_peered":"2024-05-11T13:28:35.207222+0000","last_unstale":"2024-05-11T13:29:40.535909+0000","last_undegraded":"2024-05-11T13:29:40.535909+0000","last_fullsized":"2024-05-11T13:29:40.535909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:20.023565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535394+0000","last_change":"2024-05-11T13:28:49.692046+0000","last_active":"2024-05-11T13:29:40.535394+0000","last_peered":"2024-05-11T13:29:40.535394+0000","last_clean":"2024-05-11T13:29:40.535394+0000","last_became_active":"2024-05-11T13:28:49.691618+0000","last_became_peered":"2024-05-11T13:28:49.691618+0000","last_unstale":"2024-05-11T13:29:40.535394+0000","last_undegraded":"2024-05-11T13:29:40.535394+0000","last_fullsized":"2024-05-11T13:29:40.535394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:05.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.773794+0000","last_change":"2024-05-11T13:28:30.435459+0000","last_active":"2024-05-11T13:29:57.773794+0000","last_peered":"2024-05-11T13:29:57.773794+0000","last_clean":"2024-05-11T13:29:57.773794+0000","last_became_active":"2024-05-11T13:28:30.434997+0000","last_became_peered":"2024-05-11T13:28:30.434997+0000","last_unstale":"2024-05-11T13:29:57.773794+0000","last_undegraded":"2024-05-11T13:29:57.773794+0000","last_fullsized":"2024-05-11T13:29:57.773794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:36.280546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531321+0000","last_change":"2024-05-11T13:28:35.211214+0000","last_active":"2024-05-11T13:29:40.531321+0000","last_peered":"2024-05-11T13:29:40.531321+0000","last_clean":"2024-05-11T13:29:40.531321+0000","last_became_active":"2024-05-11T13:28:35.210777+0000","last_became_peered":"2024-05-11T13:28:35.210777+0000","last_unstale":"2024-05-11T13:29:40.531321+0000","last_undegraded":"2024-05-11T13:29:40.531321+0000","last_fullsized":"2024-05-11T13:29:40.531321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:53.238549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.529049+0000","last_change":"2024-05-11T13:28:49.682336+0000","last_active":"2024-05-11T13:29:40.529049+0000","last_peered":"2024-05-11T13:29:40.529049+0000","last_clean":"2024-05-11T13:29:40.529049+0000","last_became_active":"2024-05-11T13:28:49.679081+0000","last_became_peered":"2024-05-11T13:28:49.679081+0000","last_unstale":"2024-05-11T13:29:40.529049+0000","last_undegraded":"2024-05-11T13:29:40.529049+0000","last_fullsized":"2024-05-11T13:29:40.529049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:00:21.149262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:27.896221+0000","last_change":"2024-05-11T13:28:30.432165+0000","last_active":"2024-05-11T13:30:27.896221+0000","last_peered":"2024-05-11T13:30:27.896221+0000","last_clean":"2024-05-11T13:30:27.896221+0000","last_became_active":"2024-05-11T13:28:30.431181+0000","last_became_peered":"2024-05-11T13:28:30.431181+0000","last_unstale":"2024-05-11T13:30:27.896221+0000","last_undegraded":"2024-05-11T13:30:27.896221+0000","last_fullsized":"2024-05-11T13:30:27.896221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:26:50.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530968+0000","last_change":"2024-05-11T13:28:35.205662+0000","last_active":"2024-05-11T13:29:40.530968+0000","last_peered":"2024-05-11T13:29:40.530968+0000","last_clean":"2024-05-11T13:29:40.530968+0000","last_became_active":"2024-05-11T13:28:35.205276+0000","last_became_peered":"2024-05-11T13:28:35.205276+0000","last_unstale":"2024-05-11T13:29:40.530968+0000","last_undegraded":"2024-05-11T13:29:40.530968+0000","last_fullsized":"2024-05-11T13:29:40.530968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:36.308402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537248+0000","last_change":"2024-05-11T13:28:49.689705+0000","last_active":"2024-05-11T13:29:40.537248+0000","last_peered":"2024-05-11T13:29:40.537248+0000","last_clean":"2024-05-11T13:29:40.537248+0000","last_became_active":"2024-05-11T13:28:49.689223+0000","last_became_peered":"2024-05-11T13:28:49.689223+0000","last_unstale":"2024-05-11T13:29:40.537248+0000","last_undegraded":"2024-05-11T13:29:40.537248+0000","last_fullsized":"2024-05-11T13:29:40.537248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:37:20.727773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.775555+0000","last_change":"2024-05-11T13:28:30.709014+0000","last_active":"2024-05-11T13:29:57.775555+0000","last_peered":"2024-05-11T13:29:57.775555+0000","last_clean":"2024-05-11T13:29:57.775555+0000","last_became_active":"2024-05-11T13:28:30.708000+0000","last_became_peered":"2024-05-11T13:28:30.708000+0000","last_unstale":"2024-05-11T13:29:57.775555+0000","last_undegraded":"2024-05-11T13:29:57.775555+0000","last_fullsized":"2024-05-11T13:29:57.775555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:00.813415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.544988+0000","last_change":"2024-05-11T13:28:35.215149+0000","last_active":"2024-05-11T13:29:40.544988+0000","last_peered":"2024-05-11T13:29:40.544988+0000","last_clean":"2024-05-11T13:29:40.544988+0000","last_became_active":"2024-05-11T13:28:35.214792+0000","last_became_peered":"2024-05-11T13:28:35.214792+0000","last_unstale":"2024-05-11T13:29:40.544988+0000","last_undegraded":"2024-05-11T13:29:40.544988+0000","last_fullsized":"2024-05-11T13:29:40.544988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.544787+0000","last_change":"2024-05-11T13:28:49.686347+0000","last_active":"2024-05-11T13:29:40.544787+0000","last_peered":"2024-05-11T13:29:40.544787+0000","last_clean":"2024-05-11T13:29:40.544787+0000","last_became_active":"2024-05-11T13:28:49.685391+0000","last_became_peered":"2024-05-11T13:28:49.685391+0000","last_unstale":"2024-05-11T13:29:40.544787+0000","last_undegraded":"2024-05-11T13:29:40.544787+0000","last_fullsized":"2024-05-11T13:29:40.544787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:24:50.347627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.741381+0000","last_change":"2024-05-11T13:29:40.560815+0000","last_active":"2024-05-11T13:29:56.741381+0000","last_peered":"2024-05-11T13:29:56.741381+0000","last_clean":"2024-05-11T13:29:56.741381+0000","last_became_active":"2024-05-11T13:29:40.553031+0000","last_became_peered":"2024-05-11T13:29:40.553031+0000","last_unstale":"2024-05-11T13:29:56.741381+0000","last_undegraded":"2024-05-11T13:29:56.741381+0000","last_fullsized":"2024-05-11T13:29:56.741381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:27.437654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533924+0000","last_change":"2024-05-11T13:28:35.218377+0000","last_active":"2024-05-11T13:29:40.533924+0000","last_peered":"2024-05-11T13:29:40.533924+0000","last_clean":"2024-05-11T13:29:40.533924+0000","last_became_active":"2024-05-11T13:28:35.217922+0000","last_became_peered":"2024-05-11T13:28:35.217922+0000","last_unstale":"2024-05-11T13:29:40.533924+0000","last_undegraded":"2024-05-11T13:29:40.533924+0000","last_fullsized":"2024-05-11T13:29:40.533924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:58.383381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549510+0000","last_change":"2024-05-11T13:28:49.723400+0000","last_active":"2024-05-11T13:29:40.549510+0000","last_peered":"2024-05-11T13:29:40.549510+0000","last_clean":"2024-05-11T13:29:40.549510+0000","last_became_active":"2024-05-11T13:28:49.722675+0000","last_became_peered":"2024-05-11T13:28:49.722675+0000","last_unstale":"2024-05-11T13:29:40.549510+0000","last_undegraded":"2024-05-11T13:29:40.549510+0000","last_fullsized":"2024-05-11T13:29:40.549510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:14:44.810541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.528321+0000","last_change":"2024-05-11T13:28:30.710667+0000","last_active":"2024-05-11T13:29:40.528321+0000","last_peered":"2024-05-11T13:29:40.528321+0000","last_clean":"2024-05-11T13:29:40.528321+0000","last_became_active":"2024-05-11T13:28:30.709933+0000","last_became_peered":"2024-05-11T13:28:30.709933+0000","last_unstale":"2024-05-11T13:29:40.528321+0000","last_undegraded":"2024-05-11T13:29:40.528321+0000","last_fullsized":"2024-05-11T13:29:40.528321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:34.942946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.539110+0000","last_change":"2024-05-11T13:28:35.215691+0000","last_active":"2024-05-11T13:29:40.539110+0000","last_peered":"2024-05-11T13:29:40.539110+0000","last_clean":"2024-05-11T13:29:40.539110+0000","last_became_active":"2024-05-11T13:28:35.209873+0000","last_became_peered":"2024-05-11T13:28:35.209873+0000","last_unstale":"2024-05-11T13:29:40.539110+0000","last_undegraded":"2024-05-11T13:29:40.539110+0000","last_fullsized":"2024-05-11T13:29:40.539110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:42:12.935570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536722+0000","last_change":"2024-05-11T13:28:49.694322+0000","last_active":"2024-05-11T13:29:40.536722+0000","last_peered":"2024-05-11T13:29:40.536722+0000","last_clean":"2024-05-11T13:29:40.536722+0000","last_became_active":"2024-05-11T13:28:49.690900+0000","last_became_peered":"2024-05-11T13:28:49.690900+0000","last_unstale":"2024-05-11T13:29:40.536722+0000","last_undegraded":"2024-05-11T13:29:40.536722+0000","last_fullsized":"2024-05-11T13:29:40.536722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:00:07.433371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:31.889892+0000","last_change":"2024-05-11T13:28:30.432529+0000","last_active":"2024-05-11T13:30:31.889892+0000","last_peered":"2024-05-11T13:30:31.889892+0000","last_clean":"2024-05-11T13:30:31.889892+0000","last_became_active":"2024-05-11T13:28:30.428077+0000","last_became_peered":"2024-05-11T13:28:30.428077+0000","last_unstale":"2024-05-11T13:30:31.889892+0000","last_undegraded":"2024-05-11T13:30:31.889892+0000","last_fullsized":"2024-05-11T13:30:31.889892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:42.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.538376+0000","last_change":"2024-05-11T13:28:35.219160+0000","last_active":"2024-05-11T13:29:40.538376+0000","last_peered":"2024-05-11T13:29:40.538376+0000","last_clean":"2024-05-11T13:29:40.538376+0000","last_became_active":"2024-05-11T13:28:35.216810+0000","last_became_peered":"2024-05-11T13:28:35.216810+0000","last_unstale":"2024-05-11T13:29:40.538376+0000","last_undegraded":"2024-05-11T13:29:40.538376+0000","last_fullsized":"2024-05-11T13:29:40.538376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:35:54.465030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536984+0000","last_change":"2024-05-11T13:28:49.725190+0000","last_active":"2024-05-11T13:29:40.536984+0000","last_peered":"2024-05-11T13:29:40.536984+0000","last_clean":"2024-05-11T13:29:40.536984+0000","last_became_active":"2024-05-11T13:28:49.724193+0000","last_became_peered":"2024-05-11T13:28:49.724193+0000","last_unstale":"2024-05-11T13:29:40.536984+0000","last_undegraded":"2024-05-11T13:29:40.536984+0000","last_fullsized":"2024-05-11T13:29:40.536984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:47:41.181606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543059+0000","last_change":"2024-05-11T13:28:35.211262+0000","last_active":"2024-05-11T13:29:40.543059+0000","last_peered":"2024-05-11T13:29:40.543059+0000","last_clean":"2024-05-11T13:29:40.543059+0000","last_became_active":"2024-05-11T13:28:35.210859+0000","last_became_peered":"2024-05-11T13:28:35.210859+0000","last_unstale":"2024-05-11T13:29:40.543059+0000","last_undegraded":"2024-05-11T13:29:40.543059+0000","last_fullsized":"2024-05-11T13:29:40.543059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:40:33.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":80,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:30.903889+0000","last_change":"2024-05-11T13:28:30.707326+0000","last_active":"2024-05-11T13:30:30.903889+0000","last_peered":"2024-05-11T13:30:30.903889+0000","last_clean":"2024-05-11T13:30:30.903889+0000","last_became_active":"2024-05-11T13:28:30.705496+0000","last_became_peered":"2024-05-11T13:28:30.705496+0000","last_unstale":"2024-05-11T13:30:30.903889+0000","last_undegraded":"2024-05-11T13:30:30.903889+0000","last_fullsized":"2024-05-11T13:30:30.903889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:48.117551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531497+0000","last_change":"2024-05-11T13:28:49.687260+0000","last_active":"2024-05-11T13:29:40.531497+0000","last_peered":"2024-05-11T13:29:40.531497+0000","last_clean":"2024-05-11T13:29:40.531497+0000","last_became_active":"2024-05-11T13:28:49.686593+0000","last_became_peered":"2024-05-11T13:28:49.686593+0000","last_unstale":"2024-05-11T13:29:40.531497+0000","last_undegraded":"2024-05-11T13:29:40.531497+0000","last_fullsized":"2024-05-11T13:29:40.531497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:48.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546734+0000","last_change":"2024-05-11T13:28:35.216643+0000","last_active":"2024-05-11T13:29:40.546734+0000","last_peered":"2024-05-11T13:29:40.546734+0000","last_clean":"2024-05-11T13:29:40.546734+0000","last_became_active":"2024-05-11T13:28:35.216300+0000","last_became_peered":"2024-05-11T13:28:35.216300+0000","last_unstale":"2024-05-11T13:29:40.546734+0000","last_undegraded":"2024-05-11T13:29:40.546734+0000","last_fullsized":"2024-05-11T13:29:40.546734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:20:01.351678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:58.789699+0000","last_change":"2024-05-11T13:28:30.427465+0000","last_active":"2024-05-11T13:29:58.789699+0000","last_peered":"2024-05-11T13:29:58.789699+0000","last_clean":"2024-05-11T13:29:58.789699+0000","last_became_active":"2024-05-11T13:28:30.426968+0000","last_became_peered":"2024-05-11T13:28:30.426968+0000","last_unstale":"2024-05-11T13:29:58.789699+0000","last_undegraded":"2024-05-11T13:29:58.789699+0000","last_fullsized":"2024-05-11T13:29:58.789699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:02:44.969069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530098+0000","last_change":"2024-05-11T13:26:37.738725+0000","last_active":"2024-05-11T13:29:40.530098+0000","last_peered":"2024-05-11T13:29:40.530098+0000","last_clean":"2024-05-11T13:29:40.530098+0000","last_became_active":"2024-05-11T13:26:37.724302+0000","last_became_peered":"2024-05-11T13:26:37.724302+0000","last_unstale":"2024-05-11T13:29:40.530098+0000","last_undegraded":"2024-05-11T13:29:40.530098+0000","last_fullsized":"2024-05-11T13:29:40.530098+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_clean_scrub_stamp":"2024-05-11T13:21:55.918148+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:32:56.277780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543703+0000","last_change":"2024-05-11T13:28:49.723829+0000","last_active":"2024-05-11T13:29:40.543703+0000","last_peered":"2024-05-11T13:29:40.543703+0000","last_clean":"2024-05-11T13:29:40.543703+0000","last_became_active":"2024-05-11T13:28:49.723190+0000","last_became_peered":"2024-05-11T13:28:49.723190+0000","last_unstale":"2024-05-11T13:29:40.543703+0000","last_undegraded":"2024-05-11T13:29:40.543703+0000","last_fullsized":"2024-05-11T13:29:40.543703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.742374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535793+0000","last_change":"2024-05-11T13:28:35.213486+0000","last_active":"2024-05-11T13:29:40.535793+0000","last_peered":"2024-05-11T13:29:40.535793+0000","last_clean":"2024-05-11T13:29:40.535793+0000","last_became_active":"2024-05-11T13:28:35.211962+0000","last_became_peered":"2024-05-11T13:28:35.211962+0000","last_unstale":"2024-05-11T13:29:40.535793+0000","last_undegraded":"2024-05-11T13:29:40.535793+0000","last_fullsized":"2024-05-11T13:29:40.535793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:28:34.046898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.774165+0000","last_change":"2024-05-11T13:28:30.431023+0000","last_active":"2024-05-11T13:29:57.774165+0000","last_peered":"2024-05-11T13:29:57.774165+0000","last_clean":"2024-05-11T13:29:57.774165+0000","last_became_active":"2024-05-11T13:28:30.430609+0000","last_became_peered":"2024-05-11T13:28:30.430609+0000","last_unstale":"2024-05-11T13:29:57.774165+0000","last_undegraded":"2024-05-11T13:29:57.774165+0000","last_fullsized":"2024-05-11T13:29:57.774165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:50.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533480+0000","last_change":"2024-05-11T13:28:49.695550+0000","last_active":"2024-05-11T13:29:40.533480+0000","last_peered":"2024-05-11T13:29:40.533480+0000","last_clean":"2024-05-11T13:29:40.533480+0000","last_became_active":"2024-05-11T13:28:49.694350+0000","last_became_peered":"2024-05-11T13:28:49.694350+0000","last_unstale":"2024-05-11T13:29:40.533480+0000","last_undegraded":"2024-05-11T13:29:40.533480+0000","last_fullsized":"2024-05-11T13:29:40.533480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:49.617673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.775618+0000","last_change":"2024-05-11T13:28:30.710826+0000","last_active":"2024-05-11T13:29:57.775618+0000","last_peered":"2024-05-11T13:29:57.775618+0000","last_clean":"2024-05-11T13:29:57.775618+0000","last_became_active":"2024-05-11T13:28:30.709854+0000","last_became_peered":"2024-05-11T13:28:30.709854+0000","last_unstale":"2024-05-11T13:29:57.775618+0000","last_undegraded":"2024-05-11T13:29:57.775618+0000","last_fullsized":"2024-05-11T13:29:57.775618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:35:16.384559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537252+0000","last_change":"2024-05-11T13:28:35.205521+0000","last_active":"2024-05-11T13:29:40.537252+0000","last_peered":"2024-05-11T13:29:40.537252+0000","last_clean":"2024-05-11T13:29:40.537252+0000","last_became_active":"2024-05-11T13:28:35.205167+0000","last_became_peered":"2024-05-11T13:28:35.205167+0000","last_unstale":"2024-05-11T13:29:40.537252+0000","last_undegraded":"2024-05-11T13:29:40.537252+0000","last_fullsized":"2024-05-11T13:29:40.537252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:10:27.473991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545565+0000","last_change":"2024-05-11T13:28:49.723652+0000","last_active":"2024-05-11T13:29:40.545565+0000","last_peered":"2024-05-11T13:29:40.545565+0000","last_clean":"2024-05-11T13:29:40.545565+0000","last_became_active":"2024-05-11T13:28:49.723135+0000","last_became_peered":"2024-05-11T13:28:49.723135+0000","last_unstale":"2024-05-11T13:29:40.545565+0000","last_undegraded":"2024-05-11T13:29:40.545565+0000","last_fullsized":"2024-05-11T13:29:40.545565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:21:18.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:25.254699+0000","last_change":"2024-05-11T13:28:30.428197+0000","last_active":"2024-05-11T13:30:25.254699+0000","last_peered":"2024-05-11T13:30:25.254699+0000","last_clean":"2024-05-11T13:30:25.254699+0000","last_became_active":"2024-05-11T13:28:30.427753+0000","last_became_peered":"2024-05-11T13:28:30.427753+0000","last_unstale":"2024-05-11T13:30:25.254699+0000","last_undegraded":"2024-05-11T13:30:25.254699+0000","last_fullsized":"2024-05-11T13:30:25.254699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:52:54.165993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545034+0000","last_change":"2024-05-11T13:29:40.541861+0000","last_active":"2024-05-11T13:29:40.545034+0000","last_peered":"2024-05-11T13:29:40.545034+0000","last_clean":"2024-05-11T13:29:40.545034+0000","last_became_active":"2024-05-11T13:29:40.541395+0000","last_became_peered":"2024-05-11T13:29:40.541395+0000","last_unstale":"2024-05-11T13:29:40.545034+0000","last_undegraded":"2024-05-11T13:29:40.545034+0000","last_fullsized":"2024-05-11T13:29:40.545034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:50:39.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.532327+0000","last_change":"2024-05-11T13:28:49.677549+0000","last_active":"2024-05-11T13:29:40.532327+0000","last_peered":"2024-05-11T13:29:40.532327+0000","last_clean":"2024-05-11T13:29:40.532327+0000","last_became_active":"2024-05-11T13:28:49.676625+0000","last_became_peered":"2024-05-11T13:28:49.676625+0000","last_unstale":"2024-05-11T13:29:40.532327+0000","last_undegraded":"2024-05-11T13:29:40.532327+0000","last_fullsized":"2024-05-11T13:29:40.532327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:44.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778619+0000","last_change":"2024-05-11T13:28:30.440945+0000","last_active":"2024-05-11T13:29:57.778619+0000","last_peered":"2024-05-11T13:29:57.778619+0000","last_clean":"2024-05-11T13:29:57.778619+0000","last_became_active":"2024-05-11T13:28:30.440595+0000","last_became_peered":"2024-05-11T13:28:30.440595+0000","last_unstale":"2024-05-11T13:29:57.778619+0000","last_undegraded":"2024-05-11T13:29:57.778619+0000","last_fullsized":"2024-05-11T13:29:57.778619+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:51:10.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844988+0000","last_change":"2024-05-11T13:28:35.214207+0000","last_active":"2024-05-11T13:29:40.844988+0000","last_peered":"2024-05-11T13:29:40.844988+0000","last_clean":"2024-05-11T13:29:40.844988+0000","last_became_active":"2024-05-11T13:28:35.213844+0000","last_became_peered":"2024-05-11T13:28:35.213844+0000","last_unstale":"2024-05-11T13:29:40.844988+0000","last_undegraded":"2024-05-11T13:29:40.844988+0000","last_fullsized":"2024-05-11T13:29:40.844988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:53.063329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550204+0000","last_change":"2024-05-11T13:28:49.725505+0000","last_active":"2024-05-11T13:29:40.550204+0000","last_peered":"2024-05-11T13:29:40.550204+0000","last_clean":"2024-05-11T13:29:40.550204+0000","last_became_active":"2024-05-11T13:28:49.723613+0000","last_became_peered":"2024-05-11T13:28:49.723613+0000","last_unstale":"2024-05-11T13:29:40.550204+0000","last_undegraded":"2024-05-11T13:29:40.550204+0000","last_fullsized":"2024-05-11T13:29:40.550204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:54:30.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:54.725235+0000","last_change":"2024-05-11T13:28:30.707025+0000","last_active":"2024-05-11T13:29:54.725235+0000","last_peered":"2024-05-11T13:29:54.725235+0000","last_clean":"2024-05-11T13:29:54.725235+0000","last_became_active":"2024-05-11T13:28:30.705853+0000","last_became_peered":"2024-05-11T13:28:30.705853+0000","last_unstale":"2024-05-11T13:29:54.725235+0000","last_undegraded":"2024-05-11T13:29:54.725235+0000","last_fullsized":"2024-05-11T13:29:54.725235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:15:04.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549398+0000","last_change":"2024-05-11T13:28:35.214682+0000","last_active":"2024-05-11T13:29:40.549398+0000","last_peered":"2024-05-11T13:29:40.549398+0000","last_clean":"2024-05-11T13:29:40.549398+0000","last_became_active":"2024-05-11T13:28:35.211581+0000","last_became_peered":"2024-05-11T13:28:35.211581+0000","last_unstale":"2024-05-11T13:29:40.549398+0000","last_undegraded":"2024-05-11T13:29:40.549398+0000","last_fullsized":"2024-05-11T13:29:40.549398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:19.231675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.528919+0000","last_change":"2024-05-11T13:28:49.689763+0000","last_active":"2024-05-11T13:29:40.528919+0000","last_peered":"2024-05-11T13:29:40.528919+0000","last_clean":"2024-05-11T13:29:40.528919+0000","last_became_active":"2024-05-11T13:28:49.689362+0000","last_became_peered":"2024-05-11T13:28:49.689362+0000","last_unstale":"2024-05-11T13:29:40.528919+0000","last_undegraded":"2024-05-11T13:29:40.528919+0000","last_fullsized":"2024-05-11T13:29:40.528919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:07:59.656953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.774103+0000","last_change":"2024-05-11T13:28:30.419444+0000","last_active":"2024-05-11T13:29:57.774103+0000","last_peered":"2024-05-11T13:29:57.774103+0000","last_clean":"2024-05-11T13:29:57.774103+0000","last_became_active":"2024-05-11T13:28:30.419064+0000","last_became_peered":"2024-05-11T13:28:30.419064+0000","last_unstale":"2024-05-11T13:29:57.774103+0000","last_undegraded":"2024-05-11T13:29:57.774103+0000","last_fullsized":"2024-05-11T13:29:57.774103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:02.567403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531572+0000","last_change":"2024-05-11T13:28:35.209347+0000","last_active":"2024-05-11T13:29:40.531572+0000","last_peered":"2024-05-11T13:29:40.531572+0000","last_clean":"2024-05-11T13:29:40.531572+0000","last_became_active":"2024-05-11T13:28:35.208912+0000","last_became_peered":"2024-05-11T13:28:35.208912+0000","last_unstale":"2024-05-11T13:29:40.531572+0000","last_undegraded":"2024-05-11T13:29:40.531572+0000","last_fullsized":"2024-05-11T13:29:40.531572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:08:30.832941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549961+0000","last_change":"2024-05-11T13:28:49.677398+0000","last_active":"2024-05-11T13:29:40.549961+0000","last_peered":"2024-05-11T13:29:40.549961+0000","last_clean":"2024-05-11T13:29:40.549961+0000","last_became_active":"2024-05-11T13:28:49.676725+0000","last_became_peered":"2024-05-11T13:28:49.676725+0000","last_unstale":"2024-05-11T13:29:40.549961+0000","last_undegraded":"2024-05-11T13:29:40.549961+0000","last_fullsized":"2024-05-11T13:29:40.549961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:32.476049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.762062+0000","last_change":"2024-05-11T13:28:30.428782+0000","last_active":"2024-05-11T13:29:56.762062+0000","last_peered":"2024-05-11T13:29:56.762062+0000","last_clean":"2024-05-11T13:29:56.762062+0000","last_became_active":"2024-05-11T13:28:30.427819+0000","last_became_peered":"2024-05-11T13:28:30.427819+0000","last_unstale":"2024-05-11T13:29:56.762062+0000","last_undegraded":"2024-05-11T13:29:56.762062+0000","last_fullsized":"2024-05-11T13:29:56.762062+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:23:24.533765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.547896+0000","last_change":"2024-05-11T13:28:35.210095+0000","last_active":"2024-05-11T13:29:40.547896+0000","last_peered":"2024-05-11T13:29:40.547896+0000","last_clean":"2024-05-11T13:29:40.547896+0000","last_became_active":"2024-05-11T13:28:35.208831+0000","last_became_peered":"2024-05-11T13:28:35.208831+0000","last_unstale":"2024-05-11T13:29:40.547896+0000","last_undegraded":"2024-05-11T13:29:40.547896+0000","last_fullsized":"2024-05-11T13:29:40.547896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:06.957821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548998+0000","last_change":"2024-05-11T13:28:49.724791+0000","last_active":"2024-05-11T13:29:40.548998+0000","last_peered":"2024-05-11T13:29:40.548998+0000","last_clean":"2024-05-11T13:29:40.548998+0000","last_became_active":"2024-05-11T13:28:49.723855+0000","last_became_peered":"2024-05-11T13:28:49.723855+0000","last_unstale":"2024-05-11T13:29:40.548998+0000","last_undegraded":"2024-05-11T13:29:40.548998+0000","last_fullsized":"2024-05-11T13:29:40.548998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:34.705331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:54.728986+0000","last_change":"2024-05-11T13:28:30.431206+0000","last_active":"2024-05-11T13:29:54.728986+0000","last_peered":"2024-05-11T13:29:54.728986+0000","last_clean":"2024-05-11T13:29:54.728986+0000","last_became_active":"2024-05-11T13:28:30.430714+0000","last_became_peered":"2024-05-11T13:28:30.430714+0000","last_unstale":"2024-05-11T13:29:54.728986+0000","last_undegraded":"2024-05-11T13:29:54.728986+0000","last_fullsized":"2024-05-11T13:29:54.728986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:29:21.070046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530860+0000","last_change":"2024-05-11T13:28:35.222254+0000","last_active":"2024-05-11T13:29:40.530860+0000","last_peered":"2024-05-11T13:29:40.530860+0000","last_clean":"2024-05-11T13:29:40.530860+0000","last_became_active":"2024-05-11T13:28:35.219539+0000","last_became_peered":"2024-05-11T13:28:35.219539+0000","last_unstale":"2024-05-11T13:29:40.530860+0000","last_undegraded":"2024-05-11T13:29:40.530860+0000","last_fullsized":"2024-05-11T13:29:40.530860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:55.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.552124+0000","last_change":"2024-05-11T13:28:49.692168+0000","last_active":"2024-05-11T13:29:40.552124+0000","last_peered":"2024-05-11T13:29:40.552124+0000","last_clean":"2024-05-11T13:29:40.552124+0000","last_became_active":"2024-05-11T13:28:49.691730+0000","last_became_peered":"2024-05-11T13:28:49.691730+0000","last_unstale":"2024-05-11T13:29:40.552124+0000","last_undegraded":"2024-05-11T13:29:40.552124+0000","last_fullsized":"2024-05-11T13:29:40.552124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:02.935554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":77,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:27.880025+0000","last_change":"2024-05-11T13:28:30.712115+0000","last_active":"2024-05-11T13:30:27.880025+0000","last_peered":"2024-05-11T13:30:27.880025+0000","last_clean":"2024-05-11T13:30:27.880025+0000","last_became_active":"2024-05-11T13:28:30.710613+0000","last_became_peered":"2024-05-11T13:28:30.710613+0000","last_unstale":"2024-05-11T13:30:27.880025+0000","last_undegraded":"2024-05-11T13:30:27.880025+0000","last_fullsized":"2024-05-11T13:30:27.880025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:00.521915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531999+0000","last_change":"2024-05-11T13:28:35.221340+0000","last_active":"2024-05-11T13:29:40.531999+0000","last_peered":"2024-05-11T13:29:40.531999+0000","last_clean":"2024-05-11T13:29:40.531999+0000","last_became_active":"2024-05-11T13:28:35.220702+0000","last_became_peered":"2024-05-11T13:28:35.220702+0000","last_unstale":"2024-05-11T13:29:40.531999+0000","last_undegraded":"2024-05-11T13:29:40.531999+0000","last_fullsized":"2024-05-11T13:29:40.531999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:59:37.313209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843686+0000","last_change":"2024-05-11T13:28:49.682917+0000","last_active":"2024-05-11T13:29:40.843686+0000","last_peered":"2024-05-11T13:29:40.843686+0000","last_clean":"2024-05-11T13:29:40.843686+0000","last_became_active":"2024-05-11T13:28:49.681905+0000","last_became_peered":"2024-05-11T13:28:49.681905+0000","last_unstale":"2024-05-11T13:29:40.843686+0000","last_undegraded":"2024-05-11T13:29:40.843686+0000","last_fullsized":"2024-05-11T13:29:40.843686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:09:26.191565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.773253+0000","last_change":"2024-05-11T13:28:30.437356+0000","last_active":"2024-05-11T13:29:57.773253+0000","last_peered":"2024-05-11T13:29:57.773253+0000","last_clean":"2024-05-11T13:29:57.773253+0000","last_became_active":"2024-05-11T13:28:30.437013+0000","last_became_peered":"2024-05-11T13:28:30.437013+0000","last_unstale":"2024-05-11T13:29:57.773253+0000","last_undegraded":"2024-05-11T13:29:57.773253+0000","last_fullsized":"2024-05-11T13:29:57.773253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:58.756156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530954+0000","last_change":"2024-05-11T13:28:35.206105+0000","last_active":"2024-05-11T13:29:40.530954+0000","last_peered":"2024-05-11T13:29:40.530954+0000","last_clean":"2024-05-11T13:29:40.530954+0000","last_became_active":"2024-05-11T13:28:35.205684+0000","last_became_peered":"2024-05-11T13:28:35.205684+0000","last_unstale":"2024-05-11T13:29:40.530954+0000","last_undegraded":"2024-05-11T13:29:40.530954+0000","last_fullsized":"2024-05-11T13:29:40.530954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:12.326574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.541581+0000","last_change":"2024-05-11T13:28:49.689545+0000","last_active":"2024-05-11T13:29:40.541581+0000","last_peered":"2024-05-11T13:29:40.541581+0000","last_clean":"2024-05-11T13:29:40.541581+0000","last_became_active":"2024-05-11T13:28:49.688528+0000","last_became_peered":"2024-05-11T13:28:49.688528+0000","last_unstale":"2024-05-11T13:29:40.541581+0000","last_undegraded":"2024-05-11T13:29:40.541581+0000","last_fullsized":"2024-05-11T13:29:40.541581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:35.418873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778143+0000","last_change":"2024-05-11T13:29:40.560835+0000","last_active":"2024-05-11T13:29:57.778143+0000","last_peered":"2024-05-11T13:29:57.778143+0000","last_clean":"2024-05-11T13:29:57.778143+0000","last_became_active":"2024-05-11T13:29:40.554310+0000","last_became_peered":"2024-05-11T13:29:40.554310+0000","last_unstale":"2024-05-11T13:29:57.778143+0000","last_undegraded":"2024-05-11T13:29:57.778143+0000","last_fullsized":"2024-05-11T13:29:57.778143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:19:26.805261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531123+0000","last_change":"2024-05-11T13:28:35.220001+0000","last_active":"2024-05-11T13:29:40.531123+0000","last_peered":"2024-05-11T13:29:40.531123+0000","last_clean":"2024-05-11T13:29:40.531123+0000","last_became_active":"2024-05-11T13:28:35.216768+0000","last_became_peered":"2024-05-11T13:28:35.216768+0000","last_unstale":"2024-05-11T13:29:40.531123+0000","last_undegraded":"2024-05-11T13:29:40.531123+0000","last_fullsized":"2024-05-11T13:29:40.531123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:54:23.547689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531779+0000","last_change":"2024-05-11T13:28:35.213481+0000","last_active":"2024-05-11T13:29:40.531779+0000","last_peered":"2024-05-11T13:29:40.531779+0000","last_clean":"2024-05-11T13:29:40.531779+0000","last_became_active":"2024-05-11T13:28:35.212956+0000","last_became_peered":"2024-05-11T13:28:35.212956+0000","last_unstale":"2024-05-11T13:29:40.531779+0000","last_undegraded":"2024-05-11T13:29:40.531779+0000","last_fullsized":"2024-05-11T13:29:40.531779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:14.814155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.554654+0000","last_change":"2024-05-11T13:29:40.554654+0000","last_active":"2024-05-11T13:29:40.554654+0000","last_peered":"2024-05-11T13:29:40.554654+0000","last_clean":"2024-05-11T13:29:40.554654+0000","last_became_active":"2024-05-11T13:29:40.551836+0000","last_became_peered":"2024-05-11T13:29:40.551836+0000","last_unstale":"2024-05-11T13:29:40.554654+0000","last_undegraded":"2024-05-11T13:29:40.554654+0000","last_fullsized":"2024-05-11T13:29:40.554654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:49:44.566051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535402+0000","last_change":"2024-05-11T13:28:35.208925+0000","last_active":"2024-05-11T13:29:40.535402+0000","last_peered":"2024-05-11T13:29:40.535402+0000","last_clean":"2024-05-11T13:29:40.535402+0000","last_became_active":"2024-05-11T13:28:35.208055+0000","last_became_peered":"2024-05-11T13:28:35.208055+0000","last_unstale":"2024-05-11T13:29:40.535402+0000","last_undegraded":"2024-05-11T13:29:40.535402+0000","last_fullsized":"2024-05-11T13:29:40.535402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:20.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844476+0000","last_change":"2024-05-11T13:28:35.213226+0000","last_active":"2024-05-11T13:29:40.844476+0000","last_peered":"2024-05-11T13:29:40.844476+0000","last_clean":"2024-05-11T13:29:40.844476+0000","last_became_active":"2024-05-11T13:28:35.212839+0000","last_became_peered":"2024-05-11T13:28:35.212839+0000","last_unstale":"2024-05-11T13:29:40.844476+0000","last_undegraded":"2024-05-11T13:29:40.844476+0000","last_fullsized":"2024-05-11T13:29:40.844476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:37:05.312505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.848092+0000","last_change":"2024-05-11T13:29:40.848092+0000","last_active":"2024-05-11T13:29:40.848092+0000","last_peered":"2024-05-11T13:29:40.848092+0000","last_clean":"2024-05-11T13:29:40.848092+0000","last_became_active":"2024-05-11T13:29:40.846950+0000","last_became_peered":"2024-05-11T13:29:40.846950+0000","last_unstale":"2024-05-11T13:29:40.848092+0000","last_undegraded":"2024-05-11T13:29:40.848092+0000","last_fullsized":"2024-05-11T13:29:40.848092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:18:06.675351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533197+0000","last_change":"2024-05-11T13:28:35.217529+0000","last_active":"2024-05-11T13:29:40.533197+0000","last_peered":"2024-05-11T13:29:40.533197+0000","last_clean":"2024-05-11T13:29:40.533197+0000","last_became_active":"2024-05-11T13:28:35.217062+0000","last_became_peered":"2024-05-11T13:28:35.217062+0000","last_unstale":"2024-05-11T13:29:40.533197+0000","last_undegraded":"2024-05-11T13:29:40.533197+0000","last_fullsized":"2024-05-11T13:29:40.533197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:24:33.955794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.538516+0000","last_change":"2024-05-11T13:28:35.217777+0000","last_active":"2024-05-11T13:29:40.538516+0000","last_peered":"2024-05-11T13:29:40.538516+0000","last_clean":"2024-05-11T13:29:40.538516+0000","last_became_active":"2024-05-11T13:28:35.217381+0000","last_became_peered":"2024-05-11T13:28:35.217381+0000","last_unstale":"2024-05-11T13:29:40.538516+0000","last_undegraded":"2024-05-11T13:29:40.538516+0000","last_fullsized":"2024-05-11T13:29:40.538516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:43.973192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548671+0000","last_change":"2024-05-11T13:28:35.209826+0000","last_active":"2024-05-11T13:29:40.548671+0000","last_peered":"2024-05-11T13:29:40.548671+0000","last_clean":"2024-05-11T13:29:40.548671+0000","last_became_active":"2024-05-11T13:28:35.209461+0000","last_became_peered":"2024-05-11T13:28:35.209461+0000","last_unstale":"2024-05-11T13:29:40.548671+0000","last_undegraded":"2024-05-11T13:29:40.548671+0000","last_fullsized":"2024-05-11T13:29:40.548671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:03.323045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.542844+0000","last_change":"2024-05-11T13:28:35.205252+0000","last_active":"2024-05-11T13:29:40.542844+0000","last_peered":"2024-05-11T13:29:40.542844+0000","last_clean":"2024-05-11T13:29:40.542844+0000","last_became_active":"2024-05-11T13:28:35.204835+0000","last_became_peered":"2024-05-11T13:28:35.204835+0000","last_unstale":"2024-05-11T13:29:40.542844+0000","last_undegraded":"2024-05-11T13:29:40.542844+0000","last_fullsized":"2024-05-11T13:29:40.542844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:10.032439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546583+0000","last_change":"2024-05-11T13:28:35.219718+0000","last_active":"2024-05-11T13:29:40.546583+0000","last_peered":"2024-05-11T13:29:40.546583+0000","last_clean":"2024-05-11T13:29:40.546583+0000","last_became_active":"2024-05-11T13:28:35.219032+0000","last_became_peered":"2024-05-11T13:28:35.219032+0000","last_unstale":"2024-05-11T13:29:40.546583+0000","last_undegraded":"2024-05-11T13:29:40.546583+0000","last_fullsized":"2024-05-11T13:29:40.546583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:03:58.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.552365+0000","last_change":"2024-05-11T13:28:35.217820+0000","last_active":"2024-05-11T13:29:40.552365+0000","last_peered":"2024-05-11T13:29:40.552365+0000","last_clean":"2024-05-11T13:29:40.552365+0000","last_became_active":"2024-05-11T13:28:35.216260+0000","last_became_peered":"2024-05-11T13:28:35.216260+0000","last_unstale":"2024-05-11T13:29:40.552365+0000","last_undegraded":"2024-05-11T13:29:40.552365+0000","last_fullsized":"2024-05-11T13:29:40.552365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:20:47.188238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536721+0000","last_change":"2024-05-11T13:28:35.209762+0000","last_active":"2024-05-11T13:29:40.536721+0000","last_peered":"2024-05-11T13:29:40.536721+0000","last_clean":"2024-05-11T13:29:40.536721+0000","last_became_active":"2024-05-11T13:28:35.207404+0000","last_became_peered":"2024-05-11T13:28:35.207404+0000","last_unstale":"2024-05-11T13:29:40.536721+0000","last_undegraded":"2024-05-11T13:29:40.536721+0000","last_fullsized":"2024-05-11T13:29:40.536721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:41:20.808009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535872+0000","last_change":"2024-05-11T13:28:35.215137+0000","last_active":"2024-05-11T13:29:40.535872+0000","last_peered":"2024-05-11T13:29:40.535872+0000","last_clean":"2024-05-11T13:29:40.535872+0000","last_became_active":"2024-05-11T13:28:35.214745+0000","last_became_peered":"2024-05-11T13:28:35.214745+0000","last_unstale":"2024-05-11T13:29:40.535872+0000","last_undegraded":"2024-05-11T13:29:40.535872+0000","last_fullsized":"2024-05-11T13:29:40.535872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:44.381501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69861,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":367,"num_read_kb":260,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":205318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":500505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874288,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1703936,"data_stored":1149765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037816,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":503927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":499605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430279,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1097097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":495726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":559413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738487,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":496791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":495043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953564,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":500422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790037,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1091897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T13:30:35.813 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T13:30:36.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:35 smithi063 ceph-mon[29809]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:30:36.279 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:36.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:35 smithi165 ceph-mon[33235]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:30:36.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:35 smithi071 ceph-mon[33293]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:30:37.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:36 smithi063 ceph-mon[29809]: pgmap v436: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:37.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:36 smithi165 ceph-mon[33235]: pgmap v436: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:37.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:36 smithi071 ceph-mon[33293]: pgmap v436: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-11T13:30:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.517+0000 7f56b9800700 1 -- 172.21.15.63:0/3199220989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 msgr2=0x7f56b410d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.517+0000 7f56b9800700 1 --2- 172.21.15.63:0/3199220989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b410d4c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009990 tx=0x7f56a0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.518+0000 7f56b9800700 1 -- 172.21.15.63:0/3199220989 shutdown_connections 2024-05-11T13:30:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.518+0000 7f56b9800700 1 --2- 172.21.15.63:0/3199220989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b410dae0 0x7f56b410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.518+0000 7f56b9800700 1 --2- 172.21.15.63:0/3199220989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56b41046b0 0x7f56b4104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.518+0000 7f56b9800700 1 --2- 172.21.15.63:0/3199220989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b410d4c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.518+0000 7f56b9800700 1 -- 172.21.15.63:0/3199220989 >> 172.21.15.63:0/3199220989 conn(0x7f56b40fdf00 msgr2=0x7f56b4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.518+0000 7f56b9800700 1 -- 172.21.15.63:0/3199220989 shutdown_connections 2024-05-11T13:30:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.518+0000 7f56b9800700 1 -- 172.21.15.63:0/3199220989 wait complete. 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.519+0000 7f56b9800700 1 Processor -- start 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.519+0000 7f56b9800700 1 -- start start 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.519+0000 7f56b9800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56b41046b0 0x7f56b419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.519+0000 7f56b9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.519+0000 7f56b9800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b410dae0 0x7f56b41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b9800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4113570 con 0x7f56b4104fd0 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b9800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41133f0 con 0x7f56b410dae0 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b9800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41136f0 con 0x7f56b41046b0 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54146/0 (socket says 172.21.15.63:54146) 2024-05-11T13:30:38.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 -- 172.21.15.63:0/2456799821 learned_addr learned my addr 172.21.15.63:0/2456799821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3e00700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56b41046b0 0x7f56b419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b8e00700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b410dae0 0x7f56b41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 -- 172.21.15.63:0/2456799821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56b41046b0 msgr2=0x7f56b419bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56b41046b0 0x7f56b419bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 -- 172.21.15.63:0/2456799821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b410dae0 msgr2=0x7f56b41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b410dae0 0x7f56b41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.520+0000 7f56b3400700 1 -- 172.21.15.63:0/2456799821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b41a0d50 con 0x7f56b4104fd0 2024-05-11T13:30:38.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.521+0000 7f56b3400700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b419c280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56a0000c00 tx=0x7f56a0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:38.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.521+0000 7f56b0c00700 1 -- 172.21.15.63:0/2456799821 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0027070 con 0x7f56b4104fd0 2024-05-11T13:30:38.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.521+0000 7f56b0c00700 1 -- 172.21.15.63:0/2456799821 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56a000fcc0 con 0x7f56b4104fd0 2024-05-11T13:30:38.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.521+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41a0fe0 con 0x7f56b4104fd0 2024-05-11T13:30:38.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.521+0000 7f56b0c00700 1 -- 172.21.15.63:0/2456799821 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0022c30 con 0x7f56b4104fd0 2024-05-11T13:30:38.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.521+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41a88c0 con 0x7f56b4104fd0 2024-05-11T13:30:38.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.523+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b406a240 con 0x7f56b4104fd0 2024-05-11T13:30:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.525+0000 7f56b0c00700 1 -- 172.21.15.63:0/2456799821 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f56a002a070 con 0x7f56b4104fd0 2024-05-11T13:30:38.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.526+0000 7f56b0c00700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56980b2990 0x7f56980b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.527+0000 7f56b0c00700 1 -- 172.21.15.63:0/2456799821 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f56a00e37c0 con 0x7f56b4104fd0 2024-05-11T13:30:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.527+0000 7f56b3e00700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56980b2990 0x7f56980b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.527+0000 7f56b3e00700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56980b2990 0x7f56980b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56a4005950 tx=0x7f56a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:38.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.529+0000 7f56b0c00700 1 -- 172.21.15.63:0/2456799821 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56a0031080 con 0x7f56b4104fd0 2024-05-11T13:30:38.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.764+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56b4103ac0 con 0x7f56980b2990 2024-05-11T13:30:38.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.792+0000 7f56b0c00700 1 -- 172.21.15.63:0/2456799821 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340410 (secure 0 0 0) 0x7f56b4103ac0 con 0x7f56980b2990 2024-05-11T13:30:38.796 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56980b2990 msgr2=0x7f56980b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56980b2990 0x7f56980b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56a4005950 tx=0x7f56a40058e0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 msgr2=0x7f56b419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b419c280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56a0000c00 tx=0x7f56a0003680 comp rx=0 tx=0).stop 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 shutdown_connections 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f56980b2990 0x7f56980b4e50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b410dae0 0x7f56b41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56b41046b0 0x7f56b419bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 --2- 172.21.15.63:0/2456799821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56b4104fd0 0x7f56b419c280 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:38.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 >> 172.21.15.63:0/2456799821 conn(0x7f56b40fdf00 msgr2=0x7f56b410ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:38.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 shutdown_connections 2024-05-11T13:30:38.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:38.800+0000 7f56b9800700 1 -- 172.21.15.63:0/2456799821 wait complete. 2024-05-11T13:30:38.802 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-11T13:30:39.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:38 smithi063 ceph-mon[29809]: pgmap v437: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-11T13:30:39.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:38 smithi165 ceph-mon[33235]: pgmap v437: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-11T13:30:39.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:38 smithi071 ceph-mon[33293]: pgmap v437: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-11T13:30:39.709 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-05-11T13:30:37.597641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660463,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":493,"num_read_kb":354,"num_write":373,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375584,"kb_used_data":13952,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566240,"statfs":{"total":1151940427776,"available":1151555829760,"internally_reserved":0,"allocated":14286848,"data_stored":7948811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9668,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":86,"num_write":10,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006400"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537148+0000","last_change":"2024-05-11T13:28:35.223417+0000","last_active":"2024-05-11T13:29:40.537148+0000","last_peered":"2024-05-11T13:29:40.537148+0000","last_clean":"2024-05-11T13:29:40.537148+0000","last_became_active":"2024-05-11T13:28:35.222862+0000","last_became_peered":"2024-05-11T13:28:35.222862+0000","last_unstale":"2024-05-11T13:29:40.537148+0000","last_undegraded":"2024-05-11T13:29:40.537148+0000","last_fullsized":"2024-05-11T13:29:40.537148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:20.259043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844676+0000","last_change":"2024-05-11T13:28:35.222289+0000","last_active":"2024-05-11T13:29:40.844676+0000","last_peered":"2024-05-11T13:29:40.844676+0000","last_clean":"2024-05-11T13:29:40.844676+0000","last_became_active":"2024-05-11T13:28:35.221366+0000","last_became_peered":"2024-05-11T13:28:35.221366+0000","last_unstale":"2024-05-11T13:29:40.844676+0000","last_undegraded":"2024-05-11T13:29:40.844676+0000","last_fullsized":"2024-05-11T13:29:40.844676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:02.648140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843736+0000","last_change":"2024-05-11T13:28:35.221271+0000","last_active":"2024-05-11T13:29:40.843736+0000","last_peered":"2024-05-11T13:29:40.843736+0000","last_clean":"2024-05-11T13:29:40.843736+0000","last_became_active":"2024-05-11T13:28:35.220294+0000","last_became_peered":"2024-05-11T13:28:35.220294+0000","last_unstale":"2024-05-11T13:29:40.843736+0000","last_undegraded":"2024-05-11T13:29:40.843736+0000","last_fullsized":"2024-05-11T13:29:40.843736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:04.608042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550631+0000","last_change":"2024-05-11T13:28:35.197643+0000","last_active":"2024-05-11T13:29:40.550631+0000","last_peered":"2024-05-11T13:29:40.550631+0000","last_clean":"2024-05-11T13:29:40.550631+0000","last_became_active":"2024-05-11T13:28:35.196528+0000","last_became_peered":"2024-05-11T13:28:35.196528+0000","last_unstale":"2024-05-11T13:29:40.550631+0000","last_undegraded":"2024-05-11T13:29:40.550631+0000","last_fullsized":"2024-05-11T13:29:40.550631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:02:03.088456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550105+0000","last_change":"2024-05-11T13:28:35.206833+0000","last_active":"2024-05-11T13:29:40.550105+0000","last_peered":"2024-05-11T13:29:40.550105+0000","last_clean":"2024-05-11T13:29:40.550105+0000","last_became_active":"2024-05-11T13:28:35.206287+0000","last_became_peered":"2024-05-11T13:28:35.206287+0000","last_unstale":"2024-05-11T13:29:40.550105+0000","last_undegraded":"2024-05-11T13:29:40.550105+0000","last_fullsized":"2024-05-11T13:29:40.550105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:25:20.576464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.538289+0000","last_change":"2024-05-11T13:28:35.218950+0000","last_active":"2024-05-11T13:29:40.538289+0000","last_peered":"2024-05-11T13:29:40.538289+0000","last_clean":"2024-05-11T13:29:40.538289+0000","last_became_active":"2024-05-11T13:28:35.218597+0000","last_became_peered":"2024-05-11T13:28:35.218597+0000","last_unstale":"2024-05-11T13:29:40.538289+0000","last_undegraded":"2024-05-11T13:29:40.538289+0000","last_fullsized":"2024-05-11T13:29:40.538289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:08:42.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.541106+0000","last_change":"2024-05-11T13:29:40.541106+0000","last_active":"2024-05-11T13:29:40.541106+0000","last_peered":"2024-05-11T13:29:40.541106+0000","last_clean":"2024-05-11T13:29:40.541106+0000","last_became_active":"2024-05-11T13:29:40.540007+0000","last_became_peered":"2024-05-11T13:29:40.540007+0000","last_unstale":"2024-05-11T13:29:40.541106+0000","last_undegraded":"2024-05-11T13:29:40.541106+0000","last_fullsized":"2024-05-11T13:29:40.541106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:00.596580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545783+0000","last_change":"2024-05-11T13:28:35.205483+0000","last_active":"2024-05-11T13:29:40.545783+0000","last_peered":"2024-05-11T13:29:40.545783+0000","last_clean":"2024-05-11T13:29:40.545783+0000","last_became_active":"2024-05-11T13:28:35.205101+0000","last_became_peered":"2024-05-11T13:28:35.205101+0000","last_unstale":"2024-05-11T13:29:40.545783+0000","last_undegraded":"2024-05-11T13:29:40.545783+0000","last_fullsized":"2024-05-11T13:29:40.545783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:37.917464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.552796+0000","last_change":"2024-05-11T13:28:35.217372+0000","last_active":"2024-05-11T13:29:40.552796+0000","last_peered":"2024-05-11T13:29:40.552796+0000","last_clean":"2024-05-11T13:29:40.552796+0000","last_became_active":"2024-05-11T13:28:35.217021+0000","last_became_peered":"2024-05-11T13:28:35.217021+0000","last_unstale":"2024-05-11T13:29:40.552796+0000","last_undegraded":"2024-05-11T13:29:40.552796+0000","last_fullsized":"2024-05-11T13:29:40.552796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:24.273041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548634+0000","last_change":"2024-05-11T13:29:40.548634+0000","last_active":"2024-05-11T13:29:40.548634+0000","last_peered":"2024-05-11T13:29:40.548634+0000","last_clean":"2024-05-11T13:29:40.548634+0000","last_became_active":"2024-05-11T13:29:40.547536+0000","last_became_peered":"2024-05-11T13:29:40.547536+0000","last_unstale":"2024-05-11T13:29:40.548634+0000","last_undegraded":"2024-05-11T13:29:40.548634+0000","last_fullsized":"2024-05-11T13:29:40.548634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:15.362317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531686+0000","last_change":"2024-05-11T13:28:35.214091+0000","last_active":"2024-05-11T13:29:40.531686+0000","last_peered":"2024-05-11T13:29:40.531686+0000","last_clean":"2024-05-11T13:29:40.531686+0000","last_became_active":"2024-05-11T13:28:35.213668+0000","last_became_peered":"2024-05-11T13:28:35.213668+0000","last_unstale":"2024-05-11T13:29:40.531686+0000","last_undegraded":"2024-05-11T13:29:40.531686+0000","last_fullsized":"2024-05-11T13:29:40.531686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:19:55.768203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550431+0000","last_change":"2024-05-11T13:28:35.211874+0000","last_active":"2024-05-11T13:29:40.550431+0000","last_peered":"2024-05-11T13:29:40.550431+0000","last_clean":"2024-05-11T13:29:40.550431+0000","last_became_active":"2024-05-11T13:28:35.211503+0000","last_became_peered":"2024-05-11T13:28:35.211503+0000","last_unstale":"2024-05-11T13:29:40.550431+0000","last_undegraded":"2024-05-11T13:29:40.550431+0000","last_fullsized":"2024-05-11T13:29:40.550431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:46.884110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537167+0000","last_change":"2024-05-11T13:28:35.209308+0000","last_active":"2024-05-11T13:29:40.537167+0000","last_peered":"2024-05-11T13:29:40.537167+0000","last_clean":"2024-05-11T13:29:40.537167+0000","last_became_active":"2024-05-11T13:28:35.207927+0000","last_became_peered":"2024-05-11T13:28:35.207927+0000","last_unstale":"2024-05-11T13:29:40.537167+0000","last_undegraded":"2024-05-11T13:29:40.537167+0000","last_fullsized":"2024-05-11T13:29:40.537167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:04.541849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843334+0000","last_change":"2024-05-11T13:28:35.208524+0000","last_active":"2024-05-11T13:29:40.843334+0000","last_peered":"2024-05-11T13:29:40.843334+0000","last_clean":"2024-05-11T13:29:40.843334+0000","last_became_active":"2024-05-11T13:28:35.207875+0000","last_became_peered":"2024-05-11T13:28:35.207875+0000","last_unstale":"2024-05-11T13:29:40.843334+0000","last_undegraded":"2024-05-11T13:29:40.843334+0000","last_fullsized":"2024-05-11T13:29:40.843334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:23:38.896916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843412+0000","last_change":"2024-05-11T13:28:35.216418+0000","last_active":"2024-05-11T13:29:40.843412+0000","last_peered":"2024-05-11T13:29:40.843412+0000","last_clean":"2024-05-11T13:29:40.843412+0000","last_became_active":"2024-05-11T13:28:35.215984+0000","last_became_peered":"2024-05-11T13:28:35.215984+0000","last_unstale":"2024-05-11T13:29:40.843412+0000","last_undegraded":"2024-05-11T13:29:40.843412+0000","last_fullsized":"2024-05-11T13:29:40.843412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:15:13.650821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548506+0000","last_change":"2024-05-11T13:28:35.217701+0000","last_active":"2024-05-11T13:29:40.548506+0000","last_peered":"2024-05-11T13:29:40.548506+0000","last_clean":"2024-05-11T13:29:40.548506+0000","last_became_active":"2024-05-11T13:28:35.217050+0000","last_became_peered":"2024-05-11T13:28:35.217050+0000","last_unstale":"2024-05-11T13:29:40.548506+0000","last_undegraded":"2024-05-11T13:29:40.548506+0000","last_fullsized":"2024-05-11T13:29:40.548506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:29:38.274540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536406+0000","last_change":"2024-05-11T13:28:35.223794+0000","last_active":"2024-05-11T13:29:40.536406+0000","last_peered":"2024-05-11T13:29:40.536406+0000","last_clean":"2024-05-11T13:29:40.536406+0000","last_became_active":"2024-05-11T13:28:35.223388+0000","last_became_peered":"2024-05-11T13:28:35.223388+0000","last_unstale":"2024-05-11T13:29:40.536406+0000","last_undegraded":"2024-05-11T13:29:40.536406+0000","last_fullsized":"2024-05-11T13:29:40.536406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:56:13.272357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.528208+0000","last_change":"2024-05-11T13:28:35.216452+0000","last_active":"2024-05-11T13:29:40.528208+0000","last_peered":"2024-05-11T13:29:40.528208+0000","last_clean":"2024-05-11T13:29:40.528208+0000","last_became_active":"2024-05-11T13:28:35.216075+0000","last_became_peered":"2024-05-11T13:28:35.216075+0000","last_unstale":"2024-05-11T13:29:40.528208+0000","last_undegraded":"2024-05-11T13:29:40.528208+0000","last_fullsized":"2024-05-11T13:29:40.528208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:23.012302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535020+0000","last_change":"2024-05-11T13:28:35.206528+0000","last_active":"2024-05-11T13:29:40.535020+0000","last_peered":"2024-05-11T13:29:40.535020+0000","last_clean":"2024-05-11T13:29:40.535020+0000","last_became_active":"2024-05-11T13:28:35.206102+0000","last_became_peered":"2024-05-11T13:28:35.206102+0000","last_unstale":"2024-05-11T13:29:40.535020+0000","last_undegraded":"2024-05-11T13:29:40.535020+0000","last_fullsized":"2024-05-11T13:29:40.535020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.685477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533743+0000","last_change":"2024-05-11T13:28:49.683087+0000","last_active":"2024-05-11T13:29:40.533743+0000","last_peered":"2024-05-11T13:29:40.533743+0000","last_clean":"2024-05-11T13:29:40.533743+0000","last_became_active":"2024-05-11T13:28:49.681968+0000","last_became_peered":"2024-05-11T13:28:49.681968+0000","last_unstale":"2024-05-11T13:29:40.533743+0000","last_undegraded":"2024-05-11T13:29:40.533743+0000","last_fullsized":"2024-05-11T13:29:40.533743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:08:04.594962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:34.897442+0000","last_change":"2024-05-11T13:28:30.428076+0000","last_active":"2024-05-11T13:30:34.897442+0000","last_peered":"2024-05-11T13:30:34.897442+0000","last_clean":"2024-05-11T13:30:34.897442+0000","last_became_active":"2024-05-11T13:28:30.427519+0000","last_became_peered":"2024-05-11T13:28:30.427519+0000","last_unstale":"2024-05-11T13:30:34.897442+0000","last_undegraded":"2024-05-11T13:30:34.897442+0000","last_fullsized":"2024-05-11T13:30:34.897442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:52.808051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546766+0000","last_change":"2024-05-11T13:28:35.207587+0000","last_active":"2024-05-11T13:29:40.546766+0000","last_peered":"2024-05-11T13:29:40.546766+0000","last_clean":"2024-05-11T13:29:40.546766+0000","last_became_active":"2024-05-11T13:28:35.206692+0000","last_became_peered":"2024-05-11T13:28:35.206692+0000","last_unstale":"2024-05-11T13:29:40.546766+0000","last_undegraded":"2024-05-11T13:29:40.546766+0000","last_fullsized":"2024-05-11T13:29:40.546766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:25.171342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545088+0000","last_change":"2024-05-11T13:28:49.682649+0000","last_active":"2024-05-11T13:29:40.545088+0000","last_peered":"2024-05-11T13:29:40.545088+0000","last_clean":"2024-05-11T13:29:40.545088+0000","last_became_active":"2024-05-11T13:28:49.682192+0000","last_became_peered":"2024-05-11T13:28:49.682192+0000","last_unstale":"2024-05-11T13:29:40.545088+0000","last_undegraded":"2024-05-11T13:29:40.545088+0000","last_fullsized":"2024-05-11T13:29:40.545088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:01.728705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:55.749814+0000","last_change":"2024-05-11T13:28:30.707050+0000","last_active":"2024-05-11T13:29:55.749814+0000","last_peered":"2024-05-11T13:29:55.749814+0000","last_clean":"2024-05-11T13:29:55.749814+0000","last_became_active":"2024-05-11T13:28:30.706279+0000","last_became_peered":"2024-05-11T13:28:30.706279+0000","last_unstale":"2024-05-11T13:29:55.749814+0000","last_undegraded":"2024-05-11T13:29:55.749814+0000","last_fullsized":"2024-05-11T13:29:55.749814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:14.135734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844412+0000","last_change":"2024-05-11T13:28:35.219622+0000","last_active":"2024-05-11T13:29:40.844412+0000","last_peered":"2024-05-11T13:29:40.844412+0000","last_clean":"2024-05-11T13:29:40.844412+0000","last_became_active":"2024-05-11T13:28:35.218687+0000","last_became_peered":"2024-05-11T13:28:35.218687+0000","last_unstale":"2024-05-11T13:29:40.844412+0000","last_undegraded":"2024-05-11T13:29:40.844412+0000","last_fullsized":"2024-05-11T13:29:40.844412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:53.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545586+0000","last_change":"2024-05-11T13:28:49.685770+0000","last_active":"2024-05-11T13:29:40.545586+0000","last_peered":"2024-05-11T13:29:40.545586+0000","last_clean":"2024-05-11T13:29:40.545586+0000","last_became_active":"2024-05-11T13:28:49.685091+0000","last_became_peered":"2024-05-11T13:28:49.685091+0000","last_unstale":"2024-05-11T13:29:40.545586+0000","last_undegraded":"2024-05-11T13:29:40.545586+0000","last_fullsized":"2024-05-11T13:29:40.545586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:42.458310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:58.785640+0000","last_change":"2024-05-11T13:28:30.435304+0000","last_active":"2024-05-11T13:29:58.785640+0000","last_peered":"2024-05-11T13:29:58.785640+0000","last_clean":"2024-05-11T13:29:58.785640+0000","last_became_active":"2024-05-11T13:28:30.434648+0000","last_became_peered":"2024-05-11T13:28:30.434648+0000","last_unstale":"2024-05-11T13:29:58.785640+0000","last_undegraded":"2024-05-11T13:29:58.785640+0000","last_fullsized":"2024-05-11T13:29:58.785640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:21.553376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548611+0000","last_change":"2024-05-11T13:28:35.210877+0000","last_active":"2024-05-11T13:29:40.548611+0000","last_peered":"2024-05-11T13:29:40.548611+0000","last_clean":"2024-05-11T13:29:40.548611+0000","last_became_active":"2024-05-11T13:28:35.207069+0000","last_became_peered":"2024-05-11T13:28:35.207069+0000","last_unstale":"2024-05-11T13:29:40.548611+0000","last_undegraded":"2024-05-11T13:29:40.548611+0000","last_fullsized":"2024-05-11T13:29:40.548611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:59.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545234+0000","last_change":"2024-05-11T13:28:49.689070+0000","last_active":"2024-05-11T13:29:40.545234+0000","last_peered":"2024-05-11T13:29:40.545234+0000","last_clean":"2024-05-11T13:29:40.545234+0000","last_became_active":"2024-05-11T13:28:49.688502+0000","last_became_peered":"2024-05-11T13:28:49.688502+0000","last_unstale":"2024-05-11T13:29:40.545234+0000","last_undegraded":"2024-05-11T13:29:40.545234+0000","last_fullsized":"2024-05-11T13:29:40.545234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:21.516074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778649+0000","last_change":"2024-05-11T13:29:40.564494+0000","last_active":"2024-05-11T13:29:57.778649+0000","last_peered":"2024-05-11T13:29:57.778649+0000","last_clean":"2024-05-11T13:29:57.778649+0000","last_became_active":"2024-05-11T13:29:40.555386+0000","last_became_peered":"2024-05-11T13:29:40.555386+0000","last_unstale":"2024-05-11T13:29:57.778649+0000","last_undegraded":"2024-05-11T13:29:57.778649+0000","last_fullsized":"2024-05-11T13:29:57.778649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:32:50.641372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.542113+0000","last_change":"2024-05-11T13:28:35.217950+0000","last_active":"2024-05-11T13:29:40.542113+0000","last_peered":"2024-05-11T13:29:40.542113+0000","last_clean":"2024-05-11T13:29:40.542113+0000","last_became_active":"2024-05-11T13:28:35.216698+0000","last_became_peered":"2024-05-11T13:28:35.216698+0000","last_unstale":"2024-05-11T13:29:40.542113+0000","last_undegraded":"2024-05-11T13:29:40.542113+0000","last_fullsized":"2024-05-11T13:29:40.542113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:01.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.534360+0000","last_change":"2024-05-11T13:28:49.687889+0000","last_active":"2024-05-11T13:29:40.534360+0000","last_peered":"2024-05-11T13:29:40.534360+0000","last_clean":"2024-05-11T13:29:40.534360+0000","last_became_active":"2024-05-11T13:28:49.684980+0000","last_became_peered":"2024-05-11T13:28:49.684980+0000","last_unstale":"2024-05-11T13:29:40.534360+0000","last_undegraded":"2024-05-11T13:29:40.534360+0000","last_fullsized":"2024-05-11T13:29:40.534360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:02:59.751371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:35.905521+0000","last_change":"2024-05-11T13:29:40.555238+0000","last_active":"2024-05-11T13:30:35.905521+0000","last_peered":"2024-05-11T13:30:35.905521+0000","last_clean":"2024-05-11T13:30:35.905521+0000","last_became_active":"2024-05-11T13:29:40.554661+0000","last_became_peered":"2024-05-11T13:29:40.554661+0000","last_unstale":"2024-05-11T13:30:35.905521+0000","last_undegraded":"2024-05-11T13:30:35.905521+0000","last_fullsized":"2024-05-11T13:30:35.905521+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:22.983649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549668+0000","last_change":"2024-05-11T13:28:35.209163+0000","last_active":"2024-05-11T13:29:40.549668+0000","last_peered":"2024-05-11T13:29:40.549668+0000","last_clean":"2024-05-11T13:29:40.549668+0000","last_became_active":"2024-05-11T13:28:35.204014+0000","last_became_peered":"2024-05-11T13:28:35.204014+0000","last_unstale":"2024-05-11T13:29:40.549668+0000","last_undegraded":"2024-05-11T13:29:40.549668+0000","last_fullsized":"2024-05-11T13:29:40.549668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:17.570863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.555520+0000","last_change":"2024-05-11T13:29:40.555520+0000","last_active":"2024-05-11T13:29:40.555520+0000","last_peered":"2024-05-11T13:29:40.555520+0000","last_clean":"2024-05-11T13:29:40.555520+0000","last_became_active":"2024-05-11T13:29:40.553952+0000","last_became_peered":"2024-05-11T13:29:40.553952+0000","last_unstale":"2024-05-11T13:29:40.555520+0000","last_undegraded":"2024-05-11T13:29:40.555520+0000","last_fullsized":"2024-05-11T13:29:40.555520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.035691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.761820+0000","last_change":"2024-05-11T13:28:30.429829+0000","last_active":"2024-05-11T13:29:56.761820+0000","last_peered":"2024-05-11T13:29:56.761820+0000","last_clean":"2024-05-11T13:29:56.761820+0000","last_became_active":"2024-05-11T13:28:30.428818+0000","last_became_peered":"2024-05-11T13:28:30.428818+0000","last_unstale":"2024-05-11T13:29:56.761820+0000","last_undegraded":"2024-05-11T13:29:56.761820+0000","last_fullsized":"2024-05-11T13:29:56.761820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:59:25.653181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543583+0000","last_change":"2024-05-11T13:28:35.214153+0000","last_active":"2024-05-11T13:29:40.543583+0000","last_peered":"2024-05-11T13:29:40.543583+0000","last_clean":"2024-05-11T13:29:40.543583+0000","last_became_active":"2024-05-11T13:28:35.213800+0000","last_became_peered":"2024-05-11T13:28:35.213800+0000","last_unstale":"2024-05-11T13:29:40.543583+0000","last_undegraded":"2024-05-11T13:29:40.543583+0000","last_fullsized":"2024-05-11T13:29:40.543583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:12.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536071+0000","last_change":"2024-05-11T13:28:49.697320+0000","last_active":"2024-05-11T13:29:40.536071+0000","last_peered":"2024-05-11T13:29:40.536071+0000","last_clean":"2024-05-11T13:29:40.536071+0000","last_became_active":"2024-05-11T13:28:49.696751+0000","last_became_peered":"2024-05-11T13:28:49.696751+0000","last_unstale":"2024-05-11T13:29:40.536071+0000","last_undegraded":"2024-05-11T13:29:40.536071+0000","last_fullsized":"2024-05-11T13:29:40.536071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:24.617241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:55.754505+0000","last_change":"2024-05-11T13:28:30.423943+0000","last_active":"2024-05-11T13:29:55.754505+0000","last_peered":"2024-05-11T13:29:55.754505+0000","last_clean":"2024-05-11T13:29:55.754505+0000","last_became_active":"2024-05-11T13:28:30.423454+0000","last_became_peered":"2024-05-11T13:28:30.423454+0000","last_unstale":"2024-05-11T13:29:55.754505+0000","last_undegraded":"2024-05-11T13:29:55.754505+0000","last_fullsized":"2024-05-11T13:29:55.754505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:23:10.967537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545683+0000","last_change":"2024-05-11T13:28:35.220479+0000","last_active":"2024-05-11T13:29:40.545683+0000","last_peered":"2024-05-11T13:29:40.545683+0000","last_clean":"2024-05-11T13:29:40.545683+0000","last_became_active":"2024-05-11T13:28:35.214541+0000","last_became_peered":"2024-05-11T13:28:35.214541+0000","last_unstale":"2024-05-11T13:29:40.545683+0000","last_undegraded":"2024-05-11T13:29:40.545683+0000","last_fullsized":"2024-05-11T13:29:40.545683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:07.746301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844335+0000","last_change":"2024-05-11T13:28:49.691867+0000","last_active":"2024-05-11T13:29:40.844335+0000","last_peered":"2024-05-11T13:29:40.844335+0000","last_clean":"2024-05-11T13:29:40.844335+0000","last_became_active":"2024-05-11T13:28:49.690045+0000","last_became_peered":"2024-05-11T13:28:49.690045+0000","last_unstale":"2024-05-11T13:29:40.844335+0000","last_undegraded":"2024-05-11T13:29:40.844335+0000","last_fullsized":"2024-05-11T13:29:40.844335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:16.220334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.776314+0000","last_change":"2024-05-11T13:28:30.436354+0000","last_active":"2024-05-11T13:29:57.776314+0000","last_peered":"2024-05-11T13:29:57.776314+0000","last_clean":"2024-05-11T13:29:57.776314+0000","last_became_active":"2024-05-11T13:28:30.435983+0000","last_became_peered":"2024-05-11T13:28:30.435983+0000","last_unstale":"2024-05-11T13:29:57.776314+0000","last_undegraded":"2024-05-11T13:29:57.776314+0000","last_fullsized":"2024-05-11T13:29:57.776314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:42.442829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.527843+0000","last_change":"2024-05-11T13:28:35.207284+0000","last_active":"2024-05-11T13:29:40.527843+0000","last_peered":"2024-05-11T13:29:40.527843+0000","last_clean":"2024-05-11T13:29:40.527843+0000","last_became_active":"2024-05-11T13:28:35.206945+0000","last_became_peered":"2024-05-11T13:28:35.206945+0000","last_unstale":"2024-05-11T13:29:40.527843+0000","last_undegraded":"2024-05-11T13:29:40.527843+0000","last_fullsized":"2024-05-11T13:29:40.527843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:18.493659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546335+0000","last_change":"2024-05-11T13:28:49.689558+0000","last_active":"2024-05-11T13:29:40.546335+0000","last_peered":"2024-05-11T13:29:40.546335+0000","last_clean":"2024-05-11T13:29:40.546335+0000","last_became_active":"2024-05-11T13:28:49.689156+0000","last_became_peered":"2024-05-11T13:28:49.689156+0000","last_unstale":"2024-05-11T13:29:40.546335+0000","last_undegraded":"2024-05-11T13:29:40.546335+0000","last_fullsized":"2024-05-11T13:29:40.546335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:21.530111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:34.899361+0000","last_change":"2024-05-11T13:28:30.434534+0000","last_active":"2024-05-11T13:30:34.899361+0000","last_peered":"2024-05-11T13:30:34.899361+0000","last_clean":"2024-05-11T13:30:34.899361+0000","last_became_active":"2024-05-11T13:28:30.433983+0000","last_became_peered":"2024-05-11T13:28:30.433983+0000","last_unstale":"2024-05-11T13:30:34.899361+0000","last_undegraded":"2024-05-11T13:30:34.899361+0000","last_fullsized":"2024-05-11T13:30:34.899361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:16.056662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.547958+0000","last_change":"2024-05-11T13:28:35.205021+0000","last_active":"2024-05-11T13:29:40.547958+0000","last_peered":"2024-05-11T13:29:40.547958+0000","last_clean":"2024-05-11T13:29:40.547958+0000","last_became_active":"2024-05-11T13:28:35.204243+0000","last_became_peered":"2024-05-11T13:28:35.204243+0000","last_unstale":"2024-05-11T13:29:40.547958+0000","last_undegraded":"2024-05-11T13:29:40.547958+0000","last_fullsized":"2024-05-11T13:29:40.547958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:12.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.527578+0000","last_change":"2024-05-11T13:28:49.677392+0000","last_active":"2024-05-11T13:29:40.527578+0000","last_peered":"2024-05-11T13:29:40.527578+0000","last_clean":"2024-05-11T13:29:40.527578+0000","last_became_active":"2024-05-11T13:28:49.674008+0000","last_became_peered":"2024-05-11T13:28:49.674008+0000","last_unstale":"2024-05-11T13:29:40.527578+0000","last_undegraded":"2024-05-11T13:29:40.527578+0000","last_fullsized":"2024-05-11T13:29:40.527578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:47:14.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778632+0000","last_change":"2024-05-11T13:28:30.429500+0000","last_active":"2024-05-11T13:29:57.778632+0000","last_peered":"2024-05-11T13:29:57.778632+0000","last_clean":"2024-05-11T13:29:57.778632+0000","last_became_active":"2024-05-11T13:28:30.429049+0000","last_became_peered":"2024-05-11T13:28:30.429049+0000","last_unstale":"2024-05-11T13:29:57.778632+0000","last_undegraded":"2024-05-11T13:29:57.778632+0000","last_fullsized":"2024-05-11T13:29:57.778632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:00.533603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844631+0000","last_change":"2024-05-11T13:28:35.223764+0000","last_active":"2024-05-11T13:29:40.844631+0000","last_peered":"2024-05-11T13:29:40.844631+0000","last_clean":"2024-05-11T13:29:40.844631+0000","last_became_active":"2024-05-11T13:28:35.223016+0000","last_became_peered":"2024-05-11T13:28:35.223016+0000","last_unstale":"2024-05-11T13:29:40.844631+0000","last_undegraded":"2024-05-11T13:29:40.844631+0000","last_fullsized":"2024-05-11T13:29:40.844631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:42.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546160+0000","last_change":"2024-05-11T13:28:49.683733+0000","last_active":"2024-05-11T13:29:40.546160+0000","last_peered":"2024-05-11T13:29:40.546160+0000","last_clean":"2024-05-11T13:29:40.546160+0000","last_became_active":"2024-05-11T13:28:49.681245+0000","last_became_peered":"2024-05-11T13:28:49.681245+0000","last_unstale":"2024-05-11T13:29:40.546160+0000","last_undegraded":"2024-05-11T13:29:40.546160+0000","last_fullsized":"2024-05-11T13:29:40.546160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:03.386943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.763658+0000","last_change":"2024-05-11T13:28:30.707394+0000","last_active":"2024-05-11T13:29:56.763658+0000","last_peered":"2024-05-11T13:29:56.763658+0000","last_clean":"2024-05-11T13:29:56.763658+0000","last_became_active":"2024-05-11T13:28:30.706114+0000","last_became_peered":"2024-05-11T13:28:30.706114+0000","last_unstale":"2024-05-11T13:29:56.763658+0000","last_undegraded":"2024-05-11T13:29:56.763658+0000","last_fullsized":"2024-05-11T13:29:56.763658+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:53:04.599800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549373+0000","last_change":"2024-05-11T13:28:35.219836+0000","last_active":"2024-05-11T13:29:40.549373+0000","last_peered":"2024-05-11T13:29:40.549373+0000","last_clean":"2024-05-11T13:29:40.549373+0000","last_became_active":"2024-05-11T13:28:35.214474+0000","last_became_peered":"2024-05-11T13:28:35.214474+0000","last_unstale":"2024-05-11T13:29:40.549373+0000","last_undegraded":"2024-05-11T13:29:40.549373+0000","last_fullsized":"2024-05-11T13:29:40.549373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:02.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.528108+0000","last_change":"2024-05-11T13:28:49.694677+0000","last_active":"2024-05-11T13:29:40.528108+0000","last_peered":"2024-05-11T13:29:40.528108+0000","last_clean":"2024-05-11T13:29:40.528108+0000","last_became_active":"2024-05-11T13:28:49.694176+0000","last_became_peered":"2024-05-11T13:28:49.694176+0000","last_unstale":"2024-05-11T13:29:40.528108+0000","last_undegraded":"2024-05-11T13:29:40.528108+0000","last_fullsized":"2024-05-11T13:29:40.528108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:35.461937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":159,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:34.923382+0000","last_change":"2024-05-11T13:28:30.433202+0000","last_active":"2024-05-11T13:30:34.923382+0000","last_peered":"2024-05-11T13:30:34.923382+0000","last_clean":"2024-05-11T13:30:34.923382+0000","last_became_active":"2024-05-11T13:28:30.429269+0000","last_became_peered":"2024-05-11T13:28:30.429269+0000","last_unstale":"2024-05-11T13:30:34.923382+0000","last_undegraded":"2024-05-11T13:30:34.923382+0000","last_fullsized":"2024-05-11T13:30:34.923382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:27:15.620717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62657,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":84,"num_read_kb":103,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.551139+0000","last_change":"2024-05-11T13:28:35.210882+0000","last_active":"2024-05-11T13:29:40.551139+0000","last_peered":"2024-05-11T13:29:40.551139+0000","last_clean":"2024-05-11T13:29:40.551139+0000","last_became_active":"2024-05-11T13:28:35.210506+0000","last_became_peered":"2024-05-11T13:28:35.210506+0000","last_unstale":"2024-05-11T13:29:40.551139+0000","last_undegraded":"2024-05-11T13:29:40.551139+0000","last_fullsized":"2024-05-11T13:29:40.551139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:45.752666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543867+0000","last_change":"2024-05-11T13:28:49.693254+0000","last_active":"2024-05-11T13:29:40.543867+0000","last_peered":"2024-05-11T13:29:40.543867+0000","last_clean":"2024-05-11T13:29:40.543867+0000","last_became_active":"2024-05-11T13:28:49.690307+0000","last_became_peered":"2024-05-11T13:28:49.690307+0000","last_unstale":"2024-05-11T13:29:40.543867+0000","last_undegraded":"2024-05-11T13:29:40.543867+0000","last_fullsized":"2024-05-11T13:29:40.543867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:41:31.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:58.785687+0000","last_change":"2024-05-11T13:28:30.429421+0000","last_active":"2024-05-11T13:29:58.785687+0000","last_peered":"2024-05-11T13:29:58.785687+0000","last_clean":"2024-05-11T13:29:58.785687+0000","last_became_active":"2024-05-11T13:28:30.429032+0000","last_became_peered":"2024-05-11T13:28:30.429032+0000","last_unstale":"2024-05-11T13:29:58.785687+0000","last_undegraded":"2024-05-11T13:29:58.785687+0000","last_fullsized":"2024-05-11T13:29:58.785687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:06.234650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535909+0000","last_change":"2024-05-11T13:28:35.207709+0000","last_active":"2024-05-11T13:29:40.535909+0000","last_peered":"2024-05-11T13:29:40.535909+0000","last_clean":"2024-05-11T13:29:40.535909+0000","last_became_active":"2024-05-11T13:28:35.207222+0000","last_became_peered":"2024-05-11T13:28:35.207222+0000","last_unstale":"2024-05-11T13:29:40.535909+0000","last_undegraded":"2024-05-11T13:29:40.535909+0000","last_fullsized":"2024-05-11T13:29:40.535909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:20.023565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535394+0000","last_change":"2024-05-11T13:28:49.692046+0000","last_active":"2024-05-11T13:29:40.535394+0000","last_peered":"2024-05-11T13:29:40.535394+0000","last_clean":"2024-05-11T13:29:40.535394+0000","last_became_active":"2024-05-11T13:28:49.691618+0000","last_became_peered":"2024-05-11T13:28:49.691618+0000","last_unstale":"2024-05-11T13:29:40.535394+0000","last_undegraded":"2024-05-11T13:29:40.535394+0000","last_fullsized":"2024-05-11T13:29:40.535394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:05.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.773794+0000","last_change":"2024-05-11T13:28:30.435459+0000","last_active":"2024-05-11T13:29:57.773794+0000","last_peered":"2024-05-11T13:29:57.773794+0000","last_clean":"2024-05-11T13:29:57.773794+0000","last_became_active":"2024-05-11T13:28:30.434997+0000","last_became_peered":"2024-05-11T13:28:30.434997+0000","last_unstale":"2024-05-11T13:29:57.773794+0000","last_undegraded":"2024-05-11T13:29:57.773794+0000","last_fullsized":"2024-05-11T13:29:57.773794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:36.280546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531321+0000","last_change":"2024-05-11T13:28:35.211214+0000","last_active":"2024-05-11T13:29:40.531321+0000","last_peered":"2024-05-11T13:29:40.531321+0000","last_clean":"2024-05-11T13:29:40.531321+0000","last_became_active":"2024-05-11T13:28:35.210777+0000","last_became_peered":"2024-05-11T13:28:35.210777+0000","last_unstale":"2024-05-11T13:29:40.531321+0000","last_undegraded":"2024-05-11T13:29:40.531321+0000","last_fullsized":"2024-05-11T13:29:40.531321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:53.238549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.529049+0000","last_change":"2024-05-11T13:28:49.682336+0000","last_active":"2024-05-11T13:29:40.529049+0000","last_peered":"2024-05-11T13:29:40.529049+0000","last_clean":"2024-05-11T13:29:40.529049+0000","last_became_active":"2024-05-11T13:28:49.679081+0000","last_became_peered":"2024-05-11T13:28:49.679081+0000","last_unstale":"2024-05-11T13:29:40.529049+0000","last_undegraded":"2024-05-11T13:29:40.529049+0000","last_fullsized":"2024-05-11T13:29:40.529049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:00:21.149262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:32.911481+0000","last_change":"2024-05-11T13:28:30.432165+0000","last_active":"2024-05-11T13:30:32.911481+0000","last_peered":"2024-05-11T13:30:32.911481+0000","last_clean":"2024-05-11T13:30:32.911481+0000","last_became_active":"2024-05-11T13:28:30.431181+0000","last_became_peered":"2024-05-11T13:28:30.431181+0000","last_unstale":"2024-05-11T13:30:32.911481+0000","last_undegraded":"2024-05-11T13:30:32.911481+0000","last_fullsized":"2024-05-11T13:30:32.911481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:26:50.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530968+0000","last_change":"2024-05-11T13:28:35.205662+0000","last_active":"2024-05-11T13:29:40.530968+0000","last_peered":"2024-05-11T13:29:40.530968+0000","last_clean":"2024-05-11T13:29:40.530968+0000","last_became_active":"2024-05-11T13:28:35.205276+0000","last_became_peered":"2024-05-11T13:28:35.205276+0000","last_unstale":"2024-05-11T13:29:40.530968+0000","last_undegraded":"2024-05-11T13:29:40.530968+0000","last_fullsized":"2024-05-11T13:29:40.530968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:36.308402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537248+0000","last_change":"2024-05-11T13:28:49.689705+0000","last_active":"2024-05-11T13:29:40.537248+0000","last_peered":"2024-05-11T13:29:40.537248+0000","last_clean":"2024-05-11T13:29:40.537248+0000","last_became_active":"2024-05-11T13:28:49.689223+0000","last_became_peered":"2024-05-11T13:28:49.689223+0000","last_unstale":"2024-05-11T13:29:40.537248+0000","last_undegraded":"2024-05-11T13:29:40.537248+0000","last_fullsized":"2024-05-11T13:29:40.537248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:37:20.727773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.775555+0000","last_change":"2024-05-11T13:28:30.709014+0000","last_active":"2024-05-11T13:29:57.775555+0000","last_peered":"2024-05-11T13:29:57.775555+0000","last_clean":"2024-05-11T13:29:57.775555+0000","last_became_active":"2024-05-11T13:28:30.708000+0000","last_became_peered":"2024-05-11T13:28:30.708000+0000","last_unstale":"2024-05-11T13:29:57.775555+0000","last_undegraded":"2024-05-11T13:29:57.775555+0000","last_fullsized":"2024-05-11T13:29:57.775555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:00.813415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.544988+0000","last_change":"2024-05-11T13:28:35.215149+0000","last_active":"2024-05-11T13:29:40.544988+0000","last_peered":"2024-05-11T13:29:40.544988+0000","last_clean":"2024-05-11T13:29:40.544988+0000","last_became_active":"2024-05-11T13:28:35.214792+0000","last_became_peered":"2024-05-11T13:28:35.214792+0000","last_unstale":"2024-05-11T13:29:40.544988+0000","last_undegraded":"2024-05-11T13:29:40.544988+0000","last_fullsized":"2024-05-11T13:29:40.544988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.544787+0000","last_change":"2024-05-11T13:28:49.686347+0000","last_active":"2024-05-11T13:29:40.544787+0000","last_peered":"2024-05-11T13:29:40.544787+0000","last_clean":"2024-05-11T13:29:40.544787+0000","last_became_active":"2024-05-11T13:28:49.685391+0000","last_became_peered":"2024-05-11T13:28:49.685391+0000","last_unstale":"2024-05-11T13:29:40.544787+0000","last_undegraded":"2024-05-11T13:29:40.544787+0000","last_fullsized":"2024-05-11T13:29:40.544787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:24:50.347627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.741381+0000","last_change":"2024-05-11T13:29:40.560815+0000","last_active":"2024-05-11T13:29:56.741381+0000","last_peered":"2024-05-11T13:29:56.741381+0000","last_clean":"2024-05-11T13:29:56.741381+0000","last_became_active":"2024-05-11T13:29:40.553031+0000","last_became_peered":"2024-05-11T13:29:40.553031+0000","last_unstale":"2024-05-11T13:29:56.741381+0000","last_undegraded":"2024-05-11T13:29:56.741381+0000","last_fullsized":"2024-05-11T13:29:56.741381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:27.437654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533924+0000","last_change":"2024-05-11T13:28:35.218377+0000","last_active":"2024-05-11T13:29:40.533924+0000","last_peered":"2024-05-11T13:29:40.533924+0000","last_clean":"2024-05-11T13:29:40.533924+0000","last_became_active":"2024-05-11T13:28:35.217922+0000","last_became_peered":"2024-05-11T13:28:35.217922+0000","last_unstale":"2024-05-11T13:29:40.533924+0000","last_undegraded":"2024-05-11T13:29:40.533924+0000","last_fullsized":"2024-05-11T13:29:40.533924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:58.383381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549510+0000","last_change":"2024-05-11T13:28:49.723400+0000","last_active":"2024-05-11T13:29:40.549510+0000","last_peered":"2024-05-11T13:29:40.549510+0000","last_clean":"2024-05-11T13:29:40.549510+0000","last_became_active":"2024-05-11T13:28:49.722675+0000","last_became_peered":"2024-05-11T13:28:49.722675+0000","last_unstale":"2024-05-11T13:29:40.549510+0000","last_undegraded":"2024-05-11T13:29:40.549510+0000","last_fullsized":"2024-05-11T13:29:40.549510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:14:44.810541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.758503+0000","last_change":"2024-05-11T13:28:30.710667+0000","last_active":"2024-05-11T13:29:56.758503+0000","last_peered":"2024-05-11T13:29:56.758503+0000","last_clean":"2024-05-11T13:29:56.758503+0000","last_became_active":"2024-05-11T13:28:30.709933+0000","last_became_peered":"2024-05-11T13:28:30.709933+0000","last_unstale":"2024-05-11T13:29:56.758503+0000","last_undegraded":"2024-05-11T13:29:56.758503+0000","last_fullsized":"2024-05-11T13:29:56.758503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:34.942946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.539110+0000","last_change":"2024-05-11T13:28:35.215691+0000","last_active":"2024-05-11T13:29:40.539110+0000","last_peered":"2024-05-11T13:29:40.539110+0000","last_clean":"2024-05-11T13:29:40.539110+0000","last_became_active":"2024-05-11T13:28:35.209873+0000","last_became_peered":"2024-05-11T13:28:35.209873+0000","last_unstale":"2024-05-11T13:29:40.539110+0000","last_undegraded":"2024-05-11T13:29:40.539110+0000","last_fullsized":"2024-05-11T13:29:40.539110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:42:12.935570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536722+0000","last_change":"2024-05-11T13:28:49.694322+0000","last_active":"2024-05-11T13:29:40.536722+0000","last_peered":"2024-05-11T13:29:40.536722+0000","last_clean":"2024-05-11T13:29:40.536722+0000","last_became_active":"2024-05-11T13:28:49.690900+0000","last_became_peered":"2024-05-11T13:28:49.690900+0000","last_unstale":"2024-05-11T13:29:40.536722+0000","last_undegraded":"2024-05-11T13:29:40.536722+0000","last_fullsized":"2024-05-11T13:29:40.536722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:00:07.433371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:31.889892+0000","last_change":"2024-05-11T13:28:30.432529+0000","last_active":"2024-05-11T13:30:31.889892+0000","last_peered":"2024-05-11T13:30:31.889892+0000","last_clean":"2024-05-11T13:30:31.889892+0000","last_became_active":"2024-05-11T13:28:30.428077+0000","last_became_peered":"2024-05-11T13:28:30.428077+0000","last_unstale":"2024-05-11T13:30:31.889892+0000","last_undegraded":"2024-05-11T13:30:31.889892+0000","last_fullsized":"2024-05-11T13:30:31.889892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:42.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.538376+0000","last_change":"2024-05-11T13:28:35.219160+0000","last_active":"2024-05-11T13:29:40.538376+0000","last_peered":"2024-05-11T13:29:40.538376+0000","last_clean":"2024-05-11T13:29:40.538376+0000","last_became_active":"2024-05-11T13:28:35.216810+0000","last_became_peered":"2024-05-11T13:28:35.216810+0000","last_unstale":"2024-05-11T13:29:40.538376+0000","last_undegraded":"2024-05-11T13:29:40.538376+0000","last_fullsized":"2024-05-11T13:29:40.538376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:35:54.465030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536984+0000","last_change":"2024-05-11T13:28:49.725190+0000","last_active":"2024-05-11T13:29:40.536984+0000","last_peered":"2024-05-11T13:29:40.536984+0000","last_clean":"2024-05-11T13:29:40.536984+0000","last_became_active":"2024-05-11T13:28:49.724193+0000","last_became_peered":"2024-05-11T13:28:49.724193+0000","last_unstale":"2024-05-11T13:29:40.536984+0000","last_undegraded":"2024-05-11T13:29:40.536984+0000","last_fullsized":"2024-05-11T13:29:40.536984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:47:41.181606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543161+0000","last_change":"2024-05-11T13:28:35.211262+0000","last_active":"2024-05-11T13:29:40.543161+0000","last_peered":"2024-05-11T13:29:40.543161+0000","last_clean":"2024-05-11T13:29:40.543161+0000","last_became_active":"2024-05-11T13:28:35.210859+0000","last_became_peered":"2024-05-11T13:28:35.210859+0000","last_unstale":"2024-05-11T13:29:40.543161+0000","last_undegraded":"2024-05-11T13:29:40.543161+0000","last_fullsized":"2024-05-11T13:29:40.543161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:40:33.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:36.921900+0000","last_change":"2024-05-11T13:28:30.707326+0000","last_active":"2024-05-11T13:30:36.921900+0000","last_peered":"2024-05-11T13:30:36.921900+0000","last_clean":"2024-05-11T13:30:36.921900+0000","last_became_active":"2024-05-11T13:28:30.705496+0000","last_became_peered":"2024-05-11T13:28:30.705496+0000","last_unstale":"2024-05-11T13:30:36.921900+0000","last_undegraded":"2024-05-11T13:30:36.921900+0000","last_fullsized":"2024-05-11T13:30:36.921900+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:48.117551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531497+0000","last_change":"2024-05-11T13:28:49.687260+0000","last_active":"2024-05-11T13:29:40.531497+0000","last_peered":"2024-05-11T13:29:40.531497+0000","last_clean":"2024-05-11T13:29:40.531497+0000","last_became_active":"2024-05-11T13:28:49.686593+0000","last_became_peered":"2024-05-11T13:28:49.686593+0000","last_unstale":"2024-05-11T13:29:40.531497+0000","last_undegraded":"2024-05-11T13:29:40.531497+0000","last_fullsized":"2024-05-11T13:29:40.531497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:48.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546734+0000","last_change":"2024-05-11T13:28:35.216643+0000","last_active":"2024-05-11T13:29:40.546734+0000","last_peered":"2024-05-11T13:29:40.546734+0000","last_clean":"2024-05-11T13:29:40.546734+0000","last_became_active":"2024-05-11T13:28:35.216300+0000","last_became_peered":"2024-05-11T13:28:35.216300+0000","last_unstale":"2024-05-11T13:29:40.546734+0000","last_undegraded":"2024-05-11T13:29:40.546734+0000","last_fullsized":"2024-05-11T13:29:40.546734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:20:01.351678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:58.789699+0000","last_change":"2024-05-11T13:28:30.427465+0000","last_active":"2024-05-11T13:29:58.789699+0000","last_peered":"2024-05-11T13:29:58.789699+0000","last_clean":"2024-05-11T13:29:58.789699+0000","last_became_active":"2024-05-11T13:28:30.426968+0000","last_became_peered":"2024-05-11T13:28:30.426968+0000","last_unstale":"2024-05-11T13:29:58.789699+0000","last_undegraded":"2024-05-11T13:29:58.789699+0000","last_fullsized":"2024-05-11T13:29:58.789699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:02:44.969069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530098+0000","last_change":"2024-05-11T13:26:37.738725+0000","last_active":"2024-05-11T13:29:40.530098+0000","last_peered":"2024-05-11T13:29:40.530098+0000","last_clean":"2024-05-11T13:29:40.530098+0000","last_became_active":"2024-05-11T13:26:37.724302+0000","last_became_peered":"2024-05-11T13:26:37.724302+0000","last_unstale":"2024-05-11T13:29:40.530098+0000","last_undegraded":"2024-05-11T13:29:40.530098+0000","last_fullsized":"2024-05-11T13:29:40.530098+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_clean_scrub_stamp":"2024-05-11T13:21:55.918148+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:32:56.277780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.543703+0000","last_change":"2024-05-11T13:28:49.723829+0000","last_active":"2024-05-11T13:29:40.543703+0000","last_peered":"2024-05-11T13:29:40.543703+0000","last_clean":"2024-05-11T13:29:40.543703+0000","last_became_active":"2024-05-11T13:28:49.723190+0000","last_became_peered":"2024-05-11T13:28:49.723190+0000","last_unstale":"2024-05-11T13:29:40.543703+0000","last_undegraded":"2024-05-11T13:29:40.543703+0000","last_fullsized":"2024-05-11T13:29:40.543703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.742374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535793+0000","last_change":"2024-05-11T13:28:35.213486+0000","last_active":"2024-05-11T13:29:40.535793+0000","last_peered":"2024-05-11T13:29:40.535793+0000","last_clean":"2024-05-11T13:29:40.535793+0000","last_became_active":"2024-05-11T13:28:35.211962+0000","last_became_peered":"2024-05-11T13:28:35.211962+0000","last_unstale":"2024-05-11T13:29:40.535793+0000","last_undegraded":"2024-05-11T13:29:40.535793+0000","last_fullsized":"2024-05-11T13:29:40.535793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:28:34.046898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.774165+0000","last_change":"2024-05-11T13:28:30.431023+0000","last_active":"2024-05-11T13:29:57.774165+0000","last_peered":"2024-05-11T13:29:57.774165+0000","last_clean":"2024-05-11T13:29:57.774165+0000","last_became_active":"2024-05-11T13:28:30.430609+0000","last_became_peered":"2024-05-11T13:28:30.430609+0000","last_unstale":"2024-05-11T13:29:57.774165+0000","last_undegraded":"2024-05-11T13:29:57.774165+0000","last_fullsized":"2024-05-11T13:29:57.774165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:50.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533480+0000","last_change":"2024-05-11T13:28:49.695550+0000","last_active":"2024-05-11T13:29:40.533480+0000","last_peered":"2024-05-11T13:29:40.533480+0000","last_clean":"2024-05-11T13:29:40.533480+0000","last_became_active":"2024-05-11T13:28:49.694350+0000","last_became_peered":"2024-05-11T13:28:49.694350+0000","last_unstale":"2024-05-11T13:29:40.533480+0000","last_undegraded":"2024-05-11T13:29:40.533480+0000","last_fullsized":"2024-05-11T13:29:40.533480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:49.617673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.775618+0000","last_change":"2024-05-11T13:28:30.710826+0000","last_active":"2024-05-11T13:29:57.775618+0000","last_peered":"2024-05-11T13:29:57.775618+0000","last_clean":"2024-05-11T13:29:57.775618+0000","last_became_active":"2024-05-11T13:28:30.709854+0000","last_became_peered":"2024-05-11T13:28:30.709854+0000","last_unstale":"2024-05-11T13:29:57.775618+0000","last_undegraded":"2024-05-11T13:29:57.775618+0000","last_fullsized":"2024-05-11T13:29:57.775618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:35:16.384559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.537252+0000","last_change":"2024-05-11T13:28:35.205521+0000","last_active":"2024-05-11T13:29:40.537252+0000","last_peered":"2024-05-11T13:29:40.537252+0000","last_clean":"2024-05-11T13:29:40.537252+0000","last_became_active":"2024-05-11T13:28:35.205167+0000","last_became_peered":"2024-05-11T13:28:35.205167+0000","last_unstale":"2024-05-11T13:29:40.537252+0000","last_undegraded":"2024-05-11T13:29:40.537252+0000","last_fullsized":"2024-05-11T13:29:40.537252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:10:27.473991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545565+0000","last_change":"2024-05-11T13:28:49.723652+0000","last_active":"2024-05-11T13:29:40.545565+0000","last_peered":"2024-05-11T13:29:40.545565+0000","last_clean":"2024-05-11T13:29:40.545565+0000","last_became_active":"2024-05-11T13:28:49.723135+0000","last_became_peered":"2024-05-11T13:28:49.723135+0000","last_unstale":"2024-05-11T13:29:40.545565+0000","last_undegraded":"2024-05-11T13:29:40.545565+0000","last_fullsized":"2024-05-11T13:29:40.545565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:21:18.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:25.254699+0000","last_change":"2024-05-11T13:28:30.428197+0000","last_active":"2024-05-11T13:30:25.254699+0000","last_peered":"2024-05-11T13:30:25.254699+0000","last_clean":"2024-05-11T13:30:25.254699+0000","last_became_active":"2024-05-11T13:28:30.427753+0000","last_became_peered":"2024-05-11T13:28:30.427753+0000","last_unstale":"2024-05-11T13:30:25.254699+0000","last_undegraded":"2024-05-11T13:30:25.254699+0000","last_fullsized":"2024-05-11T13:30:25.254699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:52:54.165993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.545034+0000","last_change":"2024-05-11T13:29:40.541861+0000","last_active":"2024-05-11T13:29:40.545034+0000","last_peered":"2024-05-11T13:29:40.545034+0000","last_clean":"2024-05-11T13:29:40.545034+0000","last_became_active":"2024-05-11T13:29:40.541395+0000","last_became_peered":"2024-05-11T13:29:40.541395+0000","last_unstale":"2024-05-11T13:29:40.545034+0000","last_undegraded":"2024-05-11T13:29:40.545034+0000","last_fullsized":"2024-05-11T13:29:40.545034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:50:39.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.532327+0000","last_change":"2024-05-11T13:28:49.677549+0000","last_active":"2024-05-11T13:29:40.532327+0000","last_peered":"2024-05-11T13:29:40.532327+0000","last_clean":"2024-05-11T13:29:40.532327+0000","last_became_active":"2024-05-11T13:28:49.676625+0000","last_became_peered":"2024-05-11T13:28:49.676625+0000","last_unstale":"2024-05-11T13:29:40.532327+0000","last_undegraded":"2024-05-11T13:29:40.532327+0000","last_fullsized":"2024-05-11T13:29:40.532327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:44.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778619+0000","last_change":"2024-05-11T13:28:30.440945+0000","last_active":"2024-05-11T13:29:57.778619+0000","last_peered":"2024-05-11T13:29:57.778619+0000","last_clean":"2024-05-11T13:29:57.778619+0000","last_became_active":"2024-05-11T13:28:30.440595+0000","last_became_peered":"2024-05-11T13:28:30.440595+0000","last_unstale":"2024-05-11T13:29:57.778619+0000","last_undegraded":"2024-05-11T13:29:57.778619+0000","last_fullsized":"2024-05-11T13:29:57.778619+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:51:10.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844988+0000","last_change":"2024-05-11T13:28:35.214207+0000","last_active":"2024-05-11T13:29:40.844988+0000","last_peered":"2024-05-11T13:29:40.844988+0000","last_clean":"2024-05-11T13:29:40.844988+0000","last_became_active":"2024-05-11T13:28:35.213844+0000","last_became_peered":"2024-05-11T13:28:35.213844+0000","last_unstale":"2024-05-11T13:29:40.844988+0000","last_undegraded":"2024-05-11T13:29:40.844988+0000","last_fullsized":"2024-05-11T13:29:40.844988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:53.063329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.550204+0000","last_change":"2024-05-11T13:28:49.725505+0000","last_active":"2024-05-11T13:29:40.550204+0000","last_peered":"2024-05-11T13:29:40.550204+0000","last_clean":"2024-05-11T13:29:40.550204+0000","last_became_active":"2024-05-11T13:28:49.723613+0000","last_became_peered":"2024-05-11T13:28:49.723613+0000","last_unstale":"2024-05-11T13:29:40.550204+0000","last_undegraded":"2024-05-11T13:29:40.550204+0000","last_fullsized":"2024-05-11T13:29:40.550204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:54:30.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:54.725235+0000","last_change":"2024-05-11T13:28:30.707025+0000","last_active":"2024-05-11T13:29:54.725235+0000","last_peered":"2024-05-11T13:29:54.725235+0000","last_clean":"2024-05-11T13:29:54.725235+0000","last_became_active":"2024-05-11T13:28:30.705853+0000","last_became_peered":"2024-05-11T13:28:30.705853+0000","last_unstale":"2024-05-11T13:29:54.725235+0000","last_undegraded":"2024-05-11T13:29:54.725235+0000","last_fullsized":"2024-05-11T13:29:54.725235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:15:04.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549398+0000","last_change":"2024-05-11T13:28:35.214682+0000","last_active":"2024-05-11T13:29:40.549398+0000","last_peered":"2024-05-11T13:29:40.549398+0000","last_clean":"2024-05-11T13:29:40.549398+0000","last_became_active":"2024-05-11T13:28:35.211581+0000","last_became_peered":"2024-05-11T13:28:35.211581+0000","last_unstale":"2024-05-11T13:29:40.549398+0000","last_undegraded":"2024-05-11T13:29:40.549398+0000","last_fullsized":"2024-05-11T13:29:40.549398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:19.231675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.528919+0000","last_change":"2024-05-11T13:28:49.689763+0000","last_active":"2024-05-11T13:29:40.528919+0000","last_peered":"2024-05-11T13:29:40.528919+0000","last_clean":"2024-05-11T13:29:40.528919+0000","last_became_active":"2024-05-11T13:28:49.689362+0000","last_became_peered":"2024-05-11T13:28:49.689362+0000","last_unstale":"2024-05-11T13:29:40.528919+0000","last_undegraded":"2024-05-11T13:29:40.528919+0000","last_fullsized":"2024-05-11T13:29:40.528919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:07:59.656953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.774103+0000","last_change":"2024-05-11T13:28:30.419444+0000","last_active":"2024-05-11T13:29:57.774103+0000","last_peered":"2024-05-11T13:29:57.774103+0000","last_clean":"2024-05-11T13:29:57.774103+0000","last_became_active":"2024-05-11T13:28:30.419064+0000","last_became_peered":"2024-05-11T13:28:30.419064+0000","last_unstale":"2024-05-11T13:29:57.774103+0000","last_undegraded":"2024-05-11T13:29:57.774103+0000","last_fullsized":"2024-05-11T13:29:57.774103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:02.567403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531572+0000","last_change":"2024-05-11T13:28:35.209347+0000","last_active":"2024-05-11T13:29:40.531572+0000","last_peered":"2024-05-11T13:29:40.531572+0000","last_clean":"2024-05-11T13:29:40.531572+0000","last_became_active":"2024-05-11T13:28:35.208912+0000","last_became_peered":"2024-05-11T13:28:35.208912+0000","last_unstale":"2024-05-11T13:29:40.531572+0000","last_undegraded":"2024-05-11T13:29:40.531572+0000","last_fullsized":"2024-05-11T13:29:40.531572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:08:30.832941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.549961+0000","last_change":"2024-05-11T13:28:49.677398+0000","last_active":"2024-05-11T13:29:40.549961+0000","last_peered":"2024-05-11T13:29:40.549961+0000","last_clean":"2024-05-11T13:29:40.549961+0000","last_became_active":"2024-05-11T13:28:49.676725+0000","last_became_peered":"2024-05-11T13:28:49.676725+0000","last_unstale":"2024-05-11T13:29:40.549961+0000","last_undegraded":"2024-05-11T13:29:40.549961+0000","last_fullsized":"2024-05-11T13:29:40.549961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:32.476049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:56.762062+0000","last_change":"2024-05-11T13:28:30.428782+0000","last_active":"2024-05-11T13:29:56.762062+0000","last_peered":"2024-05-11T13:29:56.762062+0000","last_clean":"2024-05-11T13:29:56.762062+0000","last_became_active":"2024-05-11T13:28:30.427819+0000","last_became_peered":"2024-05-11T13:28:30.427819+0000","last_unstale":"2024-05-11T13:29:56.762062+0000","last_undegraded":"2024-05-11T13:29:56.762062+0000","last_fullsized":"2024-05-11T13:29:56.762062+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:23:24.533765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.547896+0000","last_change":"2024-05-11T13:28:35.210095+0000","last_active":"2024-05-11T13:29:40.547896+0000","last_peered":"2024-05-11T13:29:40.547896+0000","last_clean":"2024-05-11T13:29:40.547896+0000","last_became_active":"2024-05-11T13:28:35.208831+0000","last_became_peered":"2024-05-11T13:28:35.208831+0000","last_unstale":"2024-05-11T13:29:40.547896+0000","last_undegraded":"2024-05-11T13:29:40.547896+0000","last_fullsized":"2024-05-11T13:29:40.547896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:06.957821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548998+0000","last_change":"2024-05-11T13:28:49.724791+0000","last_active":"2024-05-11T13:29:40.548998+0000","last_peered":"2024-05-11T13:29:40.548998+0000","last_clean":"2024-05-11T13:29:40.548998+0000","last_became_active":"2024-05-11T13:28:49.723855+0000","last_became_peered":"2024-05-11T13:28:49.723855+0000","last_unstale":"2024-05-11T13:29:40.548998+0000","last_undegraded":"2024-05-11T13:29:40.548998+0000","last_fullsized":"2024-05-11T13:29:40.548998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:34.705331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:54.728986+0000","last_change":"2024-05-11T13:28:30.431206+0000","last_active":"2024-05-11T13:29:54.728986+0000","last_peered":"2024-05-11T13:29:54.728986+0000","last_clean":"2024-05-11T13:29:54.728986+0000","last_became_active":"2024-05-11T13:28:30.430714+0000","last_became_peered":"2024-05-11T13:28:30.430714+0000","last_unstale":"2024-05-11T13:29:54.728986+0000","last_undegraded":"2024-05-11T13:29:54.728986+0000","last_fullsized":"2024-05-11T13:29:54.728986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:29:21.070046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530860+0000","last_change":"2024-05-11T13:28:35.222254+0000","last_active":"2024-05-11T13:29:40.530860+0000","last_peered":"2024-05-11T13:29:40.530860+0000","last_clean":"2024-05-11T13:29:40.530860+0000","last_became_active":"2024-05-11T13:28:35.219539+0000","last_became_peered":"2024-05-11T13:28:35.219539+0000","last_unstale":"2024-05-11T13:29:40.530860+0000","last_undegraded":"2024-05-11T13:29:40.530860+0000","last_fullsized":"2024-05-11T13:29:40.530860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:55.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.552124+0000","last_change":"2024-05-11T13:28:49.692168+0000","last_active":"2024-05-11T13:29:40.552124+0000","last_peered":"2024-05-11T13:29:40.552124+0000","last_clean":"2024-05-11T13:29:40.552124+0000","last_became_active":"2024-05-11T13:28:49.691730+0000","last_became_peered":"2024-05-11T13:28:49.691730+0000","last_unstale":"2024-05-11T13:29:40.552124+0000","last_undegraded":"2024-05-11T13:29:40.552124+0000","last_fullsized":"2024-05-11T13:29:40.552124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:02.935554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:30:32.895020+0000","last_change":"2024-05-11T13:28:30.712115+0000","last_active":"2024-05-11T13:30:32.895020+0000","last_peered":"2024-05-11T13:30:32.895020+0000","last_clean":"2024-05-11T13:30:32.895020+0000","last_became_active":"2024-05-11T13:28:30.710613+0000","last_became_peered":"2024-05-11T13:28:30.710613+0000","last_unstale":"2024-05-11T13:30:32.895020+0000","last_undegraded":"2024-05-11T13:30:32.895020+0000","last_fullsized":"2024-05-11T13:30:32.895020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:00.521915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531999+0000","last_change":"2024-05-11T13:28:35.221340+0000","last_active":"2024-05-11T13:29:40.531999+0000","last_peered":"2024-05-11T13:29:40.531999+0000","last_clean":"2024-05-11T13:29:40.531999+0000","last_became_active":"2024-05-11T13:28:35.220702+0000","last_became_peered":"2024-05-11T13:28:35.220702+0000","last_unstale":"2024-05-11T13:29:40.531999+0000","last_undegraded":"2024-05-11T13:29:40.531999+0000","last_fullsized":"2024-05-11T13:29:40.531999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:59:37.313209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.843686+0000","last_change":"2024-05-11T13:28:49.682917+0000","last_active":"2024-05-11T13:29:40.843686+0000","last_peered":"2024-05-11T13:29:40.843686+0000","last_clean":"2024-05-11T13:29:40.843686+0000","last_became_active":"2024-05-11T13:28:49.681905+0000","last_became_peered":"2024-05-11T13:28:49.681905+0000","last_unstale":"2024-05-11T13:29:40.843686+0000","last_undegraded":"2024-05-11T13:29:40.843686+0000","last_fullsized":"2024-05-11T13:29:40.843686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:09:26.191565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.773253+0000","last_change":"2024-05-11T13:28:30.437356+0000","last_active":"2024-05-11T13:29:57.773253+0000","last_peered":"2024-05-11T13:29:57.773253+0000","last_clean":"2024-05-11T13:29:57.773253+0000","last_became_active":"2024-05-11T13:28:30.437013+0000","last_became_peered":"2024-05-11T13:28:30.437013+0000","last_unstale":"2024-05-11T13:29:57.773253+0000","last_undegraded":"2024-05-11T13:29:57.773253+0000","last_fullsized":"2024-05-11T13:29:57.773253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:58.756156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.530954+0000","last_change":"2024-05-11T13:28:35.206105+0000","last_active":"2024-05-11T13:29:40.530954+0000","last_peered":"2024-05-11T13:29:40.530954+0000","last_clean":"2024-05-11T13:29:40.530954+0000","last_became_active":"2024-05-11T13:28:35.205684+0000","last_became_peered":"2024-05-11T13:28:35.205684+0000","last_unstale":"2024-05-11T13:29:40.530954+0000","last_undegraded":"2024-05-11T13:29:40.530954+0000","last_fullsized":"2024-05-11T13:29:40.530954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:12.326574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.541581+0000","last_change":"2024-05-11T13:28:49.689545+0000","last_active":"2024-05-11T13:29:40.541581+0000","last_peered":"2024-05-11T13:29:40.541581+0000","last_clean":"2024-05-11T13:29:40.541581+0000","last_became_active":"2024-05-11T13:28:49.688528+0000","last_became_peered":"2024-05-11T13:28:49.688528+0000","last_unstale":"2024-05-11T13:29:40.541581+0000","last_undegraded":"2024-05-11T13:29:40.541581+0000","last_fullsized":"2024-05-11T13:29:40.541581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:35.418873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:57.778143+0000","last_change":"2024-05-11T13:29:40.560835+0000","last_active":"2024-05-11T13:29:57.778143+0000","last_peered":"2024-05-11T13:29:57.778143+0000","last_clean":"2024-05-11T13:29:57.778143+0000","last_became_active":"2024-05-11T13:29:40.554310+0000","last_became_peered":"2024-05-11T13:29:40.554310+0000","last_unstale":"2024-05-11T13:29:57.778143+0000","last_undegraded":"2024-05-11T13:29:57.778143+0000","last_fullsized":"2024-05-11T13:29:57.778143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:19:26.805261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531123+0000","last_change":"2024-05-11T13:28:35.220001+0000","last_active":"2024-05-11T13:29:40.531123+0000","last_peered":"2024-05-11T13:29:40.531123+0000","last_clean":"2024-05-11T13:29:40.531123+0000","last_became_active":"2024-05-11T13:28:35.216768+0000","last_became_peered":"2024-05-11T13:28:35.216768+0000","last_unstale":"2024-05-11T13:29:40.531123+0000","last_undegraded":"2024-05-11T13:29:40.531123+0000","last_fullsized":"2024-05-11T13:29:40.531123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:54:23.547689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.531779+0000","last_change":"2024-05-11T13:28:35.213481+0000","last_active":"2024-05-11T13:29:40.531779+0000","last_peered":"2024-05-11T13:29:40.531779+0000","last_clean":"2024-05-11T13:29:40.531779+0000","last_became_active":"2024-05-11T13:28:35.212956+0000","last_became_peered":"2024-05-11T13:28:35.212956+0000","last_unstale":"2024-05-11T13:29:40.531779+0000","last_undegraded":"2024-05-11T13:29:40.531779+0000","last_fullsized":"2024-05-11T13:29:40.531779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:14.814155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.554654+0000","last_change":"2024-05-11T13:29:40.554654+0000","last_active":"2024-05-11T13:29:40.554654+0000","last_peered":"2024-05-11T13:29:40.554654+0000","last_clean":"2024-05-11T13:29:40.554654+0000","last_became_active":"2024-05-11T13:29:40.551836+0000","last_became_peered":"2024-05-11T13:29:40.551836+0000","last_unstale":"2024-05-11T13:29:40.554654+0000","last_undegraded":"2024-05-11T13:29:40.554654+0000","last_fullsized":"2024-05-11T13:29:40.554654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:49:44.566051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535402+0000","last_change":"2024-05-11T13:28:35.208925+0000","last_active":"2024-05-11T13:29:40.535402+0000","last_peered":"2024-05-11T13:29:40.535402+0000","last_clean":"2024-05-11T13:29:40.535402+0000","last_became_active":"2024-05-11T13:28:35.208055+0000","last_became_peered":"2024-05-11T13:28:35.208055+0000","last_unstale":"2024-05-11T13:29:40.535402+0000","last_undegraded":"2024-05-11T13:29:40.535402+0000","last_fullsized":"2024-05-11T13:29:40.535402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:20.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.844476+0000","last_change":"2024-05-11T13:28:35.213226+0000","last_active":"2024-05-11T13:29:40.844476+0000","last_peered":"2024-05-11T13:29:40.844476+0000","last_clean":"2024-05-11T13:29:40.844476+0000","last_became_active":"2024-05-11T13:28:35.212839+0000","last_became_peered":"2024-05-11T13:28:35.212839+0000","last_unstale":"2024-05-11T13:29:40.844476+0000","last_undegraded":"2024-05-11T13:29:40.844476+0000","last_fullsized":"2024-05-11T13:29:40.844476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:37:05.312505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.848092+0000","last_change":"2024-05-11T13:29:40.848092+0000","last_active":"2024-05-11T13:29:40.848092+0000","last_peered":"2024-05-11T13:29:40.848092+0000","last_clean":"2024-05-11T13:29:40.848092+0000","last_became_active":"2024-05-11T13:29:40.846950+0000","last_became_peered":"2024-05-11T13:29:40.846950+0000","last_unstale":"2024-05-11T13:29:40.848092+0000","last_undegraded":"2024-05-11T13:29:40.848092+0000","last_fullsized":"2024-05-11T13:29:40.848092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:18:06.675351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.533197+0000","last_change":"2024-05-11T13:28:35.217529+0000","last_active":"2024-05-11T13:29:40.533197+0000","last_peered":"2024-05-11T13:29:40.533197+0000","last_clean":"2024-05-11T13:29:40.533197+0000","last_became_active":"2024-05-11T13:28:35.217062+0000","last_became_peered":"2024-05-11T13:28:35.217062+0000","last_unstale":"2024-05-11T13:29:40.533197+0000","last_undegraded":"2024-05-11T13:29:40.533197+0000","last_fullsized":"2024-05-11T13:29:40.533197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:24:33.955794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.538516+0000","last_change":"2024-05-11T13:28:35.217777+0000","last_active":"2024-05-11T13:29:40.538516+0000","last_peered":"2024-05-11T13:29:40.538516+0000","last_clean":"2024-05-11T13:29:40.538516+0000","last_became_active":"2024-05-11T13:28:35.217381+0000","last_became_peered":"2024-05-11T13:28:35.217381+0000","last_unstale":"2024-05-11T13:29:40.538516+0000","last_undegraded":"2024-05-11T13:29:40.538516+0000","last_fullsized":"2024-05-11T13:29:40.538516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:43.973192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.548671+0000","last_change":"2024-05-11T13:28:35.209826+0000","last_active":"2024-05-11T13:29:40.548671+0000","last_peered":"2024-05-11T13:29:40.548671+0000","last_clean":"2024-05-11T13:29:40.548671+0000","last_became_active":"2024-05-11T13:28:35.209461+0000","last_became_peered":"2024-05-11T13:28:35.209461+0000","last_unstale":"2024-05-11T13:29:40.548671+0000","last_undegraded":"2024-05-11T13:29:40.548671+0000","last_fullsized":"2024-05-11T13:29:40.548671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:03.323045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.542844+0000","last_change":"2024-05-11T13:28:35.205252+0000","last_active":"2024-05-11T13:29:40.542844+0000","last_peered":"2024-05-11T13:29:40.542844+0000","last_clean":"2024-05-11T13:29:40.542844+0000","last_became_active":"2024-05-11T13:28:35.204835+0000","last_became_peered":"2024-05-11T13:28:35.204835+0000","last_unstale":"2024-05-11T13:29:40.542844+0000","last_undegraded":"2024-05-11T13:29:40.542844+0000","last_fullsized":"2024-05-11T13:29:40.542844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:10.032439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.546583+0000","last_change":"2024-05-11T13:28:35.219718+0000","last_active":"2024-05-11T13:29:40.546583+0000","last_peered":"2024-05-11T13:29:40.546583+0000","last_clean":"2024-05-11T13:29:40.546583+0000","last_became_active":"2024-05-11T13:28:35.219032+0000","last_became_peered":"2024-05-11T13:28:35.219032+0000","last_unstale":"2024-05-11T13:29:40.546583+0000","last_undegraded":"2024-05-11T13:29:40.546583+0000","last_fullsized":"2024-05-11T13:29:40.546583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:03:58.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.552365+0000","last_change":"2024-05-11T13:28:35.217820+0000","last_active":"2024-05-11T13:29:40.552365+0000","last_peered":"2024-05-11T13:29:40.552365+0000","last_clean":"2024-05-11T13:29:40.552365+0000","last_became_active":"2024-05-11T13:28:35.216260+0000","last_became_peered":"2024-05-11T13:28:35.216260+0000","last_unstale":"2024-05-11T13:29:40.552365+0000","last_undegraded":"2024-05-11T13:29:40.552365+0000","last_fullsized":"2024-05-11T13:29:40.552365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:20:47.188238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.536721+0000","last_change":"2024-05-11T13:28:35.209762+0000","last_active":"2024-05-11T13:29:40.536721+0000","last_peered":"2024-05-11T13:29:40.536721+0000","last_clean":"2024-05-11T13:29:40.536721+0000","last_became_active":"2024-05-11T13:28:35.207404+0000","last_became_peered":"2024-05-11T13:28:35.207404+0000","last_unstale":"2024-05-11T13:29:40.536721+0000","last_undegraded":"2024-05-11T13:29:40.536721+0000","last_fullsized":"2024-05-11T13:29:40.536721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:41:20.808009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-11T13:29:40.535872+0000","last_change":"2024-05-11T13:28:35.215137+0000","last_active":"2024-05-11T13:29:40.535872+0000","last_peered":"2024-05-11T13:29:40.535872+0000","last_clean":"2024-05-11T13:29:40.535872+0000","last_became_active":"2024-05-11T13:28:35.214745+0000","last_became_peered":"2024-05-11T13:28:35.214745+0000","last_unstale":"2024-05-11T13:29:40.535872+0000","last_undegraded":"2024-05-11T13:29:40.535872+0000","last_fullsized":"2024-05-11T13:29:40.535872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:44.381501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69861,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":412,"num_read_kb":285,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":209583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":500505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874289,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1703936,"data_stored":1149765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037817,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":503927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":499605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430279,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1097097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593807,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":495726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":559413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738487,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":496791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":558620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149796,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":495043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":500422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790038,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1091897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T13:30:39.714 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-11T13:30:39.715 INFO:tasks.cephadm:Setting up client nodes... 2024-05-11T13:30:39.715 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-11T13:30:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:40.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:40 smithi165 ceph-mon[33235]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:30:40.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-05-11T13:30:40.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-11T13:30:40.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:40 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-11T13:30:40.367 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:40 smithi063 ceph-mon[29809]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:30:40.367 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-05-11T13:30:40.367 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-11T13:30:40.367 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:40 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-11T13:30:40.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:40 smithi071 ceph-mon[33293]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:30:40.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-05-11T13:30:40.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-11T13:30:40.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:40 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-11T13:30:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:41 smithi165 ceph-mon[33235]: pgmap v438: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-11T13:30:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-05-11T13:30:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-11T13:30:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-11T13:30:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:41 smithi165 ceph-mon[33235]: osdmap e89: 12 total, 12 up, 12 in 2024-05-11T13:30:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:41 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:30:41.348 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:41 smithi071 ceph-mon[33293]: pgmap v438: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-11T13:30:41.348 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-05-11T13:30:41.348 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-11T13:30:41.349 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-11T13:30:41.349 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:41 smithi071 ceph-mon[33293]: osdmap e89: 12 total, 12 up, 12 in 2024-05-11T13:30:41.349 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:41 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:30:41.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:41 smithi063 ceph-mon[29809]: pgmap v438: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-11T13:30:41.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-05-11T13:30:41.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-11T13:30:41.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-11T13:30:41.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:41 smithi063 ceph-mon[29809]: osdmap e89: 12 total, 12 up, 12 in 2024-05-11T13:30:41.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:41 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:30:42.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:42 smithi165 ceph-mon[33235]: osdmap e90: 12 total, 12 up, 12 in 2024-05-11T13:30:42.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:42 smithi063 ceph-mon[29809]: osdmap e90: 12 total, 12 up, 12 in 2024-05-11T13:30:42.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:42 smithi071 ceph-mon[33293]: osdmap e90: 12 total, 12 up, 12 in 2024-05-11T13:30:42.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 -- 172.21.15.63:0/835468620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 msgr2=0x7fdab0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:42.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 --2- 172.21.15.63:0/835468620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 0x7fdab0106e50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0099c0 tx=0x7fda9c009cd0 comp rx=0 tx=0).stop 2024-05-11T13:30:42.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 -- 172.21.15.63:0/835468620 shutdown_connections 2024-05-11T13:30:42.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 --2- 172.21.15.63:0/835468620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab0107390 0x7fdab010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:42.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 --2- 172.21.15.63:0/835468620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab01057f0 0x7fdab0105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 --2- 172.21.15.63:0/835468620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 0x7fdab0106e50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 -- 172.21.15.63:0/835468620 >> 172.21.15.63:0/835468620 conn(0x7fdab0100db0 msgr2=0x7fdab01031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 -- 172.21.15.63:0/835468620 shutdown_connections 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.757+0000 7fdab5a00700 1 -- 172.21.15.63:0/835468620 wait complete. 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 Processor -- start 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 -- start start 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab01057f0 0x7fdab019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 0x7fdab019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0107390 0x7fdab01a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111190 con 0x7fdab01069f0 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111010 con 0x7fdab0107390 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.759+0000 7fdab5a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111310 con 0x7fdab01057f0 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab01057f0 0x7fdab019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:42.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab01057f0 0x7fdab019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:42756/0 (socket says 172.21.15.63:42756) 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 0x7fdab019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 -- 172.21.15.63:0/2779455921 learned_addr learned my addr 172.21.15.63:0/2779455921 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaf400700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0107390 0x7fdab01a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 -- 172.21.15.63:0/2779455921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0107390 msgr2=0x7fdab01a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0107390 0x7fdab01a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 -- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 msgr2=0x7fdab019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 0x7fdab019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.760+0000 7fdaaea00700 1 -- 172.21.15.63:0/2779455921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab01a33d0 con 0x7fdab01057f0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.761+0000 7fdaaea00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab01057f0 0x7fdab019b770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000d9b0 tx=0x7fdaa000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.761+0000 7fdaa7400700 1 -- 172.21.15.63:0/2779455921 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0017030 con 0x7fdab01057f0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.761+0000 7fdaa7400700 1 -- 172.21.15.63:0/2779455921 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa00044b0 con 0x7fdab01057f0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.761+0000 7fdaa7400700 1 -- 172.21.15.63:0/2779455921 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00038e0 con 0x7fdab01057f0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.761+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab01a36c0 con 0x7fdab01057f0 2024-05-11T13:30:42.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.761+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a3bb0 con 0x7fdab01057f0 2024-05-11T13:30:42.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.762+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab0051a90 con 0x7fdab01057f0 2024-05-11T13:30:42.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.764+0000 7fdaa7400700 1 -- 172.21.15.63:0/2779455921 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdaa001a070 con 0x7fdab01057f0 2024-05-11T13:30:42.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.766+0000 7fdaa7400700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fda940b2a30 0x7fda940b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:42.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.766+0000 7fdaa7400700 1 -- 172.21.15.63:0/2779455921 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdaa00e1ff0 con 0x7fdab01057f0 2024-05-11T13:30:42.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.766+0000 7fdaae000700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fda940b2a30 0x7fda940b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:42.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.767+0000 7fdaae000700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fda940b2a30 0x7fda940b4ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda9c000c00 tx=0x7fda9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:42.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:42.772+0000 7fdaa7400700 1 -- 172.21.15.63:0/2779455921 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdaa00a97a0 con 0x7fdab01057f0 2024-05-11T13:30:43.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.132+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fdab002e690 con 0x7fdab01057f0 2024-05-11T13:30:43.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.143+0000 7fdaa7400700 1 -- 172.21.15.63:0/2779455921 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v34) ==== 170+0+59 (secure 0 0 0) 0x7fdaa00a90a0 con 0x7fdab01057f0 2024-05-11T13:30:43.144 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-05-11T13:30:43.144 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQADcz9m+bEOCBAATYIxtept9ZPBjtKrH9ZJug== 2024-05-11T13:30:43.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fda940b2a30 msgr2=0x7fda940b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:43.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fda940b2a30 0x7fda940b4ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda9c000c00 tx=0x7fda9c005fb0 comp rx=0 tx=0).stop 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab01057f0 msgr2=0x7fdab019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab01057f0 0x7fdab019b770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000d9b0 tx=0x7fdaa000dcc0 comp rx=0 tx=0).stop 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 shutdown_connections 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fda940b2a30 0x7fda940b4ef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0107390 0x7fdab01a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdab01057f0 0x7fdab019b770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 --2- 172.21.15.63:0/2779455921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdab01069f0 0x7fdab019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 >> 172.21.15.63:0/2779455921 conn(0x7fdab0100db0 msgr2=0x7fdab0109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 shutdown_connections 2024-05-11T13:30:43.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:43.146+0000 7fdab5a00700 1 -- 172.21.15.63:0/2779455921 wait complete. 2024-05-11T13:30:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:43 smithi165 ceph-mon[33235]: pgmap v441: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-11T13:30:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:43 smithi165 ceph-mon[33235]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-11T13:30:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:43 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:43 smithi063 ceph-mon[29809]: pgmap v441: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-11T13:30:43.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:43 smithi063 ceph-mon[29809]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-11T13:30:43.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.435 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:43 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:43 smithi071 ceph-mon[33293]: pgmap v441: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-11T13:30:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:43 smithi071 ceph-mon[33293]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-11T13:30:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:43 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:44.124 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:30:44.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-11T13:30:44.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-11T13:30:44.231 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-11T13:30:44.231 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-11T13:30:44.231 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-11T13:30:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:44 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2779455921' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T13:30:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:44 smithi165 ceph-mon[33235]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T13:30:44.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:44 smithi165 ceph-mon[33235]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-11T13:30:44.332 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:44 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2779455921' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T13:30:44.332 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:44 smithi063 ceph-mon[29809]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T13:30:44.332 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:44 smithi063 ceph-mon[29809]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-11T13:30:44.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:44 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2779455921' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T13:30:44.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:44 smithi071 ceph-mon[33293]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T13:30:44.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:44 smithi071 ceph-mon[33293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-11T13:30:44.731 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:45.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:45 smithi063 ceph-mon[29809]: pgmap v442: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-11T13:30:45.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:45 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:45.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:45 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:45.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:45 smithi165 ceph-mon[33235]: pgmap v442: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-11T13:30:45.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:45 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:45.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:45 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:45.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:45 smithi071 ceph-mon[33293]: pgmap v442: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-11T13:30:45.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:45 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:45.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:45 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 -- 172.21.15.63:0/1509655214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980106130 msgr2=0x7fb980112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 --2- 172.21.15.63:0/1509655214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980106130 0x7fb980112640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb97800b0a0 tx=0x7fb97800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 -- 172.21.15.63:0/1509655214 shutdown_connections 2024-05-11T13:30:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 --2- 172.21.15.63:0/1509655214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980106130 0x7fb980112640 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 --2- 172.21.15.63:0/1509655214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98010b770 0x7fb98010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 --2- 172.21.15.63:0/1509655214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb980105770 0x7fb980105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 -- 172.21.15.63:0/1509655214 >> 172.21.15.63:0/1509655214 conn(0x7fb980101260 msgr2=0x7fb980103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:47.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.089+0000 7fb985200700 1 -- 172.21.15.63:0/1509655214 shutdown_connections 2024-05-11T13:30:47.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.090+0000 7fb985200700 1 -- 172.21.15.63:0/1509655214 wait complete. 2024-05-11T13:30:47.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.091+0000 7fb985200700 1 Processor -- start 2024-05-11T13:30:47.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.091+0000 7fb985200700 1 -- start start 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb985200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980105770 0x7fb980077a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb985200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb980106130 0x7fb980077fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb985200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98010b770 0x7fb9800784e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb985200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801158b0 con 0x7fb980105770 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb985200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb980115730 con 0x7fb980106130 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb985200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb980115a30 con 0x7fb98010b770 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb97de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980105770 0x7fb980077a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb97de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980105770 0x7fb980077a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37034/0 (socket says 172.21.15.63:37034) 2024-05-11T13:30:47.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb97de00700 1 -- 172.21.15.63:0/4008645875 learned_addr learned my addr 172.21.15.63:0/4008645875 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb97d400700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb980106130 0x7fb980077fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb97e800700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98010b770 0x7fb9800784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb97de00700 1 -- 172.21.15.63:0/4008645875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98010b770 msgr2=0x7fb9800784e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.092+0000 7fb97de00700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98010b770 0x7fb9800784e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb97de00700 1 -- 172.21.15.63:0/4008645875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb980106130 msgr2=0x7fb980077fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb97de00700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb980106130 0x7fb980077fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb97de00700 1 -- 172.21.15.63:0/4008645875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9801ac3f0 con 0x7fb980105770 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb97e800700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98010b770 0x7fb9800784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:30:47.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb97de00700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980105770 0x7fb980077a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d9b0 tx=0x7fb97000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:47.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb976a00700 1 -- 172.21.15.63:0/4008645875 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970017070 con 0x7fb980105770 2024-05-11T13:30:47.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb976a00700 1 -- 172.21.15.63:0/4008645875 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb970004830 con 0x7fb980105770 2024-05-11T13:30:47.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9801ac680 con 0x7fb980105770 2024-05-11T13:30:47.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb976a00700 1 -- 172.21.15.63:0/4008645875 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9700053a0 con 0x7fb980105770 2024-05-11T13:30:47.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.093+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9801acbc0 con 0x7fb980105770 2024-05-11T13:30:47.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.095+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb980106eb0 con 0x7fb980105770 2024-05-11T13:30:47.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.097+0000 7fb976a00700 1 -- 172.21.15.63:0/4008645875 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb9700049e0 con 0x7fb980105770 2024-05-11T13:30:47.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.098+0000 7fb976a00700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb9640b29c0 0x7fb9640b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:47.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.099+0000 7fb976a00700 1 -- 172.21.15.63:0/4008645875 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb9700e2600 con 0x7fb980105770 2024-05-11T13:30:47.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.099+0000 7fb97d400700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb9640b29c0 0x7fb9640b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:47.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.100+0000 7fb97d400700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb9640b29c0 0x7fb9640b4e80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005950 tx=0x7fb96c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:47.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.103+0000 7fb976a00700 1 -- 172.21.15.63:0/4008645875 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb970020020 con 0x7fb980105770 2024-05-11T13:30:47.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:47 smithi063 ceph-mon[29809]: pgmap v443: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-05-11T13:30:47.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:30:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:30:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:47 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:47.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:47 smithi165 ceph-mon[33235]: pgmap v443: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-05-11T13:30:47.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:30:47.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:30:47.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:47 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:47.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.405+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb98002e570 con 0x7fb980105770 2024-05-11T13:30:47.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.413+0000 7fb976a00700 1 -- 172.21.15.63:0/4008645875 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288643 (secure 0 0 0) 0x7fb9700a9db0 con 0x7fb980105770 2024-05-11T13:30:47.418 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:30:47.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb9640b29c0 msgr2=0x7fb9640b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:47.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb9640b29c0 0x7fb9640b4e80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005950 tx=0x7fb96c0058e0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980105770 msgr2=0x7fb980077a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980105770 0x7fb980077a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d9b0 tx=0x7fb97000dd70 comp rx=0 tx=0).stop 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 shutdown_connections 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb9640b29c0 0x7fb9640b4e80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98010b770 0x7fb9800784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb980105770 0x7fb980077a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 --2- 172.21.15.63:0/4008645875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb980106130 0x7fb980077fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 >> 172.21.15.63:0/4008645875 conn(0x7fb980101260 msgr2=0x7fb980101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 shutdown_connections 2024-05-11T13:30:47.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:47.421+0000 7fb985200700 1 -- 172.21.15.63:0/4008645875 wait complete. 2024-05-11T13:30:47.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:47 smithi071 ceph-mon[33293]: pgmap v443: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-05-11T13:30:47.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:47 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:30:47.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:47 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:30:47.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:47 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:30:48.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:48 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4008645875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-11T13:30:48.324 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:48 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4008645875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-11T13:30:48.396 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1742389159},{"type":"v1","addr":"172.21.15.63:6801","nonce":1742389159}]},"active_addr":"172.21.15.63:6801/1742389159","active_change":"2024-05-11T13:18:39.354921+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.63:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":66647058}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":712822368}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":2407940590}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":531186954}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":3969946528}]}]} 2024-05-11T13:30:48.402 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-11T13:30:48.402 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-11T13:30:48.402 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:30:48.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:48 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4008645875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-11T13:30:48.903 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:49 smithi165 ceph-mon[33235]: pgmap v444: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-11T13:30:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:49 smithi165 ceph-mon[33235]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-11T13:30:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:49 smithi165 ceph-mon[33235]: Cluster is now healthy 2024-05-11T13:30:49.404 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:49 smithi063 ceph-mon[29809]: pgmap v444: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-11T13:30:49.404 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:49 smithi063 ceph-mon[29809]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-11T13:30:49.404 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:49 smithi063 ceph-mon[29809]: Cluster is now healthy 2024-05-11T13:30:49.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:49 smithi071 ceph-mon[33293]: pgmap v444: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-11T13:30:49.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:49 smithi071 ceph-mon[33293]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-11T13:30:49.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:49 smithi071 ceph-mon[33293]: Cluster is now healthy 2024-05-11T13:30:51.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:51 smithi063 ceph-mon[29809]: pgmap v445: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 -- 172.21.15.63:0/1522226799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 msgr2=0x7fa850109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 --2- 172.21.15.63:0/1522226799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa850109b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa840009990 tx=0x7fa840009ca0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 -- 172.21.15.63:0/1522226799 shutdown_connections 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 --2- 172.21.15.63:0/1522226799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa850102940 0x7fa850110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 --2- 172.21.15.63:0/1522226799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa850109b10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 --2- 172.21.15.63:0/1522226799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850101f80 0x7fa850102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 -- 172.21.15.63:0/1522226799 >> 172.21.15.63:0/1522226799 conn(0x7fa8500776c0 msgr2=0x7fa850077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 -- 172.21.15.63:0/1522226799 shutdown_connections 2024-05-11T13:30:51.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.302+0000 7fa855a00700 1 -- 172.21.15.63:0/1522226799 wait complete. 2024-05-11T13:30:51.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.304+0000 7fa855a00700 1 Processor -- start 2024-05-11T13:30:51.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.304+0000 7fa855a00700 1 -- start start 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa855a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa850101f80 0x7fa85019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa855a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850102940 0x7fa85019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa855a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa8501a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa855a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501136e0 con 0x7fa850109730 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa855a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa850113560 con 0x7fa850102940 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa855a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa850113860 con 0x7fa850101f80 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa84f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa8501a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa84f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa8501a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44870/0 (socket says 172.21.15.63:44870) 2024-05-11T13:30:51.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa84f400700 1 -- 172.21.15.63:0/3389221514 learned_addr learned my addr 172.21.15.63:0/3389221514 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa84e000700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850102940 0x7fa85019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa84ea00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa850101f80 0x7fa85019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa84f400700 1 -- 172.21.15.63:0/3389221514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa850101f80 msgr2=0x7fa85019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.305+0000 7fa84f400700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa850101f80 0x7fa85019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa84f400700 1 -- 172.21.15.63:0/3389221514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850102940 msgr2=0x7fa85019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa84f400700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850102940 0x7fa85019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa84f400700 1 -- 172.21.15.63:0/3389221514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8501a0f10 con 0x7fa850109730 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa84f400700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa8501a07d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa84800b970 tx=0x7fa84800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa847400700 1 -- 172.21.15.63:0/3389221514 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848019070 con 0x7fa850109730 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa847400700 1 -- 172.21.15.63:0/3389221514 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa84800ce40 con 0x7fa850109730 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa847400700 1 -- 172.21.15.63:0/3389221514 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848007370 con 0x7fa850109730 2024-05-11T13:30:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8501a1200 con 0x7fa850109730 2024-05-11T13:30:51.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.306+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8501a8a80 con 0x7fa850109730 2024-05-11T13:30:51.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.308+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85006a240 con 0x7fa850109730 2024-05-11T13:30:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.310+0000 7fa847400700 1 -- 172.21.15.63:0/3389221514 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa84801c070 con 0x7fa850109730 2024-05-11T13:30:51.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.311+0000 7fa847400700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8340b2990 0x7fa8340b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:51.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.312+0000 7fa847400700 1 -- 172.21.15.63:0/3389221514 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa8480e1c90 con 0x7fa850109730 2024-05-11T13:30:51.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.312+0000 7fa84ea00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8340b2990 0x7fa8340b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:51.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.312+0000 7fa84ea00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8340b2990 0x7fa8340b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa84000b5c0 tx=0x7fa84001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:51.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:51 smithi165 ceph-mon[33235]: pgmap v445: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-11T13:30:51.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.319+0000 7fa847400700 1 -- 172.21.15.63:0/3389221514 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa848051df0 con 0x7fa850109730 2024-05-11T13:30:51.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:51 smithi071 ceph-mon[33293]: pgmap v445: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-11T13:30:51.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.558+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa85002e570 con 0x7fa850109730 2024-05-11T13:30:51.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.563+0000 7fa847400700 1 -- 172.21.15.63:0/3389221514 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24760 (secure 0 0 0) 0x7fa8480a9550 con 0x7fa850109730 2024-05-11T13:30:51.564 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:30:51.565 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":90,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:30:41.007052+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:30:51.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8340b2990 msgr2=0x7fa8340b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:51.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8340b2990 0x7fa8340b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa84000b5c0 tx=0x7fa84001a040 comp rx=0 tx=0).stop 2024-05-11T13:30:51.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 msgr2=0x7fa8501a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:51.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa8501a07d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa84800b970 tx=0x7fa84800bd30 comp rx=0 tx=0).stop 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 shutdown_connections 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa8340b2990 0x7fa8340b4e50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa850109730 0x7fa8501a07d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa850101f80 0x7fa85019bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 --2- 172.21.15.63:0/3389221514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850102940 0x7fa85019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 >> 172.21.15.63:0/3389221514 conn(0x7fa8500776c0 msgr2=0x7fa850100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 shutdown_connections 2024-05-11T13:30:51.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:51.567+0000 7fa855a00700 1 -- 172.21.15.63:0/3389221514 wait complete. 2024-05-11T13:30:52.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:52 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3389221514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:52.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:52 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3389221514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:52.340 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-11T13:30:52.340 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:30:52.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:52 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3389221514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:52.814 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:53.330 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:53 smithi063 ceph-mon[29809]: pgmap v446: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:30:53.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:53 smithi071 ceph-mon[33293]: pgmap v446: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:30:53.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:53 smithi165 ceph-mon[33235]: pgmap v446: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:30:54.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:54 smithi063 ceph-mon[29809]: pgmap v447: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:30:54.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:54 smithi071 ceph-mon[33293]: pgmap v447: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:30:54.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:54 smithi165 ceph-mon[33235]: pgmap v447: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:30:55.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 -- 172.21.15.63:0/4264872799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e00762a0 msgr2=0x7f85e010f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:55.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 --2- 172.21.15.63:0/4264872799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e00762a0 0x7f85e010f770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009990 tx=0x7f85d0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 -- 172.21.15.63:0/4264872799 shutdown_connections 2024-05-11T13:30:55.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 --2- 172.21.15.63:0/4264872799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85e010fcb0 0x7f85e0112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 --2- 172.21.15.63:0/4264872799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85e0075980 0x7f85e0075d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 --2- 172.21.15.63:0/4264872799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e00762a0 0x7f85e010f770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 -- 172.21.15.63:0/4264872799 >> 172.21.15.63:0/4264872799 conn(0x7f85e00fefa0 msgr2=0x7f85e01013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:55.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.042+0000 7f85e6000700 1 -- 172.21.15.63:0/4264872799 shutdown_connections 2024-05-11T13:30:55.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.043+0000 7f85e6000700 1 -- 172.21.15.63:0/4264872799 wait complete. 2024-05-11T13:30:55.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.044+0000 7f85e6000700 1 Processor -- start 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.044+0000 7f85e6000700 1 -- start start 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.044+0000 7f85e6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e0075980 0x7f85e01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e6000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85e00762a0 0x7f85e01a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85e010fcb0 0x7f85e019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e6000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0115710 con 0x7f85e0075980 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e6000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0115590 con 0x7f85e010fcb0 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e6000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0115890 con 0x7f85e00762a0 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e0075980 0x7f85e01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:55.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e0075980 0x7f85e01a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44886/0 (socket says 172.21.15.63:44886) 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 -- 172.21.15.63:0/2308863515 learned_addr learned my addr 172.21.15.63:0/2308863515 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e5600700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85e010fcb0 0x7f85e019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85d7e00700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85e00762a0 0x7f85e01a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 -- 172.21.15.63:0/2308863515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85e00762a0 msgr2=0x7f85e01a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85e00762a0 0x7f85e01a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 -- 172.21.15.63:0/2308863515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85e010fcb0 msgr2=0x7f85e019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85e010fcb0 0x7f85e019bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e4c00700 1 -- 172.21.15.63:0/2308863515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e019c210 con 0x7f85e0075980 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85d7e00700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85e00762a0 0x7f85e01a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:30:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.045+0000 7f85e5600700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85e010fcb0 0x7f85e019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:30:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.046+0000 7f85e4c00700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e0075980 0x7f85e01a1850 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f85c800c9b0 tx=0x7f85c800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.046+0000 7f85de000700 1 -- 172.21.15.63:0/2308863515 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8007b00 con 0x7f85e0075980 2024-05-11T13:30:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.046+0000 7f85de000700 1 -- 172.21.15.63:0/2308863515 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85c8007ca0 con 0x7f85e0075980 2024-05-11T13:30:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.046+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e019c500 con 0x7f85e0075980 2024-05-11T13:30:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.046+0000 7f85de000700 1 -- 172.21.15.63:0/2308863515 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8018660 con 0x7f85e0075980 2024-05-11T13:30:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.046+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e01a8720 con 0x7f85e0075980 2024-05-11T13:30:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.048+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e006a240 con 0x7f85e0075980 2024-05-11T13:30:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.050+0000 7f85de000700 1 -- 172.21.15.63:0/2308863515 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f85c800f450 con 0x7f85e0075980 2024-05-11T13:30:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.052+0000 7f85de000700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f85c00b2990 0x7f85c00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:30:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.052+0000 7f85de000700 1 -- 172.21.15.63:0/2308863515 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f85c80e2a80 con 0x7f85e0075980 2024-05-11T13:30:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.052+0000 7f85d7e00700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f85c00b2990 0x7f85c00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:30:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.053+0000 7f85d7e00700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f85c00b2990 0x7f85c00b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85d000b5c0 tx=0x7f85d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:30:55.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.054+0000 7f85de000700 1 -- 172.21.15.63:0/2308863515 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85c8014030 con 0x7f85e0075980 2024-05-11T13:30:55.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.293+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85e019cf40 con 0x7f85e0075980 2024-05-11T13:30:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.299+0000 7f85de000700 1 -- 172.21.15.63:0/2308863515 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24760 (secure 0 0 0) 0x7f85c80aa340 con 0x7f85e0075980 2024-05-11T13:30:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:30:55.301 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":90,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:30:41.007052+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:30:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f85c00b2990 msgr2=0x7f85c00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f85c00b2990 0x7f85c00b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85d000b5c0 tx=0x7f85d0005fb0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e0075980 msgr2=0x7f85e01a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:30:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e0075980 0x7f85e01a1850 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f85c800c9b0 tx=0x7f85c800cd70 comp rx=0 tx=0).stop 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 shutdown_connections 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f85c00b2990 0x7f85c00b4e50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85e010fcb0 0x7f85e019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e0075980 0x7f85e01a1850 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 --2- 172.21.15.63:0/2308863515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85e00762a0 0x7f85e01a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 >> 172.21.15.63:0/2308863515 conn(0x7f85e00fefa0 msgr2=0x7f85e00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 shutdown_connections 2024-05-11T13:30:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:30:55.303+0000 7f85e6000700 1 -- 172.21.15.63:0/2308863515 wait complete. 2024-05-11T13:30:55.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:55 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2308863515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:55.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:55 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2308863515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:55.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:55 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2308863515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:30:56.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-11T13:30:56.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-11T13:30:56.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-11T13:30:56.285 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-11T13:30:56.285 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-11T13:30:56.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-11T13:30:56.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-11T13:30:56.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-11T13:30:56.287 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-11T13:30:56.287 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-11T13:30:56.288 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-11T13:30:56.288 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-11T13:30:56.444 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:56 smithi063 ceph-mon[29809]: pgmap v448: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:30:56.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:56 smithi165 ceph-mon[33235]: pgmap v448: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:30:56.704 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:56 smithi071 ceph-mon[33293]: pgmap v448: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:30:57.323 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:57.323 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:57.330 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:57.345 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:57.467 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:58.084 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:58.363 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:30:58.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:30:58 smithi063 ceph-mon[29809]: pgmap v449: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-11T13:30:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:30:58 smithi071 ceph-mon[33293]: pgmap v449: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-11T13:30:59.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:30:58 smithi165 ceph-mon[33235]: pgmap v449: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-11T13:31:00.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:00 smithi063 ceph-mon[29809]: pgmap v450: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:00.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:00 smithi071 ceph-mon[33293]: pgmap v450: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:01.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:00 smithi165 ceph-mon[33235]: pgmap v450: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:01.948 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:02.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:02 smithi063 ceph-mon[29809]: pgmap v451: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:02.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:02 smithi071 ceph-mon[33293]: pgmap v451: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:03.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:02 smithi165 ceph-mon[33235]: pgmap v451: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:04.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:04 smithi063 ceph-mon[29809]: pgmap v452: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:04.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:04 smithi071 ceph-mon[33293]: pgmap v452: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:05.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:04 smithi165 ceph-mon[33235]: pgmap v452: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:06.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:06 smithi165 ceph-mon[33235]: pgmap v453: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:06.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:06 smithi071 ceph-mon[33293]: pgmap v453: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:07.084 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:06 smithi063 ceph-mon[29809]: pgmap v453: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:08.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 -- 172.21.15.63:0/1774298361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a4108900 msgr2=0x7f70a4110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 --2- 172.21.15.63:0/1774298361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a4108900 0x7f70a4110480 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0d0 tx=0x7f709c00b3e0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 -- 172.21.15.63:0/1774298361 shutdown_connections 2024-05-11T13:31:08.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 --2- 172.21.15.63:0/1774298361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a4108900 0x7f70a4110480 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 --2- 172.21.15.63:0/1774298361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70a4103800 0x7f70a4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 --2- 172.21.15.63:0/1774298361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a41041b0 0x7f70a41081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 -- 172.21.15.63:0/1774298361 >> 172.21.15.63:0/1774298361 conn(0x7f70a407a6b0 msgr2=0x7f70a407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 -- 172.21.15.63:0/1774298361 shutdown_connections 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.331+0000 7f70ac200700 1 -- 172.21.15.63:0/1774298361 wait complete. 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.332+0000 7f70ac200700 1 Processor -- start 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70ac200700 1 -- start start 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70ac200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70a4103800 0x7f70a41a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70ac200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a41041b0 0x7f70a41a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70ac200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4108900 0x7f70a419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70ac200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41135e0 con 0x7f70a41041b0 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70ac200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4113460 con 0x7f70a4108900 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70ac200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4113760 con 0x7f70a4103800 2024-05-11T13:31:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70a8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a41041b0 0x7f70a41a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70a8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a41041b0 0x7f70a41a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38304/0 (socket says 172.21.15.63:38304) 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70a8e00700 1 -- 172.21.15.63:0/1152356944 learned_addr learned my addr 172.21.15.63:0/1152356944 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.333+0000 7f70a9800700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70a4103800 0x7f70a41a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.334+0000 7f70aa200700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4108900 0x7f70a419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.334+0000 7f70a8e00700 1 -- 172.21.15.63:0/1152356944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70a4103800 msgr2=0x7f70a41a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.334+0000 7f70a8e00700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70a4103800 0x7f70a41a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.334+0000 7f70a8e00700 1 -- 172.21.15.63:0/1152356944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4108900 msgr2=0x7f70a419baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.334+0000 7f70a8e00700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4108900 0x7f70a419baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.334+0000 7f70a8e00700 1 -- 172.21.15.63:0/1152356944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a419c200 con 0x7f70a41041b0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.334+0000 7f70a8e00700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a41041b0 0x7f70a41a1e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f709000d950 tx=0x7f709000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.335+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090016070 con 0x7f70a41041b0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.335+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7090009dd0 con 0x7f70a41041b0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.335+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a419c4f0 con 0x7f70a41041b0 2024-05-11T13:31:08.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.335+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709000f3c0 con 0x7f70a41041b0 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.335+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a41a87b0 con 0x7f70a41041b0 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.336+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70a4076ae0 con 0x7f70a41041b0 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.338+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7090019070 con 0x7f70a41041b0 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.340+0000 7f70a2000700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f708c0b2990 0x7f708c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.340+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f70900e28d0 con 0x7f70a41041b0 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.340+0000 7f70a9800700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f708c0b2990 0x7f708c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.340+0000 7f70a2000700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] conn(0x7f708c0bdc70 0x7f708c0c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.340+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 --> [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f708c0c07e0 con 0x7f708c0bdc70 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.340+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f70900e2c80 con 0x7f70a41041b0 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.341+0000 7f70aa200700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] conn(0x7f708c0bdc70 0x7f708c0c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.341+0000 7f70a9800700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f708c0b2990 0x7f708c0b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f70a419cbe0 tx=0x7f709800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.341+0000 7f70aa200700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] conn(0x7f708c0bdc70 0x7f708c0c0110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.342+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== osd.3 v2:172.21.15.63:6810/206479443 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f708c0c07e0 con 0x7f708c0bdc70 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.364+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 --> [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f70a4104c90 con 0x7f708c0bdc70 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.364+0000 7f70a2000700 1 -- 172.21.15.63:0/1152356944 <== osd.3 v2:172.21.15.63:6810/206479443 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f70a4104c90 con 0x7f708c0bdc70 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] conn(0x7f708c0bdc70 msgr2=0x7f708c0c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] conn(0x7f708c0bdc70 0x7f708c0c0110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f708c0b2990 msgr2=0x7f708c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f708c0b2990 0x7f708c0b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f70a419cbe0 tx=0x7f709800a380 comp rx=0 tx=0).stop 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a41041b0 msgr2=0x7f70a41a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 --2- 172.21.15.63:0/1152356944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a41041b0 0x7f70a41a1e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f709000d950 tx=0x7f709000dc60 comp rx=0 tx=0).stop 2024-05-11T13:31:08.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70aa200700 1 -- 172.21.15.63:0/1152356944 reap_dead start 2024-05-11T13:31:08.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 shutdown_connections 2024-05-11T13:31:08.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 >> 172.21.15.63:0/1152356944 conn(0x7f70a407a6b0 msgr2=0x7f70a4101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:08.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 shutdown_connections 2024-05-11T13:31:08.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.365+0000 7f70ac200700 1 -- 172.21.15.63:0/1152356944 wait complete. 2024-05-11T13:31:08.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:08 smithi063 ceph-mon[29809]: pgmap v454: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:08.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 -- 172.21.15.63:0/823070256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 msgr2=0x7f75c0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 --2- 172.21.15.63:0/823070256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 0x7f75c0103be0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75b00099c0 tx=0x7f75b0009cd0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 -- 172.21.15.63:0/823070256 shutdown_connections 2024-05-11T13:31:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 --2- 172.21.15.63:0/823070256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c0108900 0x7f75c0110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 --2- 172.21.15.63:0/823070256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 0x7f75c0103be0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 --2- 172.21.15.63:0/823070256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75c01041b0 0x7f75c01081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 -- 172.21.15.63:0/823070256 >> 172.21.15.63:0/823070256 conn(0x7f75c007a6b0 msgr2=0x7f75c007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 -- 172.21.15.63:0/823070256 shutdown_connections 2024-05-11T13:31:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.906+0000 7f75c5800700 1 -- 172.21.15.63:0/823070256 wait complete. 2024-05-11T13:31:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.907+0000 7f75c5800700 1 Processor -- start 2024-05-11T13:31:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.907+0000 7f75c5800700 1 -- start start 2024-05-11T13:31:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75c5800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 0x7f75c019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75c5800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c01041b0 0x7f75c019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75c5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75c0108900 0x7f75c01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75c5800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75c01135e0 con 0x7f75c0103800 2024-05-11T13:31:08.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75c5800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75c0113460 con 0x7f75c0108900 2024-05-11T13:31:08.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75c5800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f75c0113760 con 0x7f75c01041b0 2024-05-11T13:31:08.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75bea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 0x7f75c019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75bea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 0x7f75c019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38322/0 (socket says 172.21.15.63:38322) 2024-05-11T13:31:08.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75bea00700 1 -- 172.21.15.63:0/3569912257 learned_addr learned my addr 172.21.15.63:0/3569912257 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:08.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75bf400700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75c0108900 0x7f75c01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.908+0000 7f75b6000700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c01041b0 0x7f75c019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75bea00700 1 -- 172.21.15.63:0/3569912257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c01041b0 msgr2=0x7f75c019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75bea00700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c01041b0 0x7f75c019c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75bea00700 1 -- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75c0108900 msgr2=0x7f75c01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75bea00700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75c0108900 0x7f75c01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75bea00700 1 -- 172.21.15.63:0/3569912257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c01a0da0 con 0x7f75c0103800 2024-05-11T13:31:08.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75bea00700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 0x7f75c019bd90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f75b0000c00 tx=0x7f75b000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:08.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0027070 con 0x7f75c0103800 2024-05-11T13:31:08.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75b000fe50 con 0x7f75c0103800 2024-05-11T13:31:08.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c01a1030 con 0x7f75c0103800 2024-05-11T13:31:08.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.909+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b00173f0 con 0x7f75c0103800 2024-05-11T13:31:08.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.910+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c01a8910 con 0x7f75c0103800 2024-05-11T13:31:08.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.910+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75c010faf0 con 0x7f75c0103800 2024-05-11T13:31:08.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.913+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f75b002a070 con 0x7f75c0103800 2024-05-11T13:31:08.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.915+0000 7f75b7e00700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f75a40b2990 0x7f75a40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.915+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f75b00e3a60 con 0x7f75c0103800 2024-05-11T13:31:08.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.915+0000 7f75b6000700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f75a40b2990 0x7f75a40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.915+0000 7f75b7e00700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] conn(0x7f75a40bdb40 0x7f75a40bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:08.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.915+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 --> [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75b000b0b0 con 0x7f75a40bdb40 2024-05-11T13:31:08.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.915+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f75b00e3e10 con 0x7f75c0103800 2024-05-11T13:31:08.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.916+0000 7f75b6000700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f75a40b2990 0x7f75a40b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75a8005950 tx=0x7f75a80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:08.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.927+0000 7f75bf400700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] conn(0x7f75a40bdb40 0x7f75a40bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:08.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.928+0000 7f75bf400700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] conn(0x7f75a40bdb40 0x7f75a40bffe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:08.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.931+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== osd.4 v2:172.21.15.71:6808/2625617400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75b000b0b0 con 0x7f75a40bdb40 2024-05-11T13:31:08.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.949+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 --> [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75c0104c90 con 0x7f75a40bdb40 2024-05-11T13:31:08.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.950+0000 7f75b7e00700 1 -- 172.21.15.63:0/3569912257 <== osd.4 v2:172.21.15.71:6808/2625617400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f75c0104c90 con 0x7f75a40bdb40 2024-05-11T13:31:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] conn(0x7f75a40bdb40 msgr2=0x7f75a40bffe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] conn(0x7f75a40bdb40 0x7f75a40bffe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f75a40b2990 msgr2=0x7f75a40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f75a40b2990 0x7f75a40b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75a8005950 tx=0x7f75a80058e0 comp rx=0 tx=0).stop 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 msgr2=0x7f75c019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 --2- 172.21.15.63:0/3569912257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75c0103800 0x7f75c019bd90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f75b0000c00 tx=0x7f75b000f510 comp rx=0 tx=0).stop 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75bf400700 1 -- 172.21.15.63:0/3569912257 reap_dead start 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 shutdown_connections 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.951+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 >> 172.21.15.63:0/3569912257 conn(0x7f75c007a6b0 msgr2=0x7f75c0101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.952+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 shutdown_connections 2024-05-11T13:31:08.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:08.952+0000 7f75c5800700 1 -- 172.21.15.63:0/3569912257 wait complete. 2024-05-11T13:31:08.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:08 smithi071 ceph-mon[33293]: pgmap v454: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:09.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:08 smithi165 ceph-mon[33235]: pgmap v454: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:11.041 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:10 smithi063 ceph-mon[29809]: pgmap v455: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:11.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:10 smithi165 ceph-mon[33235]: pgmap v455: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:11.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:10 smithi071 ceph-mon[33293]: pgmap v455: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:11.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 -- 172.21.15.63:0/1790917710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c075780 msgr2=0x7f994c075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:11.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 --2- 172.21.15.63:0/1790917710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c075780 0x7f994c075b60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f993c009990 tx=0x7f993c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 -- 172.21.15.63:0/1790917710 shutdown_connections 2024-05-11T13:31:11.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 --2- 172.21.15.63:0/1790917710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f994c114720 0x7f994c116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 --2- 172.21.15.63:0/1790917710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c075780 0x7f994c075b60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 --2- 172.21.15.63:0/1790917710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f994c0760a0 0x7f994c1141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 -- 172.21.15.63:0/1790917710 >> 172.21.15.63:0/1790917710 conn(0x7f994c07a6b0 msgr2=0x7f994c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:11.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 -- 172.21.15.63:0/1790917710 shutdown_connections 2024-05-11T13:31:11.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.729+0000 7f9952800700 1 -- 172.21.15.63:0/1790917710 wait complete. 2024-05-11T13:31:11.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 Processor -- start 2024-05-11T13:31:11.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 -- start start 2024-05-11T13:31:11.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f994c075780 0x7f994c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:11.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f994c0760a0 0x7f994c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:11.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c114720 0x7f994c1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:11.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f994c119be0 con 0x7f994c114720 2024-05-11T13:31:11.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f994c119a60 con 0x7f994c075780 2024-05-11T13:31:11.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.731+0000 7f9952800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f994c119d60 con 0x7f994c0760a0 2024-05-11T13:31:11.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.732+0000 7f994be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c114720 0x7f994c1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:11.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.732+0000 7f994be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c114720 0x7f994c1a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35096/0 (socket says 172.21.15.63:35096) 2024-05-11T13:31:11.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.732+0000 7f994be00700 1 -- 172.21.15.63:0/3860574023 learned_addr learned my addr 172.21.15.63:0/3860574023 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:11.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.732+0000 7f994b400700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f994c075780 0x7f994c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:11.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.732+0000 7f9943e00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f994c0760a0 0x7f994c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:11.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f994be00700 1 -- 172.21.15.63:0/3860574023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f994c0760a0 msgr2=0x7f994c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f994be00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f994c0760a0 0x7f994c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f994be00700 1 -- 172.21.15.63:0/3860574023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f994c075780 msgr2=0x7f994c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f994be00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f994c075780 0x7f994c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f994be00700 1 -- 172.21.15.63:0/3860574023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1a51a0 con 0x7f994c114720 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f994be00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c114720 0x7f994c1a4a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f99440094f0 tx=0x7f994400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994400ddd0 con 0x7f994c114720 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9944009e20 con 0x7f994c114720 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1a5490 con 0x7f994c114720 2024-05-11T13:31:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.733+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944015400 con 0x7f994c114720 2024-05-11T13:31:11.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.734+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1acd10 con 0x7f994c114720 2024-05-11T13:31:11.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.735+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f994c113f30 con 0x7f994c114720 2024-05-11T13:31:11.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.737+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9944004dd0 con 0x7f994c114720 2024-05-11T13:31:11.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.739+0000 7f9948c00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99300b2910 0x7f99300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:11.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.739+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f99440e2d90 con 0x7f994c114720 2024-05-11T13:31:11.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.739+0000 7f994b400700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99300b2910 0x7f99300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:11.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.740+0000 7f9948c00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] conn(0x7f99300bdbf0 0x7f99300c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:11.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.740+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 --> [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f994400be50 con 0x7f99300bdbf0 2024-05-11T13:31:11.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.740+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f99440e3140 con 0x7f994c114720 2024-05-11T13:31:11.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.740+0000 7f9943e00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] conn(0x7f99300bdbf0 0x7f99300c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:11.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.740+0000 7f994b400700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99300b2910 0x7f99300b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f993c00b5c0 tx=0x7f993c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:11.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.741+0000 7f9943e00700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] conn(0x7f99300bdbf0 0x7f99300c0090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:11.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.742+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== osd.9 v2:172.21.15.63:6826/1803317888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f994400be50 con 0x7f99300bdbf0 2024-05-11T13:31:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.761+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 --> [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f994c076e70 con 0x7f99300bdbf0 2024-05-11T13:31:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.762+0000 7f9948c00700 1 -- 172.21.15.63:0/3860574023 <== osd.9 v2:172.21.15.63:6826/1803317888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f994c076e70 con 0x7f99300bdbf0 2024-05-11T13:31:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.762+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] conn(0x7f99300bdbf0 msgr2=0x7f99300c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.762+0000 7f9952800700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] conn(0x7f99300bdbf0 0x7f99300c0090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99300b2910 msgr2=0x7f99300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99300b2910 0x7f99300b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f993c00b5c0 tx=0x7f993c005960 comp rx=0 tx=0).stop 2024-05-11T13:31:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c114720 msgr2=0x7f994c1a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 --2- 172.21.15.63:0/3860574023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f994c114720 0x7f994c1a4a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f99440094f0 tx=0x7f994400d3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:11.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f994be00700 1 -- 172.21.15.63:0/3860574023 reap_dead start 2024-05-11T13:31:11.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 shutdown_connections 2024-05-11T13:31:11.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 >> 172.21.15.63:0/3860574023 conn(0x7f994c07a6b0 msgr2=0x7f994c113550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:11.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 shutdown_connections 2024-05-11T13:31:11.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:11.763+0000 7f9952800700 1 -- 172.21.15.63:0/3860574023 wait complete. 2024-05-11T13:31:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 -- 172.21.15.63:0/2957670886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c1095c0 msgr2=0x7f3d9c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 --2- 172.21.15.63:0/2957670886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c1095c0 0x7f3d9c1099a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009990 tx=0x7f3d90009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 -- 172.21.15.63:0/2957670886 shutdown_connections 2024-05-11T13:31:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 --2- 172.21.15.63:0/2957670886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c06bbf0 0x7f3d9c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 --2- 172.21.15.63:0/2957670886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c1095c0 0x7f3d9c1099a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 --2- 172.21.15.63:0/2957670886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9c06b230 0x7f3d9c06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 -- 172.21.15.63:0/2957670886 >> 172.21.15.63:0/2957670886 conn(0x7f3d9c079400 msgr2=0x7f3d9c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 -- 172.21.15.63:0/2957670886 shutdown_connections 2024-05-11T13:31:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.729+0000 7f3da3c00700 1 -- 172.21.15.63:0/2957670886 wait complete. 2024-05-11T13:31:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.730+0000 7f3da3c00700 1 Processor -- start 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.730+0000 7f3da3c00700 1 -- start start 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da3c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c06b230 0x7f3d9c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da3c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9c06bbf0 0x7f3d9c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da3c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c1095c0 0x7f3d9c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da3c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c1135f0 con 0x7f3d9c06b230 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da3c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c113470 con 0x7f3d9c1095c0 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da3c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c113770 con 0x7f3d9c06bbf0 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da1200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c06b230 0x7f3d9c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da1200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c06b230 0x7f3d9c19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35100/0 (socket says 172.21.15.63:35100) 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.731+0000 7f3da1200700 1 -- 172.21.15.63:0/3689498912 learned_addr learned my addr 172.21.15.63:0/3689498912 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3da1200700 1 -- 172.21.15.63:0/3689498912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9c06bbf0 msgr2=0x7f3d9c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3da1200700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9c06bbf0 0x7f3d9c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3da1200700 1 -- 172.21.15.63:0/3689498912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c1095c0 msgr2=0x7f3d9c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3da1200700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c1095c0 0x7f3d9c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3da1200700 1 -- 172.21.15.63:0/3689498912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c1a0d50 con 0x7f3d9c06b230 2024-05-11T13:31:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3da1200700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c06b230 0x7f3d9c19bd40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d90000c00 tx=0x7f3d90005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90027070 con 0x7f3d9c06b230 2024-05-11T13:31:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d9000fb80 con 0x7f3d9c06b230 2024-05-11T13:31:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.732+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90022c30 con 0x7f3d9c06b230 2024-05-11T13:31:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.733+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c1a0fe0 con 0x7f3d9c06b230 2024-05-11T13:31:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.733+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c1a88c0 con 0x7f3d9c06b230 2024-05-11T13:31:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.734+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d9c10f190 con 0x7f3d9c06b230 2024-05-11T13:31:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.736+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3d9002a070 con 0x7f3d9c06b230 2024-05-11T13:31:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.738+0000 7f3d99600700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3d840b2910 0x7f3d840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.738+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3d900e3bc0 con 0x7f3d9c06b230 2024-05-11T13:31:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.738+0000 7f3d9be00700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3d840b2910 0x7f3d840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.738+0000 7f3d99600700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] conn(0x7f3d840bdbf0 0x7f3d840c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.738+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 --> [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d90007e80 con 0x7f3d840bdbf0 2024-05-11T13:31:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.738+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3d900e3f70 con 0x7f3d9c06b230 2024-05-11T13:31:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.739+0000 7f3d9be00700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3d840b2910 0x7f3d840b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005950 tx=0x7f3d880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.739+0000 7f3da1c00700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] conn(0x7f3d840bdbf0 0x7f3d840c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:12.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.740+0000 7f3da1c00700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] conn(0x7f3d840bdbf0 0x7f3d840c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:12.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.742+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== osd.11 v2:172.21.15.165:6824/2607132784 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d90007e80 con 0x7f3d840bdbf0 2024-05-11T13:31:12.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.760+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 --> [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d9c104c80 con 0x7f3d840bdbf0 2024-05-11T13:31:12.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.761+0000 7f3d99600700 1 -- 172.21.15.63:0/3689498912 <== osd.11 v2:172.21.15.165:6824/2607132784 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d9c104c80 con 0x7f3d840bdbf0 2024-05-11T13:31:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 >> [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] conn(0x7f3d840bdbf0 msgr2=0x7f3d840c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] conn(0x7f3d840bdbf0 0x7f3d840c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3d840b2910 msgr2=0x7f3d840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3d840b2910 0x7f3d840b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005950 tx=0x7f3d880058e0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c06b230 msgr2=0x7f3d9c19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 --2- 172.21.15.63:0/3689498912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c06b230 0x7f3d9c19bd40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d90000c00 tx=0x7f3d90005ef0 comp rx=0 tx=0).stop 2024-05-11T13:31:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da1c00700 1 -- 172.21.15.63:0/3689498912 reap_dead start 2024-05-11T13:31:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 shutdown_connections 2024-05-11T13:31:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.762+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 >> 172.21.15.63:0/3689498912 conn(0x7f3d9c079400 msgr2=0x7f3d9c10e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.763+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 shutdown_connections 2024-05-11T13:31:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:12.763+0000 7f3da3c00700 1 -- 172.21.15.63:0/3689498912 wait complete. 2024-05-11T13:31:13.016 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:12 smithi063 ceph-mon[29809]: pgmap v456: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:13.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:12 smithi165 ceph-mon[33235]: pgmap v456: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:13.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:12 smithi071 ceph-mon[33293]: pgmap v456: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:13.615 INFO:teuthology.orchestra.run.smithi063.stdout:111669149803 2024-05-11T13:31:13.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-11T13:31:14.980 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:14 smithi063 ceph-mon[29809]: pgmap v457: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:31:14.984 INFO:teuthology.orchestra.run.smithi063.stdout:304942678067 2024-05-11T13:31:14.984 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-11T13:31:15.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:14 smithi165 ceph-mon[33235]: pgmap v457: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:31:15.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:14 smithi071 ceph-mon[33293]: pgmap v457: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 -- 172.21.15.63:0/1818721561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06bbf0 msgr2=0x7f0ccc1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 --2- 172.21.15.63:0/1818721561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06bbf0 0x7f0ccc1103e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400b0a0 tx=0x7f0cc400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 -- 172.21.15.63:0/1818721561 shutdown_connections 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 --2- 172.21.15.63:0/1818721561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06bbf0 0x7f0ccc1103e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 --2- 172.21.15.63:0/1818721561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ccc1095c0 0x7f0ccc1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 --2- 172.21.15.63:0/1818721561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ccc06b230 0x7f0ccc06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 -- 172.21.15.63:0/1818721561 >> 172.21.15.63:0/1818721561 conn(0x7f0ccc079400 msgr2=0x7f0ccc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 -- 172.21.15.63:0/1818721561 shutdown_connections 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.613+0000 7f0cd1200700 1 -- 172.21.15.63:0/1818721561 wait complete. 2024-05-11T13:31:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.614+0000 7f0cd1200700 1 Processor -- start 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.614+0000 7f0cd1200700 1 -- start start 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.614+0000 7f0cd1200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06b230 0x7f0ccc102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cd1200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ccc06bbf0 0x7f0ccc102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cd1200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ccc1095c0 0x7f0ccc102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cd1200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc113580 con 0x7f0ccc06b230 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cd1200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc113400 con 0x7f0ccc1095c0 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cd1200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc113700 con 0x7f0ccc06bbf0 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06b230 0x7f0ccc102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06b230 0x7f0ccc102520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35148/0 (socket says 172.21.15.63:35148) 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 -- 172.21.15.63:0/1178596683 learned_addr learned my addr 172.21.15.63:0/1178596683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 -- 172.21.15.63:0/1178596683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ccc06bbf0 msgr2=0x7f0ccc102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9400700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ccc06bbf0 0x7f0ccc102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cca800700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ccc1095c0 0x7f0ccc102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ccc06bbf0 0x7f0ccc102a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 -- 172.21.15.63:0/1178596683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ccc1095c0 msgr2=0x7f0ccc102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ccc1095c0 0x7f0ccc102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9e00700 1 -- 172.21.15.63:0/1178596683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc1a8000 con 0x7f0ccc06b230 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cca800700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ccc1095c0 0x7f0ccc102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.615+0000 7f0cc9400700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ccc06bbf0 0x7f0ccc102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.616+0000 7f0cc9e00700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06b230 0x7f0ccc102520 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00d950 tx=0x7f0cbc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.616+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc016070 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.616+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cbc010460 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.616+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc1a8290 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.616+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00f620 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.616+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a87d0 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.617+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ccc063c60 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.620+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0cbc019070 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.622+0000 7f0cc2a00700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0cb00b2910 0x7f0cb00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.622+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0cbc0e2840 con 0x7f0ccc06b230 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.622+0000 7f0cc9400700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0cb00b2910 0x7f0cb00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.622+0000 7f0cc2a00700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] conn(0x7f0cb00bdbf0 0x7f0cb00c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.622+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 --> [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cb00c0760 con 0x7f0cb00bdbf0 2024-05-11T13:31:15.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.622+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0cbc0e2bf0 con 0x7f0ccc06b230 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.623+0000 7f0cca800700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] conn(0x7f0cb00bdbf0 0x7f0cb00c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.623+0000 7f0cc9400700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0cb00b2910 0x7f0cb00b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8005fd0 tx=0x7f0cb8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.623+0000 7f0cca800700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] conn(0x7f0cb00bdbf0 0x7f0cb00c0090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.624+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== osd.6 v2:172.21.15.63:6818/1008047147 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cb00c0760 con 0x7f0cb00bdbf0 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.644+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 --> [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ccc0640e0 con 0x7f0cb00bdbf0 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.645+0000 7f0cc2a00700 1 -- 172.21.15.63:0/1178596683 <== osd.6 v2:172.21.15.63:6818/1008047147 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0ccc0640e0 con 0x7f0cb00bdbf0 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.645+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] conn(0x7f0cb00bdbf0 msgr2=0x7f0cb00c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.645+0000 7f0cd1200700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] conn(0x7f0cb00bdbf0 0x7f0cb00c0090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.645+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0cb00b2910 msgr2=0x7f0cb00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.645+0000 7f0cd1200700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f0cb00b2910 0x7f0cb00b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8005fd0 tx=0x7f0cb8005ee0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.645+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06b230 msgr2=0x7f0ccc102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.645+0000 7f0cd1200700 1 --2- 172.21.15.63:0/1178596683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc06b230 0x7f0ccc102520 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00d950 tx=0x7f0cbc00dd10 comp rx=0 tx=0).stop 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.646+0000 7f0cca800700 1 -- 172.21.15.63:0/1178596683 reap_dead start 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.646+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 shutdown_connections 2024-05-11T13:31:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.646+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 >> 172.21.15.63:0/1178596683 conn(0x7f0ccc079400 msgr2=0x7f0ccc101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:15.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.646+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 shutdown_connections 2024-05-11T13:31:15.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.646+0000 7f0cd1200700 1 -- 172.21.15.63:0/1178596683 wait complete. 2024-05-11T13:31:15.756 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 -- 172.21.15.63:0/2635412897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d8108900 msgr2=0x7f99d8110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 --2- 172.21.15.63:0/2635412897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d8108900 0x7f99d8110480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f99d000b0d0 tx=0x7f99d000b3e0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 -- 172.21.15.63:0/2635412897 shutdown_connections 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 --2- 172.21.15.63:0/2635412897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d8108900 0x7f99d8110480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 --2- 172.21.15.63:0/2635412897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d8103800 0x7f99d8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 --2- 172.21.15.63:0/2635412897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d81041b0 0x7f99d81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 -- 172.21.15.63:0/2635412897 >> 172.21.15.63:0/2635412897 conn(0x7f99d807a6b0 msgr2=0x7f99d807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 -- 172.21.15.63:0/2635412897 shutdown_connections 2024-05-11T13:31:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.798+0000 7f99e0600700 1 -- 172.21.15.63:0/2635412897 wait complete. 2024-05-11T13:31:15.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.799+0000 7f99e0600700 1 Processor -- start 2024-05-11T13:31:15.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.800+0000 7f99e0600700 1 -- start start 2024-05-11T13:31:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.800+0000 7f99e0600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d8103800 0x7f99d810f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.800+0000 7f99e0600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d81041b0 0x7f99d810fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.800+0000 7f99e0600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d8108900 0x7f99d81a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.800+0000 7f99e0600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8113600 con 0x7f99d81041b0 2024-05-11T13:31:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.800+0000 7f99e0600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8113480 con 0x7f99d8103800 2024-05-11T13:31:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.800+0000 7f99e0600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8113780 con 0x7f99d8108900 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.801+0000 7f99ddc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d8103800 0x7f99d810f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.801+0000 7f99ddc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d8103800 0x7f99d810f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:59214/0 (socket says 172.21.15.63:59214) 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.801+0000 7f99ddc00700 1 -- 172.21.15.63:0/2431401663 learned_addr learned my addr 172.21.15.63:0/2431401663 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.801+0000 7f99de600700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d8108900 0x7f99d81a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.802+0000 7f99ddc00700 1 -- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d8108900 msgr2=0x7f99d81a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.802+0000 7f99ddc00700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d8108900 0x7f99d81a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.802+0000 7f99ddc00700 1 -- 172.21.15.63:0/2431401663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d81041b0 msgr2=0x7f99d810fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.802+0000 7f99ddc00700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d81041b0 0x7f99d810fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.802+0000 7f99ddc00700 1 -- 172.21.15.63:0/2431401663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d81a50c0 con 0x7f99d8103800 2024-05-11T13:31:15.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.802+0000 7f99ddc00700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d8103800 0x7f99d810f860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d980 tx=0x7f99cc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.803+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc017070 con 0x7f99d8103800 2024-05-11T13:31:15.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.803+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99cc005020 con 0x7f99d8103800 2024-05-11T13:31:15.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.803+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d81a53b0 con 0x7f99d8103800 2024-05-11T13:31:15.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.803+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc009ab0 con 0x7f99d8103800 2024-05-11T13:31:15.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.803+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d81acc60 con 0x7f99d8103800 2024-05-11T13:31:15.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.804+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99d81021e0 con 0x7f99d8103800 2024-05-11T13:31:15.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.809+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f99cc0051d0 con 0x7f99d8103800 2024-05-11T13:31:15.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.810+0000 7f99d6000700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99c00b2910 0x7f99c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.811+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f99cc0e2c20 con 0x7f99d8103800 2024-05-11T13:31:15.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.811+0000 7f99dd200700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99c00b2910 0x7f99c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.811+0000 7f99d6000700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] conn(0x7f99c00bdbf0 0x7f99c00c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.811+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 --> [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99c00c0760 con 0x7f99c00bdbf0 2024-05-11T13:31:15.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.811+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f99cc0e2fd0 con 0x7f99d8103800 2024-05-11T13:31:15.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.812+0000 7f99de600700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] conn(0x7f99c00bdbf0 0x7f99c00c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.812+0000 7f99dd200700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99c00b2910 0x7f99c00b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99c4005950 tx=0x7f99c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.812+0000 7f99de600700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] conn(0x7f99c00bdbf0 0x7f99c00c0090 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.814+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== osd.8 v2:172.21.15.165:6816/4105128904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99c00c0760 con 0x7f99c00bdbf0 2024-05-11T13:31:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.833+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 --> [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99d8104c90 con 0x7f99c00bdbf0 2024-05-11T13:31:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.834+0000 7f99d6000700 1 -- 172.21.15.63:0/2431401663 <== osd.8 v2:172.21.15.165:6816/4105128904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f99d8104c90 con 0x7f99c00bdbf0 2024-05-11T13:31:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.174+0000 7f94e0e00700 1 -- 172.21.15.63:0/352862488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 msgr2=0x7f94dc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.174+0000 7f94e0e00700 1 --2- 172.21.15.63:0/352862488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 0x7f94dc10bb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009990 tx=0x7f94cc009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.174+0000 7f94e0e00700 1 -- 172.21.15.63:0/352862488 shutdown_connections 2024-05-11T13:31:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.174+0000 7f94e0e00700 1 --2- 172.21.15.63:0/352862488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc106130 0x7f94dc112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.174+0000 7f94e0e00700 1 --2- 172.21.15.63:0/352862488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 0x7f94dc10bb50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.174+0000 7f94e0e00700 1 --2- 172.21.15.63:0/352862488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94dc105770 0x7f94dc105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.174+0000 7f94e0e00700 1 -- 172.21.15.63:0/352862488 >> 172.21.15.63:0/352862488 conn(0x7f94dc101260 msgr2=0x7f94dc103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.175+0000 7f94e0e00700 1 -- 172.21.15.63:0/352862488 shutdown_connections 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.175+0000 7f94e0e00700 1 -- 172.21.15.63:0/352862488 wait complete. 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.176+0000 7f94e0e00700 1 Processor -- start 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.176+0000 7f94e0e00700 1 -- start start 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94e0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc105770 0x7f94dc19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94e0e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94dc106130 0x7f94dc19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94e0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 0x7f94dc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94e0e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1158b0 con 0x7f94dc10b770 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94e0e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc115730 con 0x7f94dc105770 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94e0e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc115a30 con 0x7f94dc106130 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 0x7f94dc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 0x7f94dc1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35122/0 (socket says 172.21.15.63:35122) 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94da800700 1 -- 172.21.15.63:0/1934984066 learned_addr learned my addr 172.21.15.63:0/1934984066 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.177+0000 7f94d9e00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc105770 0x7f94dc19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.178+0000 7f94d9400700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94dc106130 0x7f94dc19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.178+0000 7f94da800700 1 -- 172.21.15.63:0/1934984066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94dc106130 msgr2=0x7f94dc19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.178+0000 7f94da800700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94dc106130 0x7f94dc19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.178+0000 7f94da800700 1 -- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc105770 msgr2=0x7f94dc19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.178+0000 7f94da800700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc105770 0x7f94dc19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.178+0000 7f94da800700 1 -- 172.21.15.63:0/1934984066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc1a0dd0 con 0x7f94dc10b770 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.178+0000 7f94da800700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 0x7f94dc1a0690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f94d400ada0 tx=0x7f94d400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.179+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4017020 con 0x7f94dc10b770 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.179+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc1a10c0 con 0x7f94dc10b770 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.179+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94d400cd60 con 0x7f94dc10b770 2024-05-11T13:31:15.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.179+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4005220 con 0x7f94dc10b770 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.179+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1a8940 con 0x7f94dc10b770 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.180+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94dc063ce0 con 0x7f94dc10b770 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.182+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f94d400c8d0 con 0x7f94dc10b770 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.184+0000 7f94d2a00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f94c00b2910 0x7f94c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.184+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f94d40e2830 con 0x7f94dc10b770 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.184+0000 7f94d9e00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f94c00b2910 0x7f94c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.184+0000 7f94d2a00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] conn(0x7f94c00bdbf0 0x7f94c00c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.184+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 --> [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94c00c0760 con 0x7f94c00bdbf0 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.184+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f94d40e2c80 con 0x7f94dc10b770 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.185+0000 7f94d9e00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f94c00b2910 0x7f94c00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94cc003760 tx=0x7f94cc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.185+0000 7f94d9400700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] conn(0x7f94c00bdbf0 0x7f94c00c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.186+0000 7f94d9400700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] conn(0x7f94c00bdbf0 0x7f94c00c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.188+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== osd.1 v2:172.21.15.71:6800/969700086 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94c00c0760 con 0x7f94c00bdbf0 2024-05-11T13:31:15.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.209+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 --> [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94dc064110 con 0x7f94c00bdbf0 2024-05-11T13:31:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.210+0000 7f94d2a00700 1 -- 172.21.15.63:0/1934984066 <== osd.1 v2:172.21.15.71:6800/969700086 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f94dc064110 con 0x7f94c00bdbf0 2024-05-11T13:31:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.210+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] conn(0x7f94c00bdbf0 msgr2=0x7f94c00c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.210+0000 7f94e0e00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] conn(0x7f94c00bdbf0 0x7f94c00c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f94c00b2910 msgr2=0x7f94c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f94c00b2910 0x7f94c00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94cc003760 tx=0x7f94cc00b540 comp rx=0 tx=0).stop 2024-05-11T13:31:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 msgr2=0x7f94dc1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 --2- 172.21.15.63:0/1934984066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94dc10b770 0x7f94dc1a0690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f94d400ada0 tx=0x7f94d400c330 comp rx=0 tx=0).stop 2024-05-11T13:31:15.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94da800700 1 -- 172.21.15.63:0/1934984066 reap_dead start 2024-05-11T13:31:15.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 shutdown_connections 2024-05-11T13:31:15.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 >> 172.21.15.63:0/1934984066 conn(0x7f94dc101260 msgr2=0x7f94dc1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:15.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 shutdown_connections 2024-05-11T13:31:15.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:14.211+0000 7f94e0e00700 1 -- 172.21.15.63:0/1934984066 wait complete. 2024-05-11T13:31:15.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] conn(0x7f99c00bdbf0 msgr2=0x7f99c00c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] conn(0x7f99c00bdbf0 0x7f99c00c0090 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99c00b2910 msgr2=0x7f99c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f99c00b2910 0x7f99c00b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99c4005950 tx=0x7f99c40058e0 comp rx=0 tx=0).stop 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d8103800 msgr2=0x7f99d810f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 --2- 172.21.15.63:0/2431401663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d8103800 0x7f99d810f860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d980 tx=0x7f99cc00dc90 comp rx=0 tx=0).stop 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99de600700 1 -- 172.21.15.63:0/2431401663 reap_dead start 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 shutdown_connections 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 >> 172.21.15.63:0/2431401663 conn(0x7f99d807a6b0 msgr2=0x7f99d8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.835+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 shutdown_connections 2024-05-11T13:31:15.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:15.836+0000 7f99e0600700 1 -- 172.21.15.63:0/2431401663 wait complete. 2024-05-11T13:31:16.761 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:16 smithi063 ceph-mon[29809]: pgmap v458: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:31:17.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:16 smithi165 ceph-mon[33235]: pgmap v458: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:31:17.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:16 smithi071 ceph-mon[33293]: pgmap v458: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-11T13:31:17.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.007+0000 7f8ede400700 1 -- 172.21.15.63:0/3836699610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 msgr2=0x7f8ed810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.007+0000 7f8ede400700 1 --2- 172.21.15.63:0/3836699610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 0x7f8ed810e7e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099c0 tx=0x7f8ec4009cd0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.007+0000 7f8ede400700 1 -- 172.21.15.63:0/3836699610 shutdown_connections 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.007+0000 7f8ede400700 1 --2- 172.21.15.63:0/3836699610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ed810eec0 0x7f8ed8116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.007+0000 7f8ede400700 1 --2- 172.21.15.63:0/3836699610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ed8109de0 0x7f8ed810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.007+0000 7f8ede400700 1 --2- 172.21.15.63:0/3836699610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 0x7f8ed810e7e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.007+0000 7f8ede400700 1 -- 172.21.15.63:0/3836699610 >> 172.21.15.63:0/3836699610 conn(0x7f8ed8076d20 msgr2=0x7f8ed8108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.008+0000 7f8ede400700 1 -- 172.21.15.63:0/3836699610 shutdown_connections 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.008+0000 7f8ede400700 1 -- 172.21.15.63:0/3836699610 wait complete. 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 Processor -- start 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 -- start start 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ed8109de0 0x7f8ed81a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 0x7f8ed81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ed810eec0 0x7f8ed81a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:17.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8119bf0 con 0x7f8ed810a790 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8119a70 con 0x7f8ed810eec0 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ede400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8119d70 con 0x7f8ed8109de0 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.009+0000 7f8ed6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 0x7f8ed81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 0x7f8ed81a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35178/0 (socket says 172.21.15.63:35178) 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed6a00700 1 -- 172.21.15.63:0/957727018 learned_addr learned my addr 172.21.15.63:0/957727018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed7400700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ed8109de0 0x7f8ed81a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed7e00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ed810eec0 0x7f8ed81a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed6a00700 1 -- 172.21.15.63:0/957727018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ed8109de0 msgr2=0x7f8ed81a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed6a00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ed8109de0 0x7f8ed81a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed6a00700 1 -- 172.21.15.63:0/957727018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ed810eec0 msgr2=0x7f8ed81a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed6a00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ed810eec0 0x7f8ed81a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.010+0000 7f8ed6a00700 1 -- 172.21.15.63:0/957727018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed81a5140 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.011+0000 7f8ed6a00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 0x7f8ed81a06c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b580 tx=0x7f8ec40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.011+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4027070 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.011+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec400fd30 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.011+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40173f0 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.011+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed81a53d0 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.011+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81accb0 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.012+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ed8063ce0 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.015+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8ec402a070 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.016+0000 7f8ecfe00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ebc0b2990 0x7f8ebc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.016+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8ec40e3940 con 0x7f8ed810a790 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.016+0000 7f8ed7400700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ebc0b2990 0x7f8ebc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.017+0000 7f8ecfe00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] conn(0x7f8ebc0bdc70 0x7f8ebc0c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:17.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.017+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 --> [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ec4007e80 con 0x7f8ebc0bdc70 2024-05-11T13:31:17.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.017+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8ec40e3cf0 con 0x7f8ed810a790 2024-05-11T13:31:17.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.017+0000 7f8ed7e00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] conn(0x7f8ebc0bdc70 0x7f8ebc0c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:17.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.017+0000 7f8ed7400700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ebc0b2990 0x7f8ebc0b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005950 tx=0x7f8ec800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:17.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.018+0000 7f8ed7e00700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] conn(0x7f8ebc0bdc70 0x7f8ebc0c0110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:17.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.019+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== osd.0 v2:172.21.15.63:6802/1611555800 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ec4007e80 con 0x7f8ebc0bdc70 2024-05-11T13:31:17.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.038+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 --> [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ed80641b0 con 0x7f8ebc0bdc70 2024-05-11T13:31:17.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.038+0000 7f8ecfe00700 1 -- 172.21.15.63:0/957727018 <== osd.0 v2:172.21.15.63:6802/1611555800 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8ed80641b0 con 0x7f8ebc0bdc70 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] conn(0x7f8ebc0bdc70 msgr2=0x7f8ebc0c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] conn(0x7f8ebc0bdc70 0x7f8ebc0c0110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ebc0b2990 msgr2=0x7f8ebc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8ebc0b2990 0x7f8ebc0b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005950 tx=0x7f8ec800b410 comp rx=0 tx=0).stop 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 msgr2=0x7f8ed81a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 --2- 172.21.15.63:0/957727018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed810a790 0x7f8ed81a06c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b580 tx=0x7f8ec40057c0 comp rx=0 tx=0).stop 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ed7e00700 1 -- 172.21.15.63:0/957727018 reap_dead start 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 shutdown_connections 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.039+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 >> 172.21.15.63:0/957727018 conn(0x7f8ed8076d20 msgr2=0x7f8ed8110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.040+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 shutdown_connections 2024-05-11T13:31:17.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:17.040+0000 7f8ede400700 1 -- 172.21.15.63:0/957727018 wait complete. 2024-05-11T13:31:18.340 INFO:teuthology.orchestra.run.smithi063.stdout:257698037825 2024-05-11T13:31:18.341 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-11T13:31:18.725 INFO:teuthology.orchestra.run.smithi063.stdout:55834574969 2024-05-11T13:31:18.725 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 -- 172.21.15.63:0/2150486164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 msgr2=0x7f533c06bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 --2- 172.21.15.63:0/2150486164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 0x7f533c06bf40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5328009990 tx=0x7f5328009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 -- 172.21.15.63:0/2150486164 shutdown_connections 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 --2- 172.21.15.63:0/2150486164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533c06c480 0x7f533c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 --2- 172.21.15.63:0/2150486164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f533c1095b0 0x7f533c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 --2- 172.21.15.63:0/2150486164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 0x7f533c06bf40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 -- 172.21.15.63:0/2150486164 >> 172.21.15.63:0/2150486164 conn(0x7f533c0793c0 msgr2=0x7f533c0797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 -- 172.21.15.63:0/2150486164 shutdown_connections 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.135+0000 7f5344000700 1 -- 172.21.15.63:0/2150486164 wait complete. 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.136+0000 7f5344000700 1 Processor -- start 2024-05-11T13:31:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.136+0000 7f5344000700 1 -- start start 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5344000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 0x7f533c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5344000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533c06c480 0x7f533c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5344000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f533c1095b0 0x7f533c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5344000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f533c113570 con 0x7f533c06bac0 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5344000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f533c1133f0 con 0x7f533c1095b0 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5344000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f533c1136f0 con 0x7f533c06c480 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 0x7f533c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 0x7f533c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35194/0 (socket says 172.21.15.63:35194) 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 -- 172.21.15.63:0/3438803894 learned_addr learned my addr 172.21.15.63:0/3438803894 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 -- 172.21.15.63:0/3438803894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533c06c480 msgr2=0x7f533c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533c06c480 0x7f533c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5342000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f533c1095b0 0x7f533c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 -- 172.21.15.63:0/3438803894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f533c1095b0 msgr2=0x7f533c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f533c1095b0 0x7f533c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.137+0000 7f5341600700 1 -- 172.21.15.63:0/3438803894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c19a030 con 0x7f533c06bac0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.138+0000 7f5342000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f533c1095b0 0x7f533c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.138+0000 7f5341600700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 0x7f533c19f710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f532c007e20 tx=0x7f532c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.138+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c016020 con 0x7f533c06bac0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.138+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f532c00ae00 con 0x7f533c06bac0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.138+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c011670 con 0x7f533c06bac0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.138+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533c19a320 con 0x7f533c06bac0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.138+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533c1a65e0 con 0x7f533c06bac0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.139+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f533c063ca0 con 0x7f533c06bac0 2024-05-11T13:31:18.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.142+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f532c019020 con 0x7f533c06bac0 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.143+0000 7f533a000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f53200b2990 0x7f53200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.143+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f532c0e2b40 con 0x7f533c06bac0 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.143+0000 7f5340c00700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f53200b2990 0x7f53200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.143+0000 7f533a000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] conn(0x7f53200bdc70 0x7f53200c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.144+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 --> [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53200c07e0 con 0x7f53200bdc70 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.144+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f532c0e2ef0 con 0x7f533c06bac0 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.144+0000 7f5340c00700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f53200b2990 0x7f53200b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f532800b5c0 tx=0x7f5328005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.144+0000 7f5342000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] conn(0x7f53200bdc70 0x7f53200c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.145+0000 7f5342000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] conn(0x7f53200bdc70 0x7f53200c0110 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.149+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== osd.5 v2:172.21.15.165:6808/2071542018 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53200c07e0 con 0x7f53200bdc70 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.167+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 --> [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f533c064170 con 0x7f53200bdc70 2024-05-11T13:31:18.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.168+0000 7f533a000700 1 -- 172.21.15.63:0/3438803894 <== osd.5 v2:172.21.15.165:6808/2071542018 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f533c064170 con 0x7f53200bdc70 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 >> [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] conn(0x7f53200bdc70 msgr2=0x7f53200c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] conn(0x7f53200bdc70 0x7f53200c0110 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f53200b2990 msgr2=0x7f53200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f53200b2990 0x7f53200b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f532800b5c0 tx=0x7f5328005fb0 comp rx=0 tx=0).stop 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 msgr2=0x7f533c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 --2- 172.21.15.63:0/3438803894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533c06bac0 0x7f533c19f710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f532c007e20 tx=0x7f532c00a340 comp rx=0 tx=0).stop 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5342000700 1 -- 172.21.15.63:0/3438803894 reap_dead start 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 shutdown_connections 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.169+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 >> 172.21.15.63:0/3438803894 conn(0x7f533c0793c0 msgr2=0x7f533c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.170+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 shutdown_connections 2024-05-11T13:31:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:18.170+0000 7f5344000700 1 -- 172.21.15.63:0/3438803894 wait complete. 2024-05-11T13:31:18.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:18 smithi063 ceph-mon[29809]: pgmap v459: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:19.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:18 smithi165 ceph-mon[33235]: pgmap v459: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:19.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:18 smithi071 ceph-mon[33293]: pgmap v459: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:19.560 INFO:teuthology.orchestra.run.smithi063.stdout:137438953572 2024-05-11T13:31:19.560 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-11T13:31:19.676 INFO:teuthology.orchestra.run.smithi063.stdout:231928234057 2024-05-11T13:31:19.677 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-11T13:31:21.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:20 smithi165 ceph-mon[33235]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:21.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:20 smithi063 ceph-mon[29809]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:21.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:20 smithi071 ceph-mon[33293]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:21.243 INFO:teuthology.orchestra.run.smithi063.stdout:34359738496 2024-05-11T13:31:21.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-11T13:31:21.855 INFO:teuthology.orchestra.run.smithi063.stdout:158913790047 2024-05-11T13:31:21.856 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-11T13:31:22.210 INFO:teuthology.orchestra.run.smithi063.stdout:184683593815 2024-05-11T13:31:22.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-11T13:31:22.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:22 smithi063 ceph-mon[29809]: pgmap v461: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:23.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:22 smithi165 ceph-mon[33235]: pgmap v461: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:23.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:22 smithi071 ceph-mon[33293]: pgmap v461: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:23.461 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 -- 172.21.15.63:0/889137516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 msgr2=0x7fa92806c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 --2- 172.21.15.63:0/889137516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 0x7fa92806c0a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa910009990 tx=0x7fa910009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 -- 172.21.15.63:0/889137516 shutdown_connections 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 --2- 172.21.15.63:0/889137516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa92806c5e0 0x7fa928110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 --2- 172.21.15.63:0/889137516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9281095e0 0x7fa9281099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 --2- 172.21.15.63:0/889137516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 0x7fa92806c0a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 -- 172.21.15.63:0/889137516 >> 172.21.15.63:0/889137516 conn(0x7fa928079400 msgr2=0x7fa928079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 -- 172.21.15.63:0/889137516 shutdown_connections 2024-05-11T13:31:23.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.475+0000 7fa92de00700 1 -- 172.21.15.63:0/889137516 wait complete. 2024-05-11T13:31:23.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.476+0000 7fa92de00700 1 Processor -- start 2024-05-11T13:31:23.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.477+0000 7fa92de00700 1 -- start start 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.477+0000 7fa92de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 0x7fa92819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.477+0000 7fa92de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa92806c5e0 0x7fa92819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.477+0000 7fa92de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9281095e0 0x7fa9281a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.477+0000 7fa92de00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa928113680 con 0x7fa92806bc20 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa92de00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa928113500 con 0x7fa9281095e0 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa92de00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa928113800 con 0x7fa92806c5e0 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 0x7fa92819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 0x7fa92819beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35986/0 (socket says 172.21.15.63:35986) 2024-05-11T13:31:23.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 -- 172.21.15.63:0/923422963 learned_addr learned my addr 172.21.15.63:0/923422963 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa927400700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9281095e0 0x7fa9281a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 -- 172.21.15.63:0/923422963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa92806c5e0 msgr2=0x7fa92819c3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa92806c5e0 0x7fa92819c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 -- 172.21.15.63:0/923422963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9281095e0 msgr2=0x7fa9281a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9281095e0 0x7fa9281a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa926a00700 1 -- 172.21.15.63:0/923422963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281a0ec0 con 0x7fa92806bc20 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.478+0000 7fa927400700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9281095e0 0x7fa9281a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:23.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.479+0000 7fa926a00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 0x7fa92819beb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa91800d950 tx=0x7fa91800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:23.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.479+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918016070 con 0x7fa92806bc20 2024-05-11T13:31:23.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.479+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa918010460 con 0x7fa92806bc20 2024-05-11T13:31:23.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.479+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91800f620 con 0x7fa92806bc20 2024-05-11T13:31:23.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.479+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9281a11b0 con 0x7fa92806bc20 2024-05-11T13:31:23.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.479+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9281a8a30 con 0x7fa92806bc20 2024-05-11T13:31:23.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.480+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa92810f110 con 0x7fa92806bc20 2024-05-11T13:31:23.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.483+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa918019070 con 0x7fa92806bc20 2024-05-11T13:31:23.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.485+0000 7fa92ca00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa90c0b2990 0x7fa90c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.485+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa9180e2830 con 0x7fa92806bc20 2024-05-11T13:31:23.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.485+0000 7fa91fe00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa90c0b2990 0x7fa90c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.485+0000 7fa92ca00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] conn(0x7fa90c0bdc70 0x7fa90c0c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.485+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 --> [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa91800ae50 con 0x7fa90c0bdc70 2024-05-11T13:31:23.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.485+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa9180e2be0 con 0x7fa92806bc20 2024-05-11T13:31:23.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.486+0000 7fa91fe00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa90c0b2990 0x7fa90c0b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098e0 tx=0x7fa910005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:23.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.486+0000 7fa927400700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] conn(0x7fa90c0bdc70 0x7fa90c0c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.487+0000 7fa927400700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] conn(0x7fa90c0bdc70 0x7fa90c0c0110 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:23.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.489+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== osd.2 v2:172.21.15.165:6800/1296073597 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa91800ae50 con 0x7fa90c0bdc70 2024-05-11T13:31:23.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.509+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 --> [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa92806b4d0 con 0x7fa90c0bdc70 2024-05-11T13:31:23.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.510+0000 7fa92ca00700 1 -- 172.21.15.63:0/923422963 <== osd.2 v2:172.21.15.165:6800/1296073597 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa92806b4d0 con 0x7fa90c0bdc70 2024-05-11T13:31:23.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.510+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 >> [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] conn(0x7fa90c0bdc70 msgr2=0x7fa90c0c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.510+0000 7fa92de00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] conn(0x7fa90c0bdc70 0x7fa90c0c0110 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa90c0b2990 msgr2=0x7fa90c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa90c0b2990 0x7fa90c0b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098e0 tx=0x7fa910005e80 comp rx=0 tx=0).stop 2024-05-11T13:31:23.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 msgr2=0x7fa92819beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 --2- 172.21.15.63:0/923422963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa92806bc20 0x7fa92819beb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa91800d950 tx=0x7fa91800dd10 comp rx=0 tx=0).stop 2024-05-11T13:31:23.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa927400700 1 -- 172.21.15.63:0/923422963 reap_dead start 2024-05-11T13:31:23.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 shutdown_connections 2024-05-11T13:31:23.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 >> 172.21.15.63:0/923422963 conn(0x7fa928079400 msgr2=0x7fa92810e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:23.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 shutdown_connections 2024-05-11T13:31:23.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.511+0000 7fa92de00700 1 -- 172.21.15.63:0/923422963 wait complete. 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 -- 172.21.15.63:0/2676774853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103e50 msgr2=0x7f874c110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 --2- 172.21.15.63:0/2676774853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103e50 0x7f874c110310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f874400b0d0 tx=0x7f874400b3e0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 -- 172.21.15.63:0/2676774853 shutdown_connections 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 --2- 172.21.15.63:0/2676774853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103e50 0x7f874c110310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 --2- 172.21.15.63:0/2676774853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f874c109540 0x7f874c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 --2- 172.21.15.63:0/2676774853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f874c103490 0x7f874c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 -- 172.21.15.63:0/2676774853 >> 172.21.15.63:0/2676774853 conn(0x7f874c0fef60 msgr2=0x7f874c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 -- 172.21.15.63:0/2676774853 shutdown_connections 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.723+0000 7f8752a00700 1 -- 172.21.15.63:0/2676774853 wait complete. 2024-05-11T13:31:23.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.724+0000 7f8752a00700 1 Processor -- start 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8752a00700 1 -- start start 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8752a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103490 0x7f874c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8752a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f874c103e50 0x7f874c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8752a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f874c109540 0x7f874c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8752a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1134d0 con 0x7f874c103490 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8752a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f874c113350 con 0x7f874c103e50 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8752a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f874c113650 con 0x7f874c109540 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8751600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103490 0x7f874c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8751600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103490 0x7f874c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35998/0 (socket says 172.21.15.63:35998) 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.725+0000 7f8751600700 1 -- 172.21.15.63:0/4199169016 learned_addr learned my addr 172.21.15.63:0/4199169016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8750c00700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f874c103e50 0x7f874c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8752000700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f874c109540 0x7f874c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8751600700 1 -- 172.21.15.63:0/4199169016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f874c109540 msgr2=0x7f874c199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8751600700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f874c109540 0x7f874c199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8751600700 1 -- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f874c103e50 msgr2=0x7f874c19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8751600700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f874c103e50 0x7f874c19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8751600700 1 -- 172.21.15.63:0/4199169016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874c19a070 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8752000700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f874c109540 0x7f874c199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8751600700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103490 0x7f874c19f670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f874000d950 tx=0x7f874000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740016070 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8740010460 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.726+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874c19a360 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.727+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874000f620 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.727+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874c1a65f0 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.728+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f874c063ca0 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.730+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8740019070 con 0x7f874c103490 2024-05-11T13:31:23.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.732+0000 7f874a000700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f87340b2910 0x7f87340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.732+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f87400e2840 con 0x7f874c103490 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.732+0000 7f8750c00700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f87340b2910 0x7f87340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.732+0000 7f874a000700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] conn(0x7f87340bdbf0 0x7f87340c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.732+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 --> [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8740025ea0 con 0x7f87340bdbf0 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.732+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f87400e2bf0 con 0x7f874c103490 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.733+0000 7f8750c00700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f87340b2910 0x7f87340b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8738005950 tx=0x7f87380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.733+0000 7f8752000700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] conn(0x7f87340bdbf0 0x7f87340c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.734+0000 7f8752000700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] conn(0x7f87340bdbf0 0x7f87340c0090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.736+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== osd.10 v2:172.21.15.71:6824/3023940928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8740025ea0 con 0x7f87340bdbf0 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.755+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 --> [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f874c064120 con 0x7f87340bdbf0 2024-05-11T13:31:23.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.757+0000 7f874a000700 1 -- 172.21.15.63:0/4199169016 <== osd.10 v2:172.21.15.71:6824/3023940928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f874c064120 con 0x7f87340bdbf0 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] conn(0x7f87340bdbf0 msgr2=0x7f87340c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] conn(0x7f87340bdbf0 0x7f87340c0090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f87340b2910 msgr2=0x7f87340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f87340b2910 0x7f87340b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8738005950 tx=0x7f87380058e0 comp rx=0 tx=0).stop 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103490 msgr2=0x7f874c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 --2- 172.21.15.63:0/4199169016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f874c103490 0x7f874c19f670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f874000d950 tx=0x7f874000dd10 comp rx=0 tx=0).stop 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752000700 1 -- 172.21.15.63:0/4199169016 reap_dead start 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 shutdown_connections 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 >> 172.21.15.63:0/4199169016 conn(0x7f874c0fef60 msgr2=0x7f874c1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 shutdown_connections 2024-05-11T13:31:23.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:23.758+0000 7f8752a00700 1 -- 172.21.15.63:0/4199169016 wait complete. 2024-05-11T13:31:24.360 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:25.018 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:24 smithi063 ceph-mon[29809]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:25.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:24 smithi165 ceph-mon[33235]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 -- 172.21.15.63:0/3927337484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c109550 msgr2=0x7f2b4c109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 --2- 172.21.15.63:0/3927337484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c109550 0x7f2b4c109930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00b0d0 tx=0x7f2b3c00b3e0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 -- 172.21.15.63:0/3927337484 shutdown_connections 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 --2- 172.21.15.63:0/3927337484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b4c103e60 0x7f2b4c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 --2- 172.21.15.63:0/3927337484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c109550 0x7f2b4c109930 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 --2- 172.21.15.63:0/3927337484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4c1034a0 0x7f2b4c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 -- 172.21.15.63:0/3927337484 >> 172.21.15.63:0/3927337484 conn(0x7f2b4c0fefd0 msgr2=0x7f2b4c1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 -- 172.21.15.63:0/3927337484 shutdown_connections 2024-05-11T13:31:25.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.773+0000 7f2b4be00700 1 -- 172.21.15.63:0/3927337484 wait complete. 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.774+0000 7f2b4be00700 1 Processor -- start 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4be00700 1 -- start start 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c1034a0 0x7f2b4c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4c103e60 0x7f2b4c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b4c109550 0x7f2b4c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1135e0 con 0x7f2b4c1034a0 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4be00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c113460 con 0x7f2b4c109550 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4be00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c113760 con 0x7f2b4c103e60 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c1034a0 0x7f2b4c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c1034a0 0x7f2b4c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36026/0 (socket says 172.21.15.63:36026) 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.775+0000 7f2b4aa00700 1 -- 172.21.15.63:0/1882840646 learned_addr learned my addr 172.21.15.63:0/1882840646 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b42000700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4c103e60 0x7f2b4c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:25.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4b400700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b4c109550 0x7f2b4c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:25.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4aa00700 1 -- 172.21.15.63:0/1882840646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4c103e60 msgr2=0x7f2b4c19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:25.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4aa00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4c103e60 0x7f2b4c19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4aa00700 1 -- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b4c109550 msgr2=0x7f2b4c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:25.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4aa00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b4c109550 0x7f2b4c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4aa00700 1 -- 172.21.15.63:0/1882840646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c1a0d20 con 0x7f2b4c1034a0 2024-05-11T13:31:25.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4b400700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b4c109550 0x7f2b4c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:25.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4aa00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c1034a0 0x7f2b4c19bd10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c007dc0 tx=0x7f2b3c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:25.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c01f030 con 0x7f2b4c1034a0 2024-05-11T13:31:25.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c1a0fb0 con 0x7f2b4c1034a0 2024-05-11T13:31:25.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b3c01bb90 con 0x7f2b4c1034a0 2024-05-11T13:31:25.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.776+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c012bb0 con 0x7f2b4c1034a0 2024-05-11T13:31:25.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.777+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c1a8890 con 0x7f2b4c1034a0 2024-05-11T13:31:25.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.777+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b4c101060 con 0x7f2b4c1034a0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.780+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2b3c022030 con 0x7f2b4c1034a0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.781+0000 7f2b43e00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2b300b2910 0x7f2b300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.782+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2b3c00f020 con 0x7f2b4c1034a0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.782+0000 7f2b42000700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2b300b2910 0x7f2b300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.782+0000 7f2b43e00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] conn(0x7f2b300bdbf0 0x7f2b300c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.782+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 --> [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b3c01ce80 con 0x7f2b300bdbf0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.782+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f2b3c0e42a0 con 0x7f2b4c1034a0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.782+0000 7f2b42000700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2b300b2910 0x7f2b300b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b34006fd0 tx=0x7f2b34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.783+0000 7f2b4b400700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] conn(0x7f2b300bdbf0 0x7f2b300c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.784+0000 7f2b4b400700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] conn(0x7f2b300bdbf0 0x7f2b300c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.786+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== osd.7 v2:172.21.15.71:6816/3194467426 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b3c01ce80 con 0x7f2b300bdbf0 2024-05-11T13:31:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.804+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 --> [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b4c104bc0 con 0x7f2b300bdbf0 2024-05-11T13:31:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.805+0000 7f2b43e00700 1 -- 172.21.15.63:0/1882840646 <== osd.7 v2:172.21.15.71:6816/3194467426 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2b4c104bc0 con 0x7f2b300bdbf0 2024-05-11T13:31:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.806+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] conn(0x7f2b300bdbf0 msgr2=0x7f2b300c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.806+0000 7f2b4be00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] conn(0x7f2b300bdbf0 0x7f2b300c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.806+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2b300b2910 msgr2=0x7f2b300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.806+0000 7f2b4be00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2b300b2910 0x7f2b300b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b34006fd0 tx=0x7f2b34008040 comp rx=0 tx=0).stop 2024-05-11T13:31:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.806+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c1034a0 msgr2=0x7f2b4c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.806+0000 7f2b4be00700 1 --2- 172.21.15.63:0/1882840646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b4c1034a0 0x7f2b4c19bd10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c007dc0 tx=0x7f2b3c007ea0 comp rx=0 tx=0).stop 2024-05-11T13:31:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.806+0000 7f2b4b400700 1 -- 172.21.15.63:0/1882840646 reap_dead start 2024-05-11T13:31:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.807+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 shutdown_connections 2024-05-11T13:31:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.807+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 >> 172.21.15.63:0/1882840646 conn(0x7f2b4c0fefd0 msgr2=0x7f2b4c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.807+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 shutdown_connections 2024-05-11T13:31:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:24.807+0000 7f2b4be00700 1 -- 172.21.15.63:0/1882840646 wait complete. 2024-05-11T13:31:25.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:24 smithi071 ceph-mon[33293]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:26.187 INFO:teuthology.orchestra.run.smithi063.stdout:279172874299 2024-05-11T13:31:26.188 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-11T13:31:27.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:26 smithi165 ceph-mon[33235]: pgmap v463: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-11T13:31:27.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:26 smithi063 ceph-mon[29809]: pgmap v463: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-11T13:31:27.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:26 smithi071 ceph-mon[33293]: pgmap v463: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-11T13:31:27.647 INFO:teuthology.orchestra.run.smithi063.stdout:206158430290 2024-05-11T13:31:27.648 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-11T13:31:27.914 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:27.914 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:27.923 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:27.923 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:27.935 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:27.935 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:27.999 INFO:teuthology.orchestra.run.smithi063.stdout:77309411444 2024-05-11T13:31:28.000 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-11T13:31:28.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:28 smithi063 ceph-mon[29809]: pgmap v464: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-11T13:31:29.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:28 smithi165 ceph-mon[33235]: pgmap v464: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-11T13:31:29.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:28 smithi071 ceph-mon[33293]: pgmap v464: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-11T13:31:30.892 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:30 smithi063 ceph-mon[29809]: pgmap v465: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:31.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:30 smithi165 ceph-mon[33235]: pgmap v465: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:31.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:30 smithi071 ceph-mon[33293]: pgmap v465: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:32.265 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:33.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:32 smithi165 ceph-mon[33235]: pgmap v466: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:33.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:32 smithi063 ceph-mon[29809]: pgmap v466: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:33.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:32 smithi071 ceph-mon[33293]: pgmap v466: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 -- 172.21.15.63:0/2290747821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806b230 msgr2=0x7fd6d806b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 --2- 172.21.15.63:0/2290747821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806b230 0x7fd6d806b6b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009990 tx=0x7fd6c4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 -- 172.21.15.63:0/2290747821 shutdown_connections 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 --2- 172.21.15.63:0/2290747821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d806bbf0 0x7fd6d81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 --2- 172.21.15.63:0/2290747821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d81095c0 0x7fd6d81099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 --2- 172.21.15.63:0/2290747821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806b230 0x7fd6d806b6b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 -- 172.21.15.63:0/2290747821 >> 172.21.15.63:0/2290747821 conn(0x7fd6d8079400 msgr2=0x7fd6d8079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 -- 172.21.15.63:0/2290747821 shutdown_connections 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.041+0000 7fd6dde00700 1 -- 172.21.15.63:0/2290747821 wait complete. 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.042+0000 7fd6dde00700 1 Processor -- start 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.043+0000 7fd6dde00700 1 -- start start 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.043+0000 7fd6dde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d806b230 0x7fd6d8102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.043+0000 7fd6dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806bbf0 0x7fd6d8102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6dde00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d81095c0 0x7fd6d8102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6dde00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d81135a0 con 0x7fd6d806bbf0 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6dde00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8113420 con 0x7fd6d806b230 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6dde00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8113720 con 0x7fd6d81095c0 2024-05-11T13:31:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806bbf0 0x7fd6d8102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806bbf0 0x7fd6d8102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39214/0 (socket says 172.21.15.63:39214) 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 -- 172.21.15.63:0/91518801 learned_addr learned my addr 172.21.15.63:0/91518801 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6a00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d806b230 0x7fd6d8102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d7400700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d81095c0 0x7fd6d8102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 -- 172.21.15.63:0/91518801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d81095c0 msgr2=0x7fd6d8102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d81095c0 0x7fd6d8102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 -- 172.21.15.63:0/91518801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d806b230 msgr2=0x7fd6d8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d806b230 0x7fd6d8102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6000700 1 -- 172.21.15.63:0/91518801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d81a8080 con 0x7fd6d806bbf0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.044+0000 7fd6d6a00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d806b230 0x7fd6d8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.045+0000 7fd6d6000700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806bbf0 0x7fd6d8102a60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000c00 tx=0x7fd6c4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.045+0000 7fd6dca00700 1 -- 172.21.15.63:0/91518801 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4027070 con 0x7fd6d806bbf0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.045+0000 7fd6dca00700 1 -- 172.21.15.63:0/91518801 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6c400fc70 con 0x7fd6d806bbf0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.045+0000 7fd6dca00700 1 -- 172.21.15.63:0/91518801 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4022d40 con 0x7fd6d806bbf0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.045+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d81a8310 con 0x7fd6d806bbf0 2024-05-11T13:31:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.045+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d81a8710 con 0x7fd6d806bbf0 2024-05-11T13:31:34.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.048+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d810baa0 con 0x7fd6d806bbf0 2024-05-11T13:31:34.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.049+0000 7fd6dca00700 1 -- 172.21.15.63:0/91518801 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd6c402a070 con 0x7fd6d806bbf0 2024-05-11T13:31:34.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.051+0000 7fd6dca00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd6bc0b2910 0x7fd6bc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:34.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.051+0000 7fd6dca00700 1 -- 172.21.15.63:0/91518801 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd6c40e3550 con 0x7fd6d806bbf0 2024-05-11T13:31:34.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.051+0000 7fd6d6a00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd6bc0b2910 0x7fd6bc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:34.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.052+0000 7fd6d6a00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd6bc0b2910 0x7fd6bc0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005e10 tx=0x7fd6c8005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:34.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.055+0000 7fd6dca00700 1 -- 172.21.15.63:0/91518801 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd6c405fdf0 con 0x7fd6d806bbf0 2024-05-11T13:31:34.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.329+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd6d810b480 con 0x7fd6d806bbf0 2024-05-11T13:31:34.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.329+0000 7fd6dca00700 1 -- 172.21.15.63:0/91518801 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd6c4013070 con 0x7fd6d806bbf0 2024-05-11T13:31:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:304942678071 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd6bc0b2910 msgr2=0x7fd6bc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd6bc0b2910 0x7fd6bc0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005e10 tx=0x7fd6c8005d80 comp rx=0 tx=0).stop 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806bbf0 msgr2=0x7fd6d8102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806bbf0 0x7fd6d8102a60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000c00 tx=0x7fd6c4009f90 comp rx=0 tx=0).stop 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 shutdown_connections 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd6bc0b2910 0x7fd6bc0b4dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d81095c0 0x7fd6d8102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d806b230 0x7fd6d8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 --2- 172.21.15.63:0/91518801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6d806bbf0 0x7fd6d8102a60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:34.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.333+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 >> 172.21.15.63:0/91518801 conn(0x7fd6d8079400 msgr2=0x7fd6d81012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:34.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.334+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 shutdown_connections 2024-05-11T13:31:34.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.334+0000 7fd6dde00700 1 -- 172.21.15.63:0/91518801 wait complete. 2024-05-11T13:31:34.889 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:34 smithi063 ceph-mon[29809]: pgmap v467: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:34.889 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:34 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/91518801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-11T13:31:35.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:34 smithi165 ceph-mon[33235]: pgmap v467: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:35.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:34 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/91518801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-11T13:31:35.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:34 smithi071 ceph-mon[33293]: pgmap v467: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:35.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:34 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/91518801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-11T13:31:35.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 -- 172.21.15.63:0/1323915212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 msgr2=0x7f25f410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 --2- 172.21.15.63:0/1323915212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f410fa10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009990 tx=0x7f25e0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 -- 172.21.15.63:0/1323915212 shutdown_connections 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 --2- 172.21.15.63:0/1323915212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f410ff50 0x7f25f4112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 --2- 172.21.15.63:0/1323915212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25f4075780 0x7f25f4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 --2- 172.21.15.63:0/1323915212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f410fa10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 -- 172.21.15.63:0/1323915212 >> 172.21.15.63:0/1323915212 conn(0x7f25f407a6b0 msgr2=0x7f25f407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 -- 172.21.15.63:0/1323915212 shutdown_connections 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.623+0000 7f25fc000700 1 -- 172.21.15.63:0/1323915212 wait complete. 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.624+0000 7f25fc000700 1 Processor -- start 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.624+0000 7f25fc000700 1 -- start start 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25fc000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f4075780 0x7f25f419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25fc000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25fc000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25f410ff50 0x7f25f41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25fc000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f25f4115830 con 0x7f25f40760a0 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25fc000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41156b0 con 0x7f25f4075780 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25fc000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41159b0 con 0x7f25f410ff50 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25f8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25f8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39240/0 (socket says 172.21.15.63:39240) 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.625+0000 7f25f8c00700 1 -- 172.21.15.63:0/2266346348 learned_addr learned my addr 172.21.15.63:0/2266346348 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25fa000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25f410ff50 0x7f25f41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f9600700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f4075780 0x7f25f419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f8c00700 1 -- 172.21.15.63:0/2266346348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25f410ff50 msgr2=0x7f25f41a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f8c00700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25f410ff50 0x7f25f41a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f8c00700 1 -- 172.21.15.63:0/2266346348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f4075780 msgr2=0x7f25f419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f8c00700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f4075780 0x7f25f419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f8c00700 1 -- 172.21.15.63:0/2266346348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f41a0d90 con 0x7f25f40760a0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25fa000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25f410ff50 0x7f25f41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f9600700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f4075780 0x7f25f419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f8c00700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f419c2c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000c00 tx=0x7f25e0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f2000700 1 -- 172.21.15.63:0/2266346348 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0027070 con 0x7f25f40760a0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f41a1020 con 0x7f25f40760a0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.626+0000 7f25f2000700 1 -- 172.21.15.63:0/2266346348 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25e000fb80 con 0x7f25f40760a0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.627+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f41a8900 con 0x7f25f40760a0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.627+0000 7f25f2000700 1 -- 172.21.15.63:0/2266346348 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0022bf0 con 0x7f25f40760a0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.628+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f406a240 con 0x7f25f40760a0 2024-05-11T13:31:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.630+0000 7f25f2000700 1 -- 172.21.15.63:0/2266346348 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f25e002a070 con 0x7f25f40760a0 2024-05-11T13:31:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.631+0000 7f25f2000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f25dc0b2910 0x7f25dc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.632+0000 7f25f2000700 1 -- 172.21.15.63:0/2266346348 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f25e00e3410 con 0x7f25f40760a0 2024-05-11T13:31:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.632+0000 7f25f9600700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f25dc0b2910 0x7f25dc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.633+0000 7f25f9600700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f25dc0b2910 0x7f25dc0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25e8005fd0 tx=0x7f25e8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.636+0000 7f25f2000700 1 -- 172.21.15.63:0/2266346348 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25e0013070 con 0x7f25f40760a0 2024-05-11T13:31:35.790 INFO:teuthology.orchestra.run.smithi063.stdout:137438953577 2024-05-11T13:31:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.887+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f25f402e570 con 0x7f25f40760a0 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.888+0000 7f25f2000700 1 -- 172.21.15.63:0/2266346348 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f25e00aacb0 con 0x7f25f40760a0 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f25dc0b2910 msgr2=0x7f25dc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f25dc0b2910 0x7f25dc0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25e8005fd0 tx=0x7f25e8005ee0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 msgr2=0x7f25f419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f419c2c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000c00 tx=0x7f25e0005ef0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 shutdown_connections 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f25dc0b2910 0x7f25dc0b4dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25f410ff50 0x7f25f41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f4075780 0x7f25f419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 --2- 172.21.15.63:0/2266346348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25f40760a0 0x7f25f419c2c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 >> 172.21.15.63:0/2266346348 conn(0x7f25f407a6b0 msgr2=0x7f25f4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 shutdown_connections 2024-05-11T13:31:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:34.892+0000 7f25fc000700 1 -- 172.21.15.63:0/2266346348 wait complete. 2024-05-11T13:31:36.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:35 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2266346348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-11T13:31:36.141 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:35 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2266346348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-11T13:31:36.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:35 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2266346348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-11T13:31:36.997 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:36 smithi063 ceph-mon[29809]: pgmap v468: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 -- 172.21.15.63:0/3773063530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0106120 msgr2=0x7f44d0112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 --2- 172.21.15.63:0/3773063530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0106120 0x7f44d0112630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44c800b0a0 tx=0x7f44c800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 -- 172.21.15.63:0/3773063530 shutdown_connections 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 --2- 172.21.15.63:0/3773063530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0106120 0x7f44d0112630 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 --2- 172.21.15.63:0/3773063530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44d010b760 0x7f44d010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 --2- 172.21.15.63:0/3773063530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44d0105760 0x7f44d0105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 -- 172.21.15.63:0/3773063530 >> 172.21.15.63:0/3773063530 conn(0x7f44d0101250 msgr2=0x7f44d0103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 -- 172.21.15.63:0/3773063530 shutdown_connections 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.742+0000 7f44d5a00700 1 -- 172.21.15.63:0/3773063530 wait complete. 2024-05-11T13:31:37.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.743+0000 7f44d5a00700 1 Processor -- start 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.743+0000 7f44d5a00700 1 -- start start 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44d5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0105760 0x7f44d019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44d5a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44d0106120 0x7f44d019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44d5a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44d010b760 0x7f44d01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44d5a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44d0115860 con 0x7f44d0105760 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44d5a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f44d01156e0 con 0x7f44d0106120 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44d5a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f44d01159e0 con 0x7f44d010b760 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44cea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0105760 0x7f44d019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44cea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0105760 0x7f44d019bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39250/0 (socket says 172.21.15.63:39250) 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44cea00700 1 -- 172.21.15.63:0/4110068069 learned_addr learned my addr 172.21.15.63:0/4110068069 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44cf400700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44d010b760 0x7f44d01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.744+0000 7f44ce000700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44d0106120 0x7f44d019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:37.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44cea00700 1 -- 172.21.15.63:0/4110068069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44d010b760 msgr2=0x7f44d01a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44cea00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44d010b760 0x7f44d01a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44cea00700 1 -- 172.21.15.63:0/4110068069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44d0106120 msgr2=0x7f44d019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44cea00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44d0106120 0x7f44d019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44cea00700 1 -- 172.21.15.63:0/4110068069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d01a0d30 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44cea00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0105760 0x7f44d019bd70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f44c000d950 tx=0x7f44c000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44c7400700 1 -- 172.21.15.63:0/4110068069 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0016070 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44c7400700 1 -- 172.21.15.63:0/4110068069 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44c0010460 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d01a1020 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.745+0000 7f44c7400700 1 -- 172.21.15.63:0/4110068069 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c000f620 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.746+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d01a88a0 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.747+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d006a240 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.749+0000 7f44c7400700 1 -- 172.21.15.63:0/4110068069 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f44c0019070 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.750+0000 7f44c7400700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f44b40b2910 0x7f44b40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.751+0000 7f44c7400700 1 -- 172.21.15.63:0/4110068069 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f44c00e2350 con 0x7f44d0105760 2024-05-11T13:31:37.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.751+0000 7f44ce000700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f44b40b2910 0x7f44b40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:37.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.752+0000 7f44ce000700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f44b40b2910 0x7f44b40b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005950 tx=0x7f44bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:37.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.753+0000 7f44c7400700 1 -- 172.21.15.63:0/4110068069 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44c00a9b00 con 0x7f44d0105760 2024-05-11T13:31:37.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.996+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f44d002e570 con 0x7f44d0105760 2024-05-11T13:31:37.008 INFO:teuthology.orchestra.run.smithi063.stdout:55834574973 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:36.997+0000 7f44c7400700 1 -- 172.21.15.63:0/4110068069 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44c00a9400 con 0x7f44d0105760 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f44b40b2910 msgr2=0x7f44b40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f44b40b2910 0x7f44b40b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005950 tx=0x7f44bc0058e0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0105760 msgr2=0x7f44d019bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0105760 0x7f44d019bd70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f44c000d950 tx=0x7f44c000dd10 comp rx=0 tx=0).stop 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 shutdown_connections 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f44b40b2910 0x7f44b40b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44d010b760 0x7f44d01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d0105760 0x7f44d019bd70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 --2- 172.21.15.63:0/4110068069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44d0106120 0x7f44d019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.000+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 >> 172.21.15.63:0/4110068069 conn(0x7f44d0101250 msgr2=0x7f44d0101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.001+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 shutdown_connections 2024-05-11T13:31:37.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:37.001+0000 7f44d5a00700 1 -- 172.21.15.63:0/4110068069 wait complete. 2024-05-11T13:31:37.019 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:37.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:36 smithi165 ceph-mon[33235]: pgmap v468: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:37.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:36 smithi071 ceph-mon[33293]: pgmap v468: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:31:37.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:37 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4110068069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-11T13:31:38.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:37 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4110068069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-11T13:31:38.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:37 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4110068069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-11T13:31:38.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 -- 172.21.15.63:0/420682044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5278104470 msgr2=0x7f527806bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:38.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 --2- 172.21.15.63:0/420682044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5278104470 0x7f527806bc00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5268009990 tx=0x7f5268009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 -- 172.21.15.63:0/420682044 shutdown_connections 2024-05-11T13:31:38.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 --2- 172.21.15.63:0/420682044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f527810fde0 0x7f5278112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 --2- 172.21.15.63:0/420682044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5278104470 0x7f527806bc00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 --2- 172.21.15.63:0/420682044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527806c140 0x7f527806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 -- 172.21.15.63:0/420682044 >> 172.21.15.63:0/420682044 conn(0x7f52780791b0 msgr2=0x7f52780795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 -- 172.21.15.63:0/420682044 shutdown_connections 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.586+0000 7f527ce00700 1 -- 172.21.15.63:0/420682044 wait complete. 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.587+0000 7f527ce00700 1 Processor -- start 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.587+0000 7f527ce00700 1 -- start start 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f527ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f527806c140 0x7f527819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f527ce00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5278104470 0x7f527819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f527ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527810fde0 0x7f52781a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f527ce00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5278115810 con 0x7f527806c140 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f527ce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5278115690 con 0x7f527810fde0 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f527ce00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5278115990 con 0x7f5278104470 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f5275e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f527806c140 0x7f527819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:38.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.588+0000 7f5275400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5278104470 0x7f527819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5276800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527810fde0 0x7f52781a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5275e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f527806c140 0x7f527819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39270/0 (socket says 172.21.15.63:39270) 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5275e00700 1 -- 172.21.15.63:0/282327919 learned_addr learned my addr 172.21.15.63:0/282327919 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5275e00700 1 -- 172.21.15.63:0/282327919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5278104470 msgr2=0x7f527819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5275e00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5278104470 0x7f527819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5275e00700 1 -- 172.21.15.63:0/282327919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527810fde0 msgr2=0x7f52781a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5275e00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527810fde0 0x7f52781a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.589+0000 7f5275e00700 1 -- 172.21.15.63:0/282327919 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52781a0d90 con 0x7f527806c140 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.590+0000 7f5275e00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f527806c140 0x7f527819bd80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5268000c00 tx=0x7f52680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.590+0000 7f526ea00700 1 -- 172.21.15.63:0/282327919 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268027070 con 0x7f527806c140 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.590+0000 7f526ea00700 1 -- 172.21.15.63:0/282327919 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f526800fb00 con 0x7f527806c140 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.590+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52781a1020 con 0x7f527806c140 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.590+0000 7f526ea00700 1 -- 172.21.15.63:0/282327919 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268022c40 con 0x7f527806c140 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.590+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52781a8900 con 0x7f527806c140 2024-05-11T13:31:38.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.591+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5278102af0 con 0x7f527806c140 2024-05-11T13:31:38.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.594+0000 7f526ea00700 1 -- 172.21.15.63:0/282327919 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f526802a070 con 0x7f527806c140 2024-05-11T13:31:38.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.596+0000 7f526ea00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f525c0b2910 0x7f525c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:38.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.597+0000 7f526ea00700 1 -- 172.21.15.63:0/282327919 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f52680e3610 con 0x7f527806c140 2024-05-11T13:31:38.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.597+0000 7f5275400700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f525c0b2910 0x7f525c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:38.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.597+0000 7f5275400700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f525c0b2910 0x7f525c0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f52780799b0 tx=0x7f52640073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:38.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.601+0000 7f526ea00700 1 -- 172.21.15.63:0/282327919 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5268013070 con 0x7f527806c140 2024-05-11T13:31:38.952 INFO:teuthology.orchestra.run.smithi063.stdout:184683593820 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.852+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f527819d0e0 con 0x7f527806c140 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.853+0000 7f526ea00700 1 -- 172.21.15.63:0/282327919 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f52680aaeb0 con 0x7f527806c140 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.858+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f525c0b2910 msgr2=0x7f525c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.858+0000 7f527ce00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f525c0b2910 0x7f525c0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f52780799b0 tx=0x7f52640073d0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.858+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f527806c140 msgr2=0x7f527819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.858+0000 7f527ce00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f527806c140 0x7f527819bd80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5268000c00 tx=0x7f52680047b0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 shutdown_connections 2024-05-11T13:31:38.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f525c0b2910 0x7f525c0b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527810fde0 0x7f52781a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f527806c140 0x7f527819bd80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 --2- 172.21.15.63:0/282327919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5278104470 0x7f527819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:38.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 >> 172.21.15.63:0/282327919 conn(0x7f52780791b0 msgr2=0x7f52780ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:38.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 shutdown_connections 2024-05-11T13:31:38.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:38.859+0000 7f527ce00700 1 -- 172.21.15.63:0/282327919 wait complete. 2024-05-11T13:31:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:38 smithi165 ceph-mon[33235]: pgmap v469: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-11T13:31:39.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:38 smithi063 ceph-mon[29809]: pgmap v469: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-11T13:31:39.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:38 smithi071 ceph-mon[33293]: pgmap v469: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-11T13:31:39.335 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574969 got 55834574973 for osd.1 2024-05-11T13:31:39.335 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:40.093 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:39 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/282327919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-11T13:31:40.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:39 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/282327919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-11T13:31:40.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:39 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/282327919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 -- 172.21.15.63:0/2087686119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06bbf0 msgr2=0x7f623c1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 --2- 172.21.15.63:0/2087686119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06bbf0 0x7f623c1103e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f623400b0a0 tx=0x7f623400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 -- 172.21.15.63:0/2087686119 shutdown_connections 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 --2- 172.21.15.63:0/2087686119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06bbf0 0x7f623c1103e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 --2- 172.21.15.63:0/2087686119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623c1095c0 0x7f623c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 --2- 172.21.15.63:0/2087686119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c06b230 0x7f623c06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 -- 172.21.15.63:0/2087686119 >> 172.21.15.63:0/2087686119 conn(0x7f623c079400 msgr2=0x7f623c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 -- 172.21.15.63:0/2087686119 shutdown_connections 2024-05-11T13:31:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.719+0000 7f6244000700 1 -- 172.21.15.63:0/2087686119 wait complete. 2024-05-11T13:31:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.720+0000 7f6244000700 1 Processor -- start 2024-05-11T13:31:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6244000700 1 -- start start 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6244000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06b230 0x7f623c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6244000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c06bbf0 0x7f623c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6244000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623c1095c0 0x7f623c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6244000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f623c113600 con 0x7f623c06b230 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6244000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f623c113480 con 0x7f623c06bbf0 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6244000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f623c113780 con 0x7f623c1095c0 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.721+0000 7f6241600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06b230 0x7f623c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6241600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06b230 0x7f623c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47628/0 (socket says 172.21.15.63:47628) 2024-05-11T13:31:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6241600700 1 -- 172.21.15.63:0/1440093795 learned_addr learned my addr 172.21.15.63:0/1440093795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:40.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6240c00700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c06bbf0 0x7f623c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:40.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6242000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623c1095c0 0x7f623c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:40.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6241600700 1 -- 172.21.15.63:0/1440093795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623c1095c0 msgr2=0x7f623c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:40.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6241600700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623c1095c0 0x7f623c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:40.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6241600700 1 -- 172.21.15.63:0/1440093795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c06bbf0 msgr2=0x7f623c19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6241600700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c06bbf0 0x7f623c19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.722+0000 7f6241600700 1 -- 172.21.15.63:0/1440093795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c1a0d40 con 0x7f623c06b230 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.723+0000 7f6241600700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06b230 0x7f623c19bd30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f623000d980 tx=0x7f623000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.723+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230017070 con 0x7f623c06b230 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.723+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c1a1030 con 0x7f623c06b230 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.723+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6230004830 con 0x7f623c06b230 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.723+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300053a0 con 0x7f623c06b230 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.723+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c1a88b0 con 0x7f623c06b230 2024-05-11T13:31:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.724+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c06a240 con 0x7f623c06b230 2024-05-11T13:31:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.727+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f62300049e0 con 0x7f623c06b230 2024-05-11T13:31:40.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.728+0000 7f623a000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62240b2910 0x7f62240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:40.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.728+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f62300e26f0 con 0x7f623c06b230 2024-05-11T13:31:40.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.728+0000 7f6240c00700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62240b2910 0x7f62240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:40.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.729+0000 7f6240c00700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62240b2910 0x7f62240b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6228005950 tx=0x7f62280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:40.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.733+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6230020020 con 0x7f623c06b230 2024-05-11T13:31:40.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.836+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6230004d00 con 0x7f623c06b230 2024-05-11T13:31:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.996+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f623c1a8a80 con 0x7f623c06b230 2024-05-11T13:31:40.999 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:40 smithi063 ceph-mon[29809]: pgmap v470: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:40.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:40.998+0000 7f623a000700 1 -- 172.21.15.63:0/1440093795 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6230092b20 con 0x7f623c06b230 2024-05-11T13:31:40.999 INFO:teuthology.orchestra.run.smithi063.stdout:231928234062 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62240b2910 msgr2=0x7f62240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62240b2910 0x7f62240b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6228005950 tx=0x7f62280058e0 comp rx=0 tx=0).stop 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06b230 msgr2=0x7f623c19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06b230 0x7f623c19bd30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f623000d980 tx=0x7f623000dd40 comp rx=0 tx=0).stop 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 shutdown_connections 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62240b2910 0x7f62240b4dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623c1095c0 0x7f623c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:41.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c06b230 0x7f623c19bd30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:41.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 --2- 172.21.15.63:0/1440093795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c06bbf0 0x7f623c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:41.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.001+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 >> 172.21.15.63:0/1440093795 conn(0x7f623c079400 msgr2=0x7f623c10e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:41.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.002+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 shutdown_connections 2024-05-11T13:31:41.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:41.002+0000 7f6244000700 1 -- 172.21.15.63:0/1440093795 wait complete. 2024-05-11T13:31:41.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:40 smithi071 ceph-mon[33293]: pgmap v470: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:41.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:40 smithi165 ceph-mon[33235]: pgmap v470: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:31:42.055 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:41 smithi063 ceph-mon[29809]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-11T13:31:42.055 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:41 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1440093795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-11T13:31:42.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:41 smithi071 ceph-mon[33293]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-11T13:31:42.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:41 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1440093795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-11T13:31:42.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:41 smithi165 ceph-mon[33235]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-11T13:31:42.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:41 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1440093795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-11T13:31:42.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678067 got 304942678071 for osd.11 2024-05-11T13:31:42.614 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:42.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:42 smithi063 ceph-mon[29809]: pgmap v471: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-11T13:31:42.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:42 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/300700406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-11T13:31:43.153 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593815 got 184683593820 for osd.6 2024-05-11T13:31:43.153 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:43.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:42 smithi071 ceph-mon[33293]: pgmap v471: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-11T13:31:43.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:42 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/300700406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-11T13:31:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:42 smithi165 ceph-mon[33235]: pgmap v471: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-11T13:31:43.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/300700406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-11T13:31:43.860 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.456+0000 7fdcfa400700 1 -- 172.21.15.63:0/2558638086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf4106040 msgr2=0x7fdcf4112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.456+0000 7fdcfa400700 1 --2- 172.21.15.63:0/2558638086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf4106040 0x7fdcf4112550 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00b0a0 tx=0x7fdcec00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.456+0000 7fdcfa400700 1 -- 172.21.15.63:0/2558638086 shutdown_connections 2024-05-11T13:31:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.456+0000 7fdcfa400700 1 --2- 172.21.15.63:0/2558638086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf4106040 0x7fdcf4112550 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.456+0000 7fdcfa400700 1 --2- 172.21.15.63:0/2558638086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcf410b680 0x7fdcf410ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.456+0000 7fdcfa400700 1 --2- 172.21.15.63:0/2558638086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcf4105680 0x7fdcf4105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.456+0000 7fdcfa400700 1 -- 172.21.15.63:0/2558638086 >> 172.21.15.63:0/2558638086 conn(0x7fdcf4101170 msgr2=0x7fdcf4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.457+0000 7fdcfa400700 1 -- 172.21.15.63:0/2558638086 shutdown_connections 2024-05-11T13:31:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.457+0000 7fdcfa400700 1 -- 172.21.15.63:0/2558638086 wait complete. 2024-05-11T13:31:44.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 Processor -- start 2024-05-11T13:31:44.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 -- start start 2024-05-11T13:31:44.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcf4105680 0x7fdcf41a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcf4106040 0x7fdcf41a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf410b680 0x7fdcf419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4115780 con 0x7fdcf410b680 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4115600 con 0x7fdcf4106040 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.458+0000 7fdcfa400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4115900 con 0x7fdcf4105680 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf410b680 0x7fdcf419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf410b680 0x7fdcf419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47674/0 (socket says 172.21.15.63:47674) 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 -- 172.21.15.63:0/1662235302 learned_addr learned my addr 172.21.15.63:0/1662235302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9000700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcf4105680 0x7fdcf41a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf3e00700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcf4106040 0x7fdcf41a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 -- 172.21.15.63:0/1662235302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcf4105680 msgr2=0x7fdcf41a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:44.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcf4105680 0x7fdcf41a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 -- 172.21.15.63:0/1662235302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcf4106040 msgr2=0x7fdcf41a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:44.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcf4106040 0x7fdcf41a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9a00700 1 -- 172.21.15.63:0/1662235302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf419c160 con 0x7fdcf410b680 2024-05-11T13:31:44.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf9000700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcf4105680 0x7fdcf41a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:31:44.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.459+0000 7fdcf3e00700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcf4106040 0x7fdcf41a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:31:44.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.460+0000 7fdcf9a00700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf410b680 0x7fdcf419ba00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00b070 tx=0x7fdcec004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:44.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.460+0000 7fdcf1600700 1 -- 172.21.15.63:0/1662235302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec021070 con 0x7fdcf410b680 2024-05-11T13:31:44.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.460+0000 7fdcf1600700 1 -- 172.21.15.63:0/1662235302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcec007c00 con 0x7fdcf410b680 2024-05-11T13:31:44.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.460+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf419c3f0 con 0x7fdcf410b680 2024-05-11T13:31:44.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.460+0000 7fdcf1600700 1 -- 172.21.15.63:0/1662235302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec004780 con 0x7fdcf410b680 2024-05-11T13:31:44.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.460+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf41a86c0 con 0x7fdcf410b680 2024-05-11T13:31:44.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.462+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf406a240 con 0x7fdcf410b680 2024-05-11T13:31:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.464+0000 7fdcf1600700 1 -- 172.21.15.63:0/1662235302 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdcec003a00 con 0x7fdcf410b680 2024-05-11T13:31:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.465+0000 7fdcf1600700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcdc0b2900 0x7fdcdc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.465+0000 7fdcf1600700 1 -- 172.21.15.63:0/1662235302 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdcec00f020 con 0x7fdcf410b680 2024-05-11T13:31:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.465+0000 7fdcf9000700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcdc0b2900 0x7fdcdc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.466+0000 7fdcf9000700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcdc0b2900 0x7fdcdc0b4dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdce8005ea0 tx=0x7fdce8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.470+0000 7fdcf1600700 1 -- 172.21.15.63:0/1662235302 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcec02a020 con 0x7fdcf410b680 2024-05-11T13:31:44.556 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953572 got 137438953577 for osd.4 2024-05-11T13:31:44.556 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:44.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.713+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fdcf402e570 con 0x7fdcf410b680 2024-05-11T13:31:44.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.714+0000 7fdcf1600700 1 -- 172.21.15.63:0/1662235302 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdcec0ab4d0 con 0x7fdcf410b680 2024-05-11T13:31:44.716 INFO:teuthology.orchestra.run.smithi063.stdout:257698037831 2024-05-11T13:31:44.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcdc0b2900 msgr2=0x7fdcdc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:44.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcdc0b2900 0x7fdcdc0b4dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdce8005ea0 tx=0x7fdce8005e30 comp rx=0 tx=0).stop 2024-05-11T13:31:44.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf410b680 msgr2=0x7fdcf419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:44.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf410b680 0x7fdcf419ba00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00b070 tx=0x7fdcec004e00 comp rx=0 tx=0).stop 2024-05-11T13:31:44.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 shutdown_connections 2024-05-11T13:31:44.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcdc0b2900 0x7fdcdc0b4dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf410b680 0x7fdcf419ba00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcf4105680 0x7fdcf41a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 --2- 172.21.15.63:0/1662235302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcf4106040 0x7fdcf41a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:44.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 >> 172.21.15.63:0/1662235302 conn(0x7fdcf4101170 msgr2=0x7fdcf4101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:44.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.717+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 shutdown_connections 2024-05-11T13:31:44.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:44.718+0000 7fdcfa400700 1 -- 172.21.15.63:0/1662235302 wait complete. 2024-05-11T13:31:45.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:44 smithi063 ceph-mon[29809]: pgmap v472: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-11T13:31:45.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:44 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1662235302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-11T13:31:45.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:44 smithi071 ceph-mon[33293]: pgmap v472: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-11T13:31:45.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:44 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1662235302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-11T13:31:45.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:44 smithi165 ceph-mon[33235]: pgmap v472: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-11T13:31:45.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:44 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1662235302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-11T13:31:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 -- 172.21.15.63:0/4151580736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 msgr2=0x7fbbe410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 --2- 172.21.15.63:0/4151580736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe410fa10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009990 tx=0x7fbbcc009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 -- 172.21.15.63:0/4151580736 shutdown_connections 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 --2- 172.21.15.63:0/4151580736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbe410ff50 0x7fbbe4112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 --2- 172.21.15.63:0/4151580736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe4075780 0x7fbbe4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 --2- 172.21.15.63:0/4151580736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe410fa10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 -- 172.21.15.63:0/4151580736 >> 172.21.15.63:0/4151580736 conn(0x7fbbe407a6b0 msgr2=0x7fbbe407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 -- 172.21.15.63:0/4151580736 shutdown_connections 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.664+0000 7fbbe9200700 1 -- 172.21.15.63:0/4151580736 wait complete. 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 Processor -- start 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 -- start start 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe4075780 0x7fbbe419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbe410ff50 0x7fbbe41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41157e0 con 0x7fbbe40760a0 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4115660 con 0x7fbbe4075780 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.665+0000 7fbbe9200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4115960 con 0x7fbbe410ff50 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47690/0 (socket says 172.21.15.63:47690) 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 -- 172.21.15.63:0/2899451536 learned_addr learned my addr 172.21.15.63:0/2899451536 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbe1e00700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe4075780 0x7fbbe419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbe2800700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbe410ff50 0x7fbbe41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 -- 172.21.15.63:0/2899451536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbe410ff50 msgr2=0x7fbbe41a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbe410ff50 0x7fbbe41a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 -- 172.21.15.63:0/2899451536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe4075780 msgr2=0x7fbbe419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe4075780 0x7fbbe419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbd9400700 1 -- 172.21.15.63:0/2899451536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe41a0d60 con 0x7fbbe40760a0 2024-05-11T13:31:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.666+0000 7fbbe1e00700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe4075780 0x7fbbe419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.667+0000 7fbbd9400700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe419c290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc000c00 tx=0x7fbbcc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.667+0000 7fbbdb400700 1 -- 172.21.15.63:0/2899451536 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc027070 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.667+0000 7fbbdb400700 1 -- 172.21.15.63:0/2899451536 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbcc00fd40 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.667+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe41a0ff0 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.667+0000 7fbbdb400700 1 -- 172.21.15.63:0/2899451536 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc022ca0 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.667+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe41a88d0 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.668+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe406a240 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.671+0000 7fbbdb400700 1 -- 172.21.15.63:0/2899451536 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbbcc02a070 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.672+0000 7fbbdb400700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbbc80b2900 0x7fbbc80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.672+0000 7fbbdb400700 1 -- 172.21.15.63:0/2899451536 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbbcc0e36a0 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.672+0000 7fbbe1e00700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbbc80b2900 0x7fbbc80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.673+0000 7fbbe1e00700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbbc80b2900 0x7fbbc80b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009dd0 tx=0x7fbbd4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.675+0000 7fbbdb400700 1 -- 172.21.15.63:0/2899451536 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbcc031080 con 0x7fbbe40760a0 2024-05-11T13:31:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.934+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbbe402e570 con 0x7fbbe40760a0 2024-05-11T13:31:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.934+0000 7fbbdb400700 1 -- 172.21.15.63:0/2899451536 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbbcc013070 con 0x7fbbe40760a0 2024-05-11T13:31:46.252 INFO:teuthology.orchestra.run.smithi063.stdout:111669149811 2024-05-11T13:31:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.937+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbbc80b2900 msgr2=0x7fbbc80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.937+0000 7fbbe9200700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbbc80b2900 0x7fbbc80b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009dd0 tx=0x7fbbd4009040 comp rx=0 tx=0).stop 2024-05-11T13:31:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.937+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 msgr2=0x7fbbe419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.937+0000 7fbbe9200700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe419c290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc000c00 tx=0x7fbbcc005ef0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 shutdown_connections 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fbbc80b2900 0x7fbbc80b4dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbe410ff50 0x7fbbe41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe4075780 0x7fbbe419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 --2- 172.21.15.63:0/2899451536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbe40760a0 0x7fbbe419c290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 >> 172.21.15.63:0/2899451536 conn(0x7fbbe407a6b0 msgr2=0x7fbbe4100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 shutdown_connections 2024-05-11T13:31:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:45.938+0000 7fbbe9200700 1 -- 172.21.15.63:0/2899451536 wait complete. 2024-05-11T13:31:46.898 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:47.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:46 smithi063 ceph-mon[29809]: pgmap v473: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-11T13:31:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:46 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2899451536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-11T13:31:47.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:46 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:31:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:46 smithi071 ceph-mon[33293]: pgmap v473: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-11T13:31:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:46 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2899451536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-11T13:31:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:46 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:31:47.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234057 got 231928234062 for osd.8 2024-05-11T13:31:47.242 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:47.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:46 smithi165 ceph-mon[33235]: pgmap v473: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-11T13:31:47.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:46 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2899451536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-11T13:31:47.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:46 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 -- 172.21.15.63:0/2518363861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668104700 msgr2=0x7fd668104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 --2- 172.21.15.63:0/2518363861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668104700 0x7fd668104ae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd658009990 tx=0x7fd658009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 -- 172.21.15.63:0/2518363861 shutdown_connections 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 --2- 172.21.15.63:0/2518363861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66810dae0 0x7fd66810ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 --2- 172.21.15.63:0/2518363861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668104700 0x7fd668104ae0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 --2- 172.21.15.63:0/2518363861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd668105020 0x7fd66810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 -- 172.21.15.63:0/2518363861 >> 172.21.15.63:0/2518363861 conn(0x7fd6680fde70 msgr2=0x7fd668100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 -- 172.21.15.63:0/2518363861 shutdown_connections 2024-05-11T13:31:47.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.733+0000 7fd667e00700 1 -- 172.21.15.63:0/2518363861 wait complete. 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.734+0000 7fd667e00700 1 Processor -- start 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.734+0000 7fd667e00700 1 -- start start 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd667e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd668104700 0x7fd6681a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd667e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668105020 0x7fd6681a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd667e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66810dae0 0x7fd66819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd667e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd668113590 con 0x7fd668105020 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd667e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd668113410 con 0x7fd66810dae0 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd667e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd668113710 con 0x7fd668104700 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd666000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668105020 0x7fd6681a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd666000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668105020 0x7fd6681a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47720/0 (socket says 172.21.15.63:47720) 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.735+0000 7fd666000700 1 -- 172.21.15.63:0/2183471963 learned_addr learned my addr 172.21.15.63:0/2183471963 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd666a00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd668104700 0x7fd6681a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:47.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd667400700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66810dae0 0x7fd66819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd666000700 1 -- 172.21.15.63:0/2183471963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd668104700 msgr2=0x7fd6681a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd666000700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd668104700 0x7fd6681a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd666000700 1 -- 172.21.15.63:0/2183471963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66810dae0 msgr2=0x7fd66819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd666000700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66810dae0 0x7fd66819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd666000700 1 -- 172.21.15.63:0/2183471963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66819c410 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd667400700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66810dae0 0x7fd66819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd666000700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668105020 0x7fd6681a1fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd65400d950 tx=0x7fd65400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd65f400700 1 -- 172.21.15.63:0/2183471963 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654017070 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66819c700 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.736+0000 7fd65f400700 1 -- 172.21.15.63:0/2183471963 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd65400de30 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.737+0000 7fd65f400700 1 -- 172.21.15.63:0/2183471963 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654005310 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.737+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681a89e0 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.738+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66806ba50 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.740+0000 7fd65f400700 1 -- 172.21.15.63:0/2183471963 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd654004750 con 0x7fd668105020 2024-05-11T13:31:47.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.742+0000 7fd65f400700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd64c0b2880 0x7fd64c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.742+0000 7fd65f400700 1 -- 172.21.15.63:0/2183471963 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd6540e2330 con 0x7fd668105020 2024-05-11T13:31:47.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.742+0000 7fd666a00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd64c0b2880 0x7fd64c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:47.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.743+0000 7fd666a00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd64c0b2880 0x7fd64c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd65800b5c0 tx=0x7fd65801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:47.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:46.745+0000 7fd65f400700 1 -- 172.21.15.63:0/2183471963 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd6540a9c40 con 0x7fd668105020 2024-05-11T13:31:47.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.020+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd66819ca00 con 0x7fd668105020 2024-05-11T13:31:47.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.020+0000 7fd65f400700 1 -- 172.21.15.63:0/2183471963 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd6540a9540 con 0x7fd668105020 2024-05-11T13:31:47.325 INFO:teuthology.orchestra.run.smithi063.stdout:279172874303 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd64c0b2880 msgr2=0x7fd64c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd64c0b2880 0x7fd64c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd65800b5c0 tx=0x7fd65801a040 comp rx=0 tx=0).stop 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668105020 msgr2=0x7fd6681a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668105020 0x7fd6681a1fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd65400d950 tx=0x7fd65400dd10 comp rx=0 tx=0).stop 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 shutdown_connections 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd64c0b2880 0x7fd64c0b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66810dae0 0x7fd66819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd668104700 0x7fd6681a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 --2- 172.21.15.63:0/2183471963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd668105020 0x7fd6681a1fa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 >> 172.21.15.63:0/2183471963 conn(0x7fd6680fde70 msgr2=0x7fd668100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 shutdown_connections 2024-05-11T13:31:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:47.024+0000 7fd667e00700 1 -- 172.21.15.63:0/2183471963 wait complete. 2024-05-11T13:31:47.689 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037825 got 257698037831 for osd.9 2024-05-11T13:31:47.689 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:47.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.392+0000 7fe157200700 1 -- 172.21.15.63:0/3019980365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 msgr2=0x7fe15810ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.392+0000 7fe157200700 1 --2- 172.21.15.63:0/3019980365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15810ba60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe148009990 tx=0x7fe148009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.393+0000 7fe157200700 1 -- 172.21.15.63:0/3019980365 shutdown_connections 2024-05-11T13:31:47.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.393+0000 7fe157200700 1 --2- 172.21.15.63:0/3019980365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe158106040 0x7fe158112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.393+0000 7fe157200700 1 --2- 172.21.15.63:0/3019980365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15810ba60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.393+0000 7fe157200700 1 --2- 172.21.15.63:0/3019980365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe158105680 0x7fe158105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.393+0000 7fe157200700 1 -- 172.21.15.63:0/3019980365 >> 172.21.15.63:0/3019980365 conn(0x7fe158101170 msgr2=0x7fe158103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.393+0000 7fe157200700 1 -- 172.21.15.63:0/3019980365 shutdown_connections 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.393+0000 7fe157200700 1 -- 172.21.15.63:0/3019980365 wait complete. 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.394+0000 7fe157200700 1 Processor -- start 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.394+0000 7fe157200700 1 -- start start 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe157200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe158105680 0x7fe1581a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe157200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe158106040 0x7fe1581a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe157200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe157200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe158115710 con 0x7fe15810b680 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe157200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe158115590 con 0x7fe158106040 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe157200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe158115890 con 0x7fe158105680 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe156800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:47.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe156800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47652/0 (socket says 172.21.15.63:47652) 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.395+0000 7fe156800700 1 -- 172.21.15.63:0/300700406 learned_addr learned my addr 172.21.15.63:0/300700406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe156800700 1 -- 172.21.15.63:0/300700406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe158105680 msgr2=0x7fe1581a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe156800700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe158105680 0x7fe1581a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe156800700 1 -- 172.21.15.63:0/300700406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe158106040 msgr2=0x7fe1581a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe156800700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe158106040 0x7fe1581a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe156800700 1 -- 172.21.15.63:0/300700406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15819c1f0 con 0x7fe15810b680 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe156800700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15819ba90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe150007ee0 tx=0x7fe15000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe14ea00700 1 -- 172.21.15.63:0/300700406 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15000dbd0 con 0x7fe15810b680 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe14ea00700 1 -- 172.21.15.63:0/300700406 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe15000dd70 con 0x7fe15810b680 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe14ea00700 1 -- 172.21.15.63:0/300700406 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500058d0 con 0x7fe15810b680 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15819c4e0 con 0x7fe15810b680 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.396+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a8750 con 0x7fe15810b680 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.397+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15806a240 con 0x7fe15810b680 2024-05-11T13:31:47.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.400+0000 7fe14ea00700 1 -- 172.21.15.63:0/300700406 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe15001a020 con 0x7fe15810b680 2024-05-11T13:31:47.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.402+0000 7fe14ea00700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe13c0b2880 0x7fe13c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:47.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.402+0000 7fe14ea00700 1 -- 172.21.15.63:0/300700406 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe1500e1e40 con 0x7fe15810b680 2024-05-11T13:31:47.894 INFO:teuthology.orchestra.run.smithi063.stdout:34359738501 2024-05-11T13:31:47.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.402+0000 7fe155e00700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe13c0b2880 0x7fe13c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:47.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.403+0000 7fe155e00700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe13c0b2880 0x7fe13c0b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1480098e0 tx=0x7fe148005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.406+0000 7fe14ea00700 1 -- 172.21.15.63:0/300700406 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe150010070 con 0x7fe15810b680 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.641+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe15802e570 con 0x7fe15810b680 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.642+0000 7fe14ea00700 1 -- 172.21.15.63:0/300700406 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1500a9750 con 0x7fe15810b680 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.645+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe13c0b2880 msgr2=0x7fe13c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.645+0000 7fe157200700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe13c0b2880 0x7fe13c0b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1480098e0 tx=0x7fe148005fb0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.645+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 msgr2=0x7fe15819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.645+0000 7fe157200700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15819ba90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe150007ee0 tx=0x7fe15000d3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.645+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 shutdown_connections 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.645+0000 7fe157200700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe13c0b2880 0x7fe13c0b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.645+0000 7fe157200700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe15810b680 0x7fe15819ba90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.646+0000 7fe157200700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe158105680 0x7fe1581a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.646+0000 7fe157200700 1 --2- 172.21.15.63:0/300700406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe158106040 0x7fe1581a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.646+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 >> 172.21.15.63:0/300700406 conn(0x7fe158101170 msgr2=0x7fe1581019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.646+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 shutdown_connections 2024-05-11T13:31:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:42.646+0000 7fe157200700 1 -- 172.21.15.63:0/300700406 wait complete. 2024-05-11T13:31:47.896 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:47 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2183471963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-11T13:31:48.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:47 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2183471963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-11T13:31:48.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:47 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2183471963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-11T13:31:48.767 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874299 got 279172874303 for osd.10 2024-05-11T13:31:48.767 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:49.068 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:48 smithi063 ceph-mon[29809]: pgmap v474: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-11T13:31:49.068 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:31:49.068 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:31:49.068 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:31:49.131 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738496 got 34359738501 for osd.0 2024-05-11T13:31:49.131 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:49.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:48 smithi071 ceph-mon[33293]: pgmap v474: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-11T13:31:49.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:31:49.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:31:49.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:31:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:48 smithi165 ceph-mon[33235]: pgmap v474: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-11T13:31:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:31:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:31:49.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:31:49.611 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149803 got 111669149811 for osd.3 2024-05-11T13:31:49.612 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:50.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 -- 172.21.15.63:0/849889367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd100760a0 msgr2=0x7fdd10114240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:50.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 --2- 172.21.15.63:0/849889367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd100760a0 0x7fdd10114240 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009990 tx=0x7fdcfc009ca0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 -- 172.21.15.63:0/849889367 shutdown_connections 2024-05-11T13:31:50.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 --2- 172.21.15.63:0/849889367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd10114780 0x7fdd10116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 --2- 172.21.15.63:0/849889367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd10075780 0x7fdd10075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 --2- 172.21.15.63:0/849889367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd100760a0 0x7fdd10114240 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 -- 172.21.15.63:0/849889367 >> 172.21.15.63:0/849889367 conn(0x7fdd1007a6b0 msgr2=0x7fdd1007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 -- 172.21.15.63:0/849889367 shutdown_connections 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.364+0000 7fdd18600700 1 -- 172.21.15.63:0/849889367 wait complete. 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.365+0000 7fdd18600700 1 Processor -- start 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd18600700 1 -- start start 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd18600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd10075780 0x7fdd101a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd18600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd100760a0 0x7fdd101a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd18600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd10114780 0x7fdd101a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd18600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10119c90 con 0x7fdd10114780 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd18600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10119b10 con 0x7fdd10075780 2024-05-11T13:31:50.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd18600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10119e10 con 0x7fdd100760a0 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd16600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd10114780 0x7fdd101a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd16600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd10114780 0x7fdd101a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55124/0 (socket says 172.21.15.63:55124) 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.366+0000 7fdd16600700 1 -- 172.21.15.63:0/438389962 learned_addr learned my addr 172.21.15.63:0/438389962 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd15200700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd100760a0 0x7fdd101a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd15c00700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd10075780 0x7fdd101a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stdout:158913790053 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd16600700 1 -- 172.21.15.63:0/438389962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd100760a0 msgr2=0x7fdd101a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd16600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd100760a0 0x7fdd101a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd16600700 1 -- 172.21.15.63:0/438389962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd10075780 msgr2=0x7fdd101a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd16600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd10075780 0x7fdd101a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd16600700 1 -- 172.21.15.63:0/438389962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd101a51f0 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.367+0000 7fdd16600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd10114780 0x7fdd101a4ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd08007f10 tx=0x7fdd0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.368+0000 7fdd0e000700 1 -- 172.21.15.63:0/438389962 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08009e20 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.368+0000 7fdd0e000700 1 -- 172.21.15.63:0/438389962 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd0800dbd0 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.368+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd101a54e0 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.368+0000 7fdd0e000700 1 -- 172.21.15.63:0/438389962 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080058d0 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.368+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101acd60 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.369+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1006a240 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.371+0000 7fdd0e000700 1 -- 172.21.15.63:0/438389962 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdd0801a020 con 0x7fdd10114780 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.373+0000 7fdd0e000700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcf80b2900 0x7fdcf80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.373+0000 7fdd0e000700 1 -- 172.21.15.63:0/438389962 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdd080e1cc0 con 0x7fdd10114780 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.373+0000 7fdd15c00700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcf80b2900 0x7fdcf80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.374+0000 7fdd15c00700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcf80b2900 0x7fdcf80b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005950 tx=0x7fdd040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.377+0000 7fdd0e000700 1 -- 172.21.15.63:0/438389962 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd080e8080 con 0x7fdd10114780 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.632+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fdd101acf30 con 0x7fdd10114780 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.633+0000 7fdd0e000700 1 -- 172.21.15.63:0/438389962 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdd08010070 con 0x7fdd10114780 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcf80b2900 msgr2=0x7fdcf80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcf80b2900 0x7fdcf80b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005950 tx=0x7fdd040058e0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd10114780 msgr2=0x7fdd101a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd10114780 0x7fdd101a4ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd08007f10 tx=0x7fdd0800d3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 shutdown_connections 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdcf80b2900 0x7fdcf80b4dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd10114780 0x7fdd101a4ab0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd10075780 0x7fdd101a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 --2- 172.21.15.63:0/438389962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd100760a0 0x7fdd101a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 >> 172.21.15.63:0/438389962 conn(0x7fdd1007a6b0 msgr2=0x7fdd10110220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 shutdown_connections 2024-05-11T13:31:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:50.636+0000 7fdd18600700 1 -- 172.21.15.63:0/438389962 wait complete. 2024-05-11T13:31:51.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:50 smithi063 ceph-mon[29809]: pgmap v475: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-11T13:31:51.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:50 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/438389962' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-11T13:31:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:50 smithi071 ceph-mon[33293]: pgmap v475: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-11T13:31:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:50 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/438389962' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-11T13:31:51.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:50 smithi165 ceph-mon[33235]: pgmap v475: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-11T13:31:51.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:50 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/438389962' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-11T13:31:52.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790047 got 158913790053 for osd.5 2024-05-11T13:31:52.597 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:53.009 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:52 smithi063 ceph-mon[29809]: pgmap v476: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-11T13:31:53.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:52 smithi071 ceph-mon[33293]: pgmap v476: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-11T13:31:53.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 -- 172.21.15.63:0/154887408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c8010eec0 msgr2=0x7f9c80116a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 --2- 172.21.15.63:0/154887408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c8010eec0 0x7f9c80116a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800b0d0 tx=0x7f9c7800b3e0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 -- 172.21.15.63:0/154887408 shutdown_connections 2024-05-11T13:31:53.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 --2- 172.21.15.63:0/154887408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c8010eec0 0x7f9c80116a90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 --2- 172.21.15.63:0/154887408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c80109de0 0x7f9c8010a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 --2- 172.21.15.63:0/154887408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c8010a790 0x7f9c8010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 -- 172.21.15.63:0/154887408 >> 172.21.15.63:0/154887408 conn(0x7f9c80076d20 msgr2=0x7f9c80108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:53.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 -- 172.21.15.63:0/154887408 shutdown_connections 2024-05-11T13:31:53.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.263+0000 7f9c88600700 1 -- 172.21.15.63:0/154887408 wait complete. 2024-05-11T13:31:53.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 Processor -- start 2024-05-11T13:31:53.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 -- start start 2024-05-11T13:31:53.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c80109de0 0x7f9c801a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c8010a790 0x7f9c801a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c8010eec0 0x7f9c801a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80119c20 con 0x7f9c80109de0 2024-05-11T13:31:53.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80119aa0 con 0x7f9c8010eec0 2024-05-11T13:31:53.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.265+0000 7f9c88600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80119da0 con 0x7f9c8010a790 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c80109de0 0x7f9c801a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c80109de0 0x7f9c801a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55156/0 (socket says 172.21.15.63:55156) 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 -- 172.21.15.63:0/3016263292 learned_addr learned my addr 172.21.15.63:0/3016263292 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c86600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c8010eec0 0x7f9c801a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85200700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c8010a790 0x7f9c801a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 -- 172.21.15.63:0/3016263292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c8010a790 msgr2=0x7f9c801a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c8010a790 0x7f9c801a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 -- 172.21.15.63:0/3016263292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c8010eec0 msgr2=0x7f9c801a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c8010eec0 0x7f9c801a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c85c00700 1 -- 172.21.15.63:0/3016263292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c801a5100 con 0x7f9c80109de0 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.266+0000 7f9c86600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c8010eec0 0x7f9c801a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.267+0000 7f9c85c00700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c80109de0 0x7f9c801a00f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400d950 tx=0x7f9c7400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.267+0000 7f9c7e000700 1 -- 172.21.15.63:0/3016263292 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74016070 con 0x7f9c80109de0 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.267+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c801a53f0 con 0x7f9c80109de0 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.267+0000 7f9c7e000700 1 -- 172.21.15.63:0/3016263292 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c74010460 con 0x7f9c80109de0 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.267+0000 7f9c7e000700 1 -- 172.21.15.63:0/3016263292 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400f620 con 0x7f9c80109de0 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.267+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801acc70 con 0x7f9c80109de0 2024-05-11T13:31:53.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.268+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8006a240 con 0x7f9c80109de0 2024-05-11T13:31:53.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.271+0000 7f9c7e000700 1 -- 172.21.15.63:0/3016263292 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c74019070 con 0x7f9c80109de0 2024-05-11T13:31:53.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.273+0000 7f9c7e000700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c680b2880 0x7f9c680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.274+0000 7f9c7e000700 1 -- 172.21.15.63:0/3016263292 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9c740e2230 con 0x7f9c80109de0 2024-05-11T13:31:53.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.274+0000 7f9c85200700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c680b2880 0x7f9c680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.274+0000 7f9c85200700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c680b2880 0x7f9c680b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c80110240 tx=0x7f9c6c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:53.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.276+0000 7f9c7e000700 1 -- 172.21.15.63:0/3016263292 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c740a9b40 con 0x7f9c80109de0 2024-05-11T13:31:53.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:52 smithi165 ceph-mon[33235]: pgmap v476: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-11T13:31:53.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 -- 172.21.15.63:0/2676349087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102e10 msgr2=0x7f070410f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 --2- 172.21.15.63:0/2676349087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102e10 0x7f070410f300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00b0a0 tx=0x7f06fc00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 -- 172.21.15.63:0/2676349087 shutdown_connections 2024-05-11T13:31:53.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 --2- 172.21.15.63:0/2676349087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102e10 0x7f070410f300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 --2- 172.21.15.63:0/2676349087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0704108530 0x7f0704108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 --2- 172.21.15.63:0/2676349087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0704102450 0x7f07041028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 -- 172.21.15.63:0/2676349087 >> 172.21.15.63:0/2676349087 conn(0x7f0704079400 msgr2=0x7f0704079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:53.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 -- 172.21.15.63:0/2676349087 shutdown_connections 2024-05-11T13:31:53.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.507+0000 7f070b200700 1 -- 172.21.15.63:0/2676349087 wait complete. 2024-05-11T13:31:53.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.508+0000 7f070b200700 1 Processor -- start 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f070b200700 1 -- start start 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f070b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102450 0x7f07041a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f070b200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0704102e10 0x7f07041a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f070b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0704108530 0x7f070419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f070b200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f070406bd30 con 0x7f0704102450 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f070b200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f070406bbb0 con 0x7f0704108530 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f070b200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f070406beb0 con 0x7f0704102e10 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f0703e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102450 0x7f07041a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f0703e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102450 0x7f07041a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55174/0 (socket says 172.21.15.63:55174) 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.509+0000 7f0703e00700 1 -- 172.21.15.63:0/1311132193 learned_addr learned my addr 172.21.15.63:0/1311132193 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0703400700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0704102e10 0x7f07041a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0709200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0704108530 0x7f070419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0703e00700 1 -- 172.21.15.63:0/1311132193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0704102e10 msgr2=0x7f07041a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0703e00700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0704102e10 0x7f07041a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0703e00700 1 -- 172.21.15.63:0/1311132193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0704108530 msgr2=0x7f070419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0703e00700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0704108530 0x7f070419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0703e00700 1 -- 172.21.15.63:0/1311132193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070419c230 con 0x7f0704102450 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0709200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0704108530 0x7f070419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.510+0000 7f0703e00700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102450 0x7f07041a1910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c9b0 tx=0x7f06f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:53.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.511+0000 7f0700c00700 1 -- 172.21.15.63:0/1311132193 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007b00 con 0x7f0704102450 2024-05-11T13:31:53.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.511+0000 7f0700c00700 1 -- 172.21.15.63:0/1311132193 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06f4007ca0 con 0x7f0704102450 2024-05-11T13:31:53.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.511+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070419c4c0 con 0x7f0704102450 2024-05-11T13:31:53.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.511+0000 7f0700c00700 1 -- 172.21.15.63:0/1311132193 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4018660 con 0x7f0704102450 2024-05-11T13:31:53.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.511+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07041a87e0 con 0x7f0704102450 2024-05-11T13:31:53.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.513+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070406a240 con 0x7f0704102450 2024-05-11T13:31:53.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.514+0000 7f0700c00700 1 -- 172.21.15.63:0/1311132193 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f06f400f450 con 0x7f0704102450 2024-05-11T13:31:53.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.517+0000 7f0700c00700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06e80b2900 0x7f06e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:53.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.517+0000 7f0700c00700 1 -- 172.21.15.63:0/1311132193 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f06f40e2660 con 0x7f0704102450 2024-05-11T13:31:53.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.517+0000 7f0703400700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06e80b2900 0x7f06e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:53.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.518+0000 7f0703400700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06e80b2900 0x7f06e80b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06f0006fd0 tx=0x7f06f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:53.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.522+0000 7f0700c00700 1 -- 172.21.15.63:0/1311132193 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06f4014030 con 0x7f0704102450 2024-05-11T13:31:53.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.522+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9c8002e570 con 0x7f9c80109de0 2024-05-11T13:31:53.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.523+0000 7f9c7e000700 1 -- 172.21.15.63:0/3016263292 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c740a9440 con 0x7f9c80109de0 2024-05-11T13:31:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:77309411450 2024-05-11T13:31:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c680b2880 msgr2=0x7f9c680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c680b2880 0x7f9c680b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c80110240 tx=0x7f9c6c00b590 comp rx=0 tx=0).stop 2024-05-11T13:31:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c80109de0 msgr2=0x7f9c801a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c80109de0 0x7f9c801a00f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400d950 tx=0x7f9c7400dd10 comp rx=0 tx=0).stop 2024-05-11T13:31:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 shutdown_connections 2024-05-11T13:31:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f9c680b2880 0x7f9c680b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c8010eec0 0x7f9c801a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c80109de0 0x7f9c801a00f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 --2- 172.21.15.63:0/3016263292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c8010a790 0x7f9c801a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.526+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 >> 172.21.15.63:0/3016263292 conn(0x7f9c80076d20 msgr2=0x7f9c800772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.527+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 shutdown_connections 2024-05-11T13:31:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.527+0000 7f9c88600700 1 -- 172.21.15.63:0/3016263292 wait complete. 2024-05-11T13:31:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.773+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f07041a8a10 con 0x7f0704102450 2024-05-11T13:31:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.774+0000 7f0700c00700 1 -- 172.21.15.63:0/1311132193 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f06f40a9f70 con 0x7f0704102450 2024-05-11T13:31:53.775 INFO:teuthology.orchestra.run.smithi063.stdout:206158430295 2024-05-11T13:31:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06e80b2900 msgr2=0x7f06e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06e80b2900 0x7f06e80b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06f0006fd0 tx=0x7f06f0008040 comp rx=0 tx=0).stop 2024-05-11T13:31:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102450 msgr2=0x7f07041a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102450 0x7f07041a1910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c9b0 tx=0x7f06f400cd70 comp rx=0 tx=0).stop 2024-05-11T13:31:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 shutdown_connections 2024-05-11T13:31:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f06e80b2900 0x7f06e80b4dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0704108530 0x7f070419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0704102450 0x7f07041a1910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 --2- 172.21.15.63:0/1311132193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0704102e10 0x7f07041a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.777+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 >> 172.21.15.63:0/1311132193 conn(0x7f0704079400 msgr2=0x7f07041002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:53.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.778+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 shutdown_connections 2024-05-11T13:31:53.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:53.778+0000 7f070b200700 1 -- 172.21.15.63:0/1311132193 wait complete. 2024-05-11T13:31:54.128 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:53 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3016263292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-11T13:31:54.129 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:53 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1311132193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-11T13:31:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3016263292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-11T13:31:54.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1311132193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-11T13:31:54.218 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:53 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3016263292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-11T13:31:54.218 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:53 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1311132193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-11T13:31:54.730 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411444 got 77309411450 for osd.2 2024-05-11T13:31:54.730 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:55.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430290 got 206158430295 for osd.7 2024-05-11T13:31:55.079 DEBUG:teuthology.parallel:result is None 2024-05-11T13:31:55.079 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-11T13:31:55.079 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T13:31:55.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:54 smithi063 ceph-mon[29809]: pgmap v477: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-11T13:31:55.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:54 smithi071 ceph-mon[33293]: pgmap v477: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-11T13:31:55.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:54 smithi165 ceph-mon[33235]: pgmap v477: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-11T13:31:55.560 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:31:57.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:56 smithi063 ceph-mon[29809]: pgmap v478: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-11T13:31:57.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:56 smithi071 ceph-mon[33293]: pgmap v478: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-11T13:31:57.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:56 smithi165 ceph-mon[33235]: pgmap v478: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-11T13:31:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 -- 172.21.15.63:0/4092503944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 msgr2=0x7f64bc112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 --2- 172.21.15.63:0/4092503944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc112640 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f64b400b0a0 tx=0x7f64b400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:31:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 -- 172.21.15.63:0/4092503944 shutdown_connections 2024-05-11T13:31:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 --2- 172.21.15.63:0/4092503944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc112640 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 --2- 172.21.15.63:0/4092503944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64bc10b770 0x7f64bc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 --2- 172.21.15.63:0/4092503944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64bc105770 0x7f64bc105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 -- 172.21.15.63:0/4092503944 >> 172.21.15.63:0/4092503944 conn(0x7f64bc101260 msgr2=0x7f64bc103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 -- 172.21.15.63:0/4092503944 shutdown_connections 2024-05-11T13:31:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.838+0000 7f64c3800700 1 -- 172.21.15.63:0/4092503944 wait complete. 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.839+0000 7f64c3800700 1 Processor -- start 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.839+0000 7f64c3800700 1 -- start start 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c3800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64bc105770 0x7f64bc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c3800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c3800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64bc10b770 0x7f64bc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c3800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc115860 con 0x7f64bc106130 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c3800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1156e0 con 0x7f64bc105770 2024-05-11T13:31:57.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c3800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1159e0 con 0x7f64bc10b770 2024-05-11T13:31:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64bbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55178/0 (socket says 172.21.15.63:55178) 2024-05-11T13:31:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64bbe00700 1 -- 172.21.15.63:0/2364119803 learned_addr learned my addr 172.21.15.63:0/2364119803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:31:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c0e00700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64bc105770 0x7f64bc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.840+0000 7f64c1800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64bc10b770 0x7f64bc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64bbe00700 1 -- 172.21.15.63:0/2364119803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64bc10b770 msgr2=0x7f64bc1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64bbe00700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64bc10b770 0x7f64bc1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64bbe00700 1 -- 172.21.15.63:0/2364119803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64bc105770 msgr2=0x7f64bc19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64bbe00700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64bc105770 0x7f64bc19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64bbe00700 1 -- 172.21.15.63:0/2364119803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc1a0d20 con 0x7f64bc106130 2024-05-11T13:31:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64c0e00700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64bc105770 0x7f64bc19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:31:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64c1800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64bc10b770 0x7f64bc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:31:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64bbe00700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc19c250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f64a800d980 tx=0x7f64a800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64b9600700 1 -- 172.21.15.63:0/2364119803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8017070 con 0x7f64bc106130 2024-05-11T13:31:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64b9600700 1 -- 172.21.15.63:0/2364119803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64a8004830 con 0x7f64bc106130 2024-05-11T13:31:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64b9600700 1 -- 172.21.15.63:0/2364119803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80053a0 con 0x7f64bc106130 2024-05-11T13:31:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.841+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc1a1010 con 0x7f64bc106130 2024-05-11T13:31:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.842+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc1a8890 con 0x7f64bc106130 2024-05-11T13:31:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.843+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc06a240 con 0x7f64bc106130 2024-05-11T13:31:57.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.845+0000 7f64b9600700 1 -- 172.21.15.63:0/2364119803 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f64a80049e0 con 0x7f64bc106130 2024-05-11T13:31:57.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.846+0000 7f64b9600700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f64a40b2900 0x7f64a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:31:57.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.847+0000 7f64b9600700 1 -- 172.21.15.63:0/2364119803 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f64a80e2540 con 0x7f64bc106130 2024-05-11T13:31:57.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.847+0000 7f64c0e00700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f64a40b2900 0x7f64a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:31:57.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.848+0000 7f64c0e00700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f64a40b2900 0x7f64a40b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f64b0005e10 tx=0x7f64b0005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:31:57.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:57.851+0000 7f64b9600700 1 -- 172.21.15.63:0/2364119803 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f64a8020020 con 0x7f64bc106130 2024-05-11T13:31:58.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.088+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64bc063990 con 0x7f64a40b2900 2024-05-11T13:31:58.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.114+0000 7f64b9600700 1 -- 172.21.15.63:0/2364119803 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340517 (secure 0 0 0) 0x7f64bc063990 con 0x7f64a40b2900 2024-05-11T13:31:58.121 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:31:58.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f64a40b2900 msgr2=0x7f64a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:58.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f64a40b2900 0x7f64a40b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f64b0005e10 tx=0x7f64b0005d60 comp rx=0 tx=0).stop 2024-05-11T13:31:58.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 msgr2=0x7f64bc19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:31:58.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc19c250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f64a800d980 tx=0x7f64a800dd40 comp rx=0 tx=0).stop 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 shutdown_connections 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f64a40b2900 0x7f64a40b4dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64bc10b770 0x7f64bc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64bc105770 0x7f64bc19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 --2- 172.21.15.63:0/2364119803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64bc106130 0x7f64bc19c250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 >> 172.21.15.63:0/2364119803 conn(0x7f64bc101260 msgr2=0x7f64bc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 shutdown_connections 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:31:58.122+0000 7f64c3800700 1 -- 172.21.15.63:0/2364119803 wait complete. 2024-05-11T13:31:58.125 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-11T13:31:58.992 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-05-11T13:31:57.635869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660994,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1294,"num_read_kb":757,"num_write":382,"num_write_kb":2301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380128,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561696,"statfs":{"total":1151940427776,"available":1151551176704,"internally_reserved":0,"allocated":14680064,"data_stored":8176736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005669"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546529+0000","last_change":"2024-05-11T13:28:35.223417+0000","last_active":"2024-05-11T13:31:39.546529+0000","last_peered":"2024-05-11T13:31:39.546529+0000","last_clean":"2024-05-11T13:31:39.546529+0000","last_became_active":"2024-05-11T13:28:35.222862+0000","last_became_peered":"2024-05-11T13:28:35.222862+0000","last_unstale":"2024-05-11T13:31:39.546529+0000","last_undegraded":"2024-05-11T13:31:39.546529+0000","last_fullsized":"2024-05-11T13:31:39.546529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:20.259043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.525965+0000","last_change":"2024-05-11T13:28:35.222289+0000","last_active":"2024-05-11T13:31:39.525965+0000","last_peered":"2024-05-11T13:31:39.525965+0000","last_clean":"2024-05-11T13:31:39.525965+0000","last_became_active":"2024-05-11T13:28:35.221366+0000","last_became_peered":"2024-05-11T13:28:35.221366+0000","last_unstale":"2024-05-11T13:31:39.525965+0000","last_undegraded":"2024-05-11T13:31:39.525965+0000","last_fullsized":"2024-05-11T13:31:39.525965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:02.648140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549679+0000","last_change":"2024-05-11T13:28:35.221271+0000","last_active":"2024-05-11T13:31:39.549679+0000","last_peered":"2024-05-11T13:31:39.549679+0000","last_clean":"2024-05-11T13:31:39.549679+0000","last_became_active":"2024-05-11T13:28:35.220294+0000","last_became_peered":"2024-05-11T13:28:35.220294+0000","last_unstale":"2024-05-11T13:31:39.549679+0000","last_undegraded":"2024-05-11T13:31:39.549679+0000","last_fullsized":"2024-05-11T13:31:39.549679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:04.608042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.520852+0000","last_change":"2024-05-11T13:28:35.197643+0000","last_active":"2024-05-11T13:31:39.520852+0000","last_peered":"2024-05-11T13:31:39.520852+0000","last_clean":"2024-05-11T13:31:39.520852+0000","last_became_active":"2024-05-11T13:28:35.196528+0000","last_became_peered":"2024-05-11T13:28:35.196528+0000","last_unstale":"2024-05-11T13:31:39.520852+0000","last_undegraded":"2024-05-11T13:31:39.520852+0000","last_fullsized":"2024-05-11T13:31:39.520852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:02:03.088456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.518934+0000","last_change":"2024-05-11T13:28:35.206833+0000","last_active":"2024-05-11T13:31:39.518934+0000","last_peered":"2024-05-11T13:31:39.518934+0000","last_clean":"2024-05-11T13:31:39.518934+0000","last_became_active":"2024-05-11T13:28:35.206287+0000","last_became_peered":"2024-05-11T13:28:35.206287+0000","last_unstale":"2024-05-11T13:31:39.518934+0000","last_undegraded":"2024-05-11T13:31:39.518934+0000","last_fullsized":"2024-05-11T13:31:39.518934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:25:20.576464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.529275+0000","last_change":"2024-05-11T13:28:35.218950+0000","last_active":"2024-05-11T13:31:39.529275+0000","last_peered":"2024-05-11T13:31:39.529275+0000","last_clean":"2024-05-11T13:31:39.529275+0000","last_became_active":"2024-05-11T13:28:35.218597+0000","last_became_peered":"2024-05-11T13:28:35.218597+0000","last_unstale":"2024-05-11T13:31:39.529275+0000","last_undegraded":"2024-05-11T13:31:39.529275+0000","last_fullsized":"2024-05-11T13:31:39.529275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:08:42.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.520884+0000","last_change":"2024-05-11T13:29:40.541106+0000","last_active":"2024-05-11T13:31:39.520884+0000","last_peered":"2024-05-11T13:31:39.520884+0000","last_clean":"2024-05-11T13:31:39.520884+0000","last_became_active":"2024-05-11T13:29:40.540007+0000","last_became_peered":"2024-05-11T13:29:40.540007+0000","last_unstale":"2024-05-11T13:31:39.520884+0000","last_undegraded":"2024-05-11T13:31:39.520884+0000","last_fullsized":"2024-05-11T13:31:39.520884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:00.596580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.534232+0000","last_change":"2024-05-11T13:28:35.205483+0000","last_active":"2024-05-11T13:31:39.534232+0000","last_peered":"2024-05-11T13:31:39.534232+0000","last_clean":"2024-05-11T13:31:39.534232+0000","last_became_active":"2024-05-11T13:28:35.205101+0000","last_became_peered":"2024-05-11T13:28:35.205101+0000","last_unstale":"2024-05-11T13:31:39.534232+0000","last_undegraded":"2024-05-11T13:31:39.534232+0000","last_fullsized":"2024-05-11T13:31:39.534232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:37.917464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.547438+0000","last_change":"2024-05-11T13:28:35.217372+0000","last_active":"2024-05-11T13:31:39.547438+0000","last_peered":"2024-05-11T13:31:39.547438+0000","last_clean":"2024-05-11T13:31:39.547438+0000","last_became_active":"2024-05-11T13:28:35.217021+0000","last_became_peered":"2024-05-11T13:28:35.217021+0000","last_unstale":"2024-05-11T13:31:39.547438+0000","last_undegraded":"2024-05-11T13:31:39.547438+0000","last_fullsized":"2024-05-11T13:31:39.547438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:24.273041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.532583+0000","last_change":"2024-05-11T13:29:40.548634+0000","last_active":"2024-05-11T13:31:39.532583+0000","last_peered":"2024-05-11T13:31:39.532583+0000","last_clean":"2024-05-11T13:31:39.532583+0000","last_became_active":"2024-05-11T13:29:40.547536+0000","last_became_peered":"2024-05-11T13:29:40.547536+0000","last_unstale":"2024-05-11T13:31:39.532583+0000","last_undegraded":"2024-05-11T13:31:39.532583+0000","last_fullsized":"2024-05-11T13:31:39.532583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:15.362317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545128+0000","last_change":"2024-05-11T13:28:35.214091+0000","last_active":"2024-05-11T13:31:39.545128+0000","last_peered":"2024-05-11T13:31:39.545128+0000","last_clean":"2024-05-11T13:31:39.545128+0000","last_became_active":"2024-05-11T13:28:35.213668+0000","last_became_peered":"2024-05-11T13:28:35.213668+0000","last_unstale":"2024-05-11T13:31:39.545128+0000","last_undegraded":"2024-05-11T13:31:39.545128+0000","last_fullsized":"2024-05-11T13:31:39.545128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:19:55.768203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.527465+0000","last_change":"2024-05-11T13:28:35.211874+0000","last_active":"2024-05-11T13:31:39.527465+0000","last_peered":"2024-05-11T13:31:39.527465+0000","last_clean":"2024-05-11T13:31:39.527465+0000","last_became_active":"2024-05-11T13:28:35.211503+0000","last_became_peered":"2024-05-11T13:28:35.211503+0000","last_unstale":"2024-05-11T13:31:39.527465+0000","last_undegraded":"2024-05-11T13:31:39.527465+0000","last_fullsized":"2024-05-11T13:31:39.527465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:46.884110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530668+0000","last_change":"2024-05-11T13:28:35.209308+0000","last_active":"2024-05-11T13:31:39.530668+0000","last_peered":"2024-05-11T13:31:39.530668+0000","last_clean":"2024-05-11T13:31:39.530668+0000","last_became_active":"2024-05-11T13:28:35.207927+0000","last_became_peered":"2024-05-11T13:28:35.207927+0000","last_unstale":"2024-05-11T13:31:39.530668+0000","last_undegraded":"2024-05-11T13:31:39.530668+0000","last_fullsized":"2024-05-11T13:31:39.530668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:04.541849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.541562+0000","last_change":"2024-05-11T13:28:35.208524+0000","last_active":"2024-05-11T13:31:39.541562+0000","last_peered":"2024-05-11T13:31:39.541562+0000","last_clean":"2024-05-11T13:31:39.541562+0000","last_became_active":"2024-05-11T13:28:35.207875+0000","last_became_peered":"2024-05-11T13:28:35.207875+0000","last_unstale":"2024-05-11T13:31:39.541562+0000","last_undegraded":"2024-05-11T13:31:39.541562+0000","last_fullsized":"2024-05-11T13:31:39.541562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:23:38.896916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548589+0000","last_change":"2024-05-11T13:28:35.216418+0000","last_active":"2024-05-11T13:31:39.548589+0000","last_peered":"2024-05-11T13:31:39.548589+0000","last_clean":"2024-05-11T13:31:39.548589+0000","last_became_active":"2024-05-11T13:28:35.215984+0000","last_became_peered":"2024-05-11T13:28:35.215984+0000","last_unstale":"2024-05-11T13:31:39.548589+0000","last_undegraded":"2024-05-11T13:31:39.548589+0000","last_fullsized":"2024-05-11T13:31:39.548589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:15:13.650821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.547193+0000","last_change":"2024-05-11T13:28:35.217701+0000","last_active":"2024-05-11T13:31:39.547193+0000","last_peered":"2024-05-11T13:31:39.547193+0000","last_clean":"2024-05-11T13:31:39.547193+0000","last_became_active":"2024-05-11T13:28:35.217050+0000","last_became_peered":"2024-05-11T13:28:35.217050+0000","last_unstale":"2024-05-11T13:31:39.547193+0000","last_undegraded":"2024-05-11T13:31:39.547193+0000","last_fullsized":"2024-05-11T13:31:39.547193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:29:38.274540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546742+0000","last_change":"2024-05-11T13:28:35.223794+0000","last_active":"2024-05-11T13:31:39.546742+0000","last_peered":"2024-05-11T13:31:39.546742+0000","last_clean":"2024-05-11T13:31:39.546742+0000","last_became_active":"2024-05-11T13:28:35.223388+0000","last_became_peered":"2024-05-11T13:28:35.223388+0000","last_unstale":"2024-05-11T13:31:39.546742+0000","last_undegraded":"2024-05-11T13:31:39.546742+0000","last_fullsized":"2024-05-11T13:31:39.546742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:56:13.272357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545385+0000","last_change":"2024-05-11T13:28:35.216452+0000","last_active":"2024-05-11T13:31:39.545385+0000","last_peered":"2024-05-11T13:31:39.545385+0000","last_clean":"2024-05-11T13:31:39.545385+0000","last_became_active":"2024-05-11T13:28:35.216075+0000","last_became_peered":"2024-05-11T13:28:35.216075+0000","last_unstale":"2024-05-11T13:31:39.545385+0000","last_undegraded":"2024-05-11T13:31:39.545385+0000","last_fullsized":"2024-05-11T13:31:39.545385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:23.012302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.524485+0000","last_change":"2024-05-11T13:28:35.206528+0000","last_active":"2024-05-11T13:31:39.524485+0000","last_peered":"2024-05-11T13:31:39.524485+0000","last_clean":"2024-05-11T13:31:39.524485+0000","last_became_active":"2024-05-11T13:28:35.206102+0000","last_became_peered":"2024-05-11T13:28:35.206102+0000","last_unstale":"2024-05-11T13:31:39.524485+0000","last_undegraded":"2024-05-11T13:31:39.524485+0000","last_fullsized":"2024-05-11T13:31:39.524485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.685477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.924274+0000","last_change":"2024-05-11T13:30:41.923155+0000","last_active":"2024-05-11T13:30:41.924274+0000","last_peered":"2024-05-11T13:30:41.924274+0000","last_clean":"2024-05-11T13:30:41.924274+0000","last_became_active":"2024-05-11T13:30:41.922484+0000","last_became_peered":"2024-05-11T13:30:41.922484+0000","last_unstale":"2024-05-11T13:30:41.924274+0000","last_undegraded":"2024-05-11T13:30:41.924274+0000","last_fullsized":"2024-05-11T13:30:41.924274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:52.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:54.116469+0000","last_change":"2024-05-11T13:28:30.428076+0000","last_active":"2024-05-11T13:31:54.116469+0000","last_peered":"2024-05-11T13:31:54.116469+0000","last_clean":"2024-05-11T13:31:54.116469+0000","last_became_active":"2024-05-11T13:28:30.427519+0000","last_became_peered":"2024-05-11T13:28:30.427519+0000","last_unstale":"2024-05-11T13:31:54.116469+0000","last_undegraded":"2024-05-11T13:31:54.116469+0000","last_fullsized":"2024-05-11T13:31:54.116469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:52.808051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.533839+0000","last_change":"2024-05-11T13:28:35.207587+0000","last_active":"2024-05-11T13:31:39.533839+0000","last_peered":"2024-05-11T13:31:39.533839+0000","last_clean":"2024-05-11T13:31:39.533839+0000","last_became_active":"2024-05-11T13:28:35.206692+0000","last_became_peered":"2024-05-11T13:28:35.206692+0000","last_unstale":"2024-05-11T13:31:39.533839+0000","last_undegraded":"2024-05-11T13:31:39.533839+0000","last_fullsized":"2024-05-11T13:31:39.533839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:25.171342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939018+0000","last_change":"2024-05-11T13:28:49.682649+0000","last_active":"2024-05-11T13:30:41.939018+0000","last_peered":"2024-05-11T13:30:41.939018+0000","last_clean":"2024-05-11T13:30:41.939018+0000","last_became_active":"2024-05-11T13:28:49.682192+0000","last_became_peered":"2024-05-11T13:28:49.682192+0000","last_unstale":"2024-05-11T13:30:41.939018+0000","last_undegraded":"2024-05-11T13:30:41.939018+0000","last_fullsized":"2024-05-11T13:30:41.939018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:01.728705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.534308+0000","last_change":"2024-05-11T13:28:30.707050+0000","last_active":"2024-05-11T13:30:41.534308+0000","last_peered":"2024-05-11T13:30:41.534308+0000","last_clean":"2024-05-11T13:30:41.534308+0000","last_became_active":"2024-05-11T13:28:30.706279+0000","last_became_peered":"2024-05-11T13:28:30.706279+0000","last_unstale":"2024-05-11T13:30:41.534308+0000","last_undegraded":"2024-05-11T13:30:41.534308+0000","last_fullsized":"2024-05-11T13:30:41.534308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:14.135734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.550636+0000","last_change":"2024-05-11T13:28:35.219622+0000","last_active":"2024-05-11T13:31:39.550636+0000","last_peered":"2024-05-11T13:31:39.550636+0000","last_clean":"2024-05-11T13:31:39.550636+0000","last_became_active":"2024-05-11T13:28:35.218687+0000","last_became_peered":"2024-05-11T13:28:35.218687+0000","last_unstale":"2024-05-11T13:31:39.550636+0000","last_undegraded":"2024-05-11T13:31:39.550636+0000","last_fullsized":"2024-05-11T13:31:39.550636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:53.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939066+0000","last_change":"2024-05-11T13:28:49.685770+0000","last_active":"2024-05-11T13:30:41.939066+0000","last_peered":"2024-05-11T13:30:41.939066+0000","last_clean":"2024-05-11T13:30:41.939066+0000","last_became_active":"2024-05-11T13:28:49.685091+0000","last_became_peered":"2024-05-11T13:28:49.685091+0000","last_unstale":"2024-05-11T13:30:41.939066+0000","last_undegraded":"2024-05-11T13:30:41.939066+0000","last_fullsized":"2024-05-11T13:30:41.939066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:42.458310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921649+0000","last_change":"2024-05-11T13:28:30.435304+0000","last_active":"2024-05-11T13:30:41.921649+0000","last_peered":"2024-05-11T13:30:41.921649+0000","last_clean":"2024-05-11T13:30:41.921649+0000","last_became_active":"2024-05-11T13:28:30.434648+0000","last_became_peered":"2024-05-11T13:28:30.434648+0000","last_unstale":"2024-05-11T13:30:41.921649+0000","last_undegraded":"2024-05-11T13:30:41.921649+0000","last_fullsized":"2024-05-11T13:30:41.921649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:21.553376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549398+0000","last_change":"2024-05-11T13:28:35.210877+0000","last_active":"2024-05-11T13:31:39.549398+0000","last_peered":"2024-05-11T13:31:39.549398+0000","last_clean":"2024-05-11T13:31:39.549398+0000","last_became_active":"2024-05-11T13:28:35.207069+0000","last_became_peered":"2024-05-11T13:28:35.207069+0000","last_unstale":"2024-05-11T13:31:39.549398+0000","last_undegraded":"2024-05-11T13:31:39.549398+0000","last_fullsized":"2024-05-11T13:31:39.549398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:59.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939019+0000","last_change":"2024-05-11T13:28:49.689070+0000","last_active":"2024-05-11T13:30:41.939019+0000","last_peered":"2024-05-11T13:30:41.939019+0000","last_clean":"2024-05-11T13:30:41.939019+0000","last_became_active":"2024-05-11T13:28:49.688502+0000","last_became_peered":"2024-05-11T13:28:49.688502+0000","last_unstale":"2024-05-11T13:30:41.939019+0000","last_undegraded":"2024-05-11T13:30:41.939019+0000","last_fullsized":"2024-05-11T13:30:41.939019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:21.516074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.913426+0000","last_change":"2024-05-11T13:29:40.564494+0000","last_active":"2024-05-11T13:30:41.913426+0000","last_peered":"2024-05-11T13:30:41.913426+0000","last_clean":"2024-05-11T13:30:41.913426+0000","last_became_active":"2024-05-11T13:29:40.555386+0000","last_became_peered":"2024-05-11T13:29:40.555386+0000","last_unstale":"2024-05-11T13:30:41.913426+0000","last_undegraded":"2024-05-11T13:30:41.913426+0000","last_fullsized":"2024-05-11T13:30:41.913426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:32:50.641372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549880+0000","last_change":"2024-05-11T13:28:35.217950+0000","last_active":"2024-05-11T13:31:39.549880+0000","last_peered":"2024-05-11T13:31:39.549880+0000","last_clean":"2024-05-11T13:31:39.549880+0000","last_became_active":"2024-05-11T13:28:35.216698+0000","last_became_peered":"2024-05-11T13:28:35.216698+0000","last_unstale":"2024-05-11T13:31:39.549880+0000","last_undegraded":"2024-05-11T13:31:39.549880+0000","last_fullsized":"2024-05-11T13:31:39.549880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:01.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.924187+0000","last_change":"2024-05-11T13:30:41.923049+0000","last_active":"2024-05-11T13:30:41.924187+0000","last_peered":"2024-05-11T13:30:41.924187+0000","last_clean":"2024-05-11T13:30:41.924187+0000","last_became_active":"2024-05-11T13:30:41.922585+0000","last_became_peered":"2024-05-11T13:30:41.922585+0000","last_unstale":"2024-05-11T13:30:41.924187+0000","last_undegraded":"2024-05-11T13:30:41.924187+0000","last_fullsized":"2024-05-11T13:30:41.924187+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:34:03.716885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:56.139302+0000","last_change":"2024-05-11T13:29:40.555238+0000","last_active":"2024-05-11T13:31:56.139302+0000","last_peered":"2024-05-11T13:31:56.139302+0000","last_clean":"2024-05-11T13:31:56.139302+0000","last_became_active":"2024-05-11T13:29:40.554661+0000","last_became_peered":"2024-05-11T13:29:40.554661+0000","last_unstale":"2024-05-11T13:31:56.139302+0000","last_undegraded":"2024-05-11T13:31:56.139302+0000","last_fullsized":"2024-05-11T13:31:56.139302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:22.983649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530851+0000","last_change":"2024-05-11T13:28:35.209163+0000","last_active":"2024-05-11T13:31:39.530851+0000","last_peered":"2024-05-11T13:31:39.530851+0000","last_clean":"2024-05-11T13:31:39.530851+0000","last_became_active":"2024-05-11T13:28:35.204014+0000","last_became_peered":"2024-05-11T13:28:35.204014+0000","last_unstale":"2024-05-11T13:31:39.530851+0000","last_undegraded":"2024-05-11T13:31:39.530851+0000","last_fullsized":"2024-05-11T13:31:39.530851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:17.570863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.019977+0000","last_change":"2024-05-11T13:29:40.555520+0000","last_active":"2024-05-11T13:30:41.019977+0000","last_peered":"2024-05-11T13:30:41.019977+0000","last_clean":"2024-05-11T13:30:41.019977+0000","last_became_active":"2024-05-11T13:29:40.553952+0000","last_became_peered":"2024-05-11T13:29:40.553952+0000","last_unstale":"2024-05-11T13:30:41.019977+0000","last_undegraded":"2024-05-11T13:30:41.019977+0000","last_fullsized":"2024-05-11T13:30:41.019977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.035691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.018438+0000","last_change":"2024-05-11T13:28:30.429829+0000","last_active":"2024-05-11T13:30:41.018438+0000","last_peered":"2024-05-11T13:30:41.018438+0000","last_clean":"2024-05-11T13:30:41.018438+0000","last_became_active":"2024-05-11T13:28:30.428818+0000","last_became_peered":"2024-05-11T13:28:30.428818+0000","last_unstale":"2024-05-11T13:30:41.018438+0000","last_undegraded":"2024-05-11T13:30:41.018438+0000","last_fullsized":"2024-05-11T13:30:41.018438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:59:25.653181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535645+0000","last_change":"2024-05-11T13:28:35.214153+0000","last_active":"2024-05-11T13:31:39.535645+0000","last_peered":"2024-05-11T13:31:39.535645+0000","last_clean":"2024-05-11T13:31:39.535645+0000","last_became_active":"2024-05-11T13:28:35.213800+0000","last_became_peered":"2024-05-11T13:28:35.213800+0000","last_unstale":"2024-05-11T13:31:39.535645+0000","last_undegraded":"2024-05-11T13:31:39.535645+0000","last_fullsized":"2024-05-11T13:31:39.535645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:12.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.914038+0000","last_change":"2024-05-11T13:28:49.697320+0000","last_active":"2024-05-11T13:30:41.914038+0000","last_peered":"2024-05-11T13:30:41.914038+0000","last_clean":"2024-05-11T13:30:41.914038+0000","last_became_active":"2024-05-11T13:28:49.696751+0000","last_became_peered":"2024-05-11T13:28:49.696751+0000","last_unstale":"2024-05-11T13:30:41.914038+0000","last_undegraded":"2024-05-11T13:30:41.914038+0000","last_fullsized":"2024-05-11T13:30:41.914038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:24.617241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.015202+0000","last_change":"2024-05-11T13:28:30.423943+0000","last_active":"2024-05-11T13:30:41.015202+0000","last_peered":"2024-05-11T13:30:41.015202+0000","last_clean":"2024-05-11T13:30:41.015202+0000","last_became_active":"2024-05-11T13:28:30.423454+0000","last_became_peered":"2024-05-11T13:28:30.423454+0000","last_unstale":"2024-05-11T13:30:41.015202+0000","last_undegraded":"2024-05-11T13:30:41.015202+0000","last_fullsized":"2024-05-11T13:30:41.015202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:23:10.967537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535645+0000","last_change":"2024-05-11T13:28:35.220479+0000","last_active":"2024-05-11T13:31:39.535645+0000","last_peered":"2024-05-11T13:31:39.535645+0000","last_clean":"2024-05-11T13:31:39.535645+0000","last_became_active":"2024-05-11T13:28:35.214541+0000","last_became_peered":"2024-05-11T13:28:35.214541+0000","last_unstale":"2024-05-11T13:31:39.535645+0000","last_undegraded":"2024-05-11T13:31:39.535645+0000","last_fullsized":"2024-05-11T13:31:39.535645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:07.746301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.827235+0000","last_change":"2024-05-11T13:28:49.691867+0000","last_active":"2024-05-11T13:30:41.827235+0000","last_peered":"2024-05-11T13:30:41.827235+0000","last_clean":"2024-05-11T13:30:41.827235+0000","last_became_active":"2024-05-11T13:28:49.690045+0000","last_became_peered":"2024-05-11T13:28:49.690045+0000","last_unstale":"2024-05-11T13:30:41.827235+0000","last_undegraded":"2024-05-11T13:30:41.827235+0000","last_fullsized":"2024-05-11T13:30:41.827235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:16.220334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.015294+0000","last_change":"2024-05-11T13:28:30.436354+0000","last_active":"2024-05-11T13:30:41.015294+0000","last_peered":"2024-05-11T13:30:41.015294+0000","last_clean":"2024-05-11T13:30:41.015294+0000","last_became_active":"2024-05-11T13:28:30.435983+0000","last_became_peered":"2024-05-11T13:28:30.435983+0000","last_unstale":"2024-05-11T13:30:41.015294+0000","last_undegraded":"2024-05-11T13:30:41.015294+0000","last_fullsized":"2024-05-11T13:30:41.015294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:42.442829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.532445+0000","last_change":"2024-05-11T13:28:35.207284+0000","last_active":"2024-05-11T13:31:39.532445+0000","last_peered":"2024-05-11T13:31:39.532445+0000","last_clean":"2024-05-11T13:31:39.532445+0000","last_became_active":"2024-05-11T13:28:35.206945+0000","last_became_peered":"2024-05-11T13:28:35.206945+0000","last_unstale":"2024-05-11T13:31:39.532445+0000","last_undegraded":"2024-05-11T13:31:39.532445+0000","last_fullsized":"2024-05-11T13:31:39.532445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:18.493659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.533467+0000","last_change":"2024-05-11T13:28:49.689558+0000","last_active":"2024-05-11T13:30:41.533467+0000","last_peered":"2024-05-11T13:30:41.533467+0000","last_clean":"2024-05-11T13:30:41.533467+0000","last_became_active":"2024-05-11T13:28:49.689156+0000","last_became_peered":"2024-05-11T13:28:49.689156+0000","last_unstale":"2024-05-11T13:30:41.533467+0000","last_undegraded":"2024-05-11T13:30:41.533467+0000","last_fullsized":"2024-05-11T13:30:41.533467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:21.530111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":283,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:55.135040+0000","last_change":"2024-05-11T13:28:30.434534+0000","last_active":"2024-05-11T13:31:55.135040+0000","last_peered":"2024-05-11T13:31:55.135040+0000","last_clean":"2024-05-11T13:31:55.135040+0000","last_became_active":"2024-05-11T13:28:30.433983+0000","last_became_peered":"2024-05-11T13:28:30.433983+0000","last_unstale":"2024-05-11T13:31:55.135040+0000","last_undegraded":"2024-05-11T13:31:55.135040+0000","last_fullsized":"2024-05-11T13:31:55.135040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:16.056662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549771+0000","last_change":"2024-05-11T13:28:35.205021+0000","last_active":"2024-05-11T13:31:39.549771+0000","last_peered":"2024-05-11T13:31:39.549771+0000","last_clean":"2024-05-11T13:31:39.549771+0000","last_became_active":"2024-05-11T13:28:35.204243+0000","last_became_peered":"2024-05-11T13:28:35.204243+0000","last_unstale":"2024-05-11T13:31:39.549771+0000","last_undegraded":"2024-05-11T13:31:39.549771+0000","last_fullsized":"2024-05-11T13:31:39.549771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:12.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.767631+0000","last_change":"2024-05-11T13:28:49.677392+0000","last_active":"2024-05-11T13:30:41.767631+0000","last_peered":"2024-05-11T13:30:41.767631+0000","last_clean":"2024-05-11T13:30:41.767631+0000","last_became_active":"2024-05-11T13:28:49.674008+0000","last_became_peered":"2024-05-11T13:28:49.674008+0000","last_unstale":"2024-05-11T13:30:41.767631+0000","last_undegraded":"2024-05-11T13:30:41.767631+0000","last_fullsized":"2024-05-11T13:30:41.767631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:47:14.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.014459+0000","last_change":"2024-05-11T13:28:30.429500+0000","last_active":"2024-05-11T13:30:41.014459+0000","last_peered":"2024-05-11T13:30:41.014459+0000","last_clean":"2024-05-11T13:30:41.014459+0000","last_became_active":"2024-05-11T13:28:30.429049+0000","last_became_peered":"2024-05-11T13:28:30.429049+0000","last_unstale":"2024-05-11T13:30:41.014459+0000","last_undegraded":"2024-05-11T13:30:41.014459+0000","last_fullsized":"2024-05-11T13:30:41.014459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:00.533603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.541794+0000","last_change":"2024-05-11T13:28:35.223764+0000","last_active":"2024-05-11T13:31:39.541794+0000","last_peered":"2024-05-11T13:31:39.541794+0000","last_clean":"2024-05-11T13:31:39.541794+0000","last_became_active":"2024-05-11T13:28:35.223016+0000","last_became_peered":"2024-05-11T13:28:35.223016+0000","last_unstale":"2024-05-11T13:31:39.541794+0000","last_undegraded":"2024-05-11T13:31:39.541794+0000","last_fullsized":"2024-05-11T13:31:39.541794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:42.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.938999+0000","last_change":"2024-05-11T13:28:49.683733+0000","last_active":"2024-05-11T13:30:41.938999+0000","last_peered":"2024-05-11T13:30:41.938999+0000","last_clean":"2024-05-11T13:30:41.938999+0000","last_became_active":"2024-05-11T13:28:49.681245+0000","last_became_peered":"2024-05-11T13:28:49.681245+0000","last_unstale":"2024-05-11T13:30:41.938999+0000","last_undegraded":"2024-05-11T13:30:41.938999+0000","last_fullsized":"2024-05-11T13:30:41.938999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:03.386943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016649+0000","last_change":"2024-05-11T13:28:30.707394+0000","last_active":"2024-05-11T13:30:41.016649+0000","last_peered":"2024-05-11T13:30:41.016649+0000","last_clean":"2024-05-11T13:30:41.016649+0000","last_became_active":"2024-05-11T13:28:30.706114+0000","last_became_peered":"2024-05-11T13:28:30.706114+0000","last_unstale":"2024-05-11T13:30:41.016649+0000","last_undegraded":"2024-05-11T13:30:41.016649+0000","last_fullsized":"2024-05-11T13:30:41.016649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:53:04.599800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548448+0000","last_change":"2024-05-11T13:28:35.219836+0000","last_active":"2024-05-11T13:31:39.548448+0000","last_peered":"2024-05-11T13:31:39.548448+0000","last_clean":"2024-05-11T13:31:39.548448+0000","last_became_active":"2024-05-11T13:28:35.214474+0000","last_became_peered":"2024-05-11T13:28:35.214474+0000","last_unstale":"2024-05-11T13:31:39.548448+0000","last_undegraded":"2024-05-11T13:31:39.548448+0000","last_fullsized":"2024-05-11T13:31:39.548448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:02.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.767625+0000","last_change":"2024-05-11T13:28:49.694677+0000","last_active":"2024-05-11T13:30:41.767625+0000","last_peered":"2024-05-11T13:30:41.767625+0000","last_clean":"2024-05-11T13:30:41.767625+0000","last_became_active":"2024-05-11T13:28:49.694176+0000","last_became_peered":"2024-05-11T13:28:49.694176+0000","last_unstale":"2024-05-11T13:30:41.767625+0000","last_undegraded":"2024-05-11T13:30:41.767625+0000","last_fullsized":"2024-05-11T13:30:41.767625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:35.461937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":328,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:55.158323+0000","last_change":"2024-05-11T13:28:30.433202+0000","last_active":"2024-05-11T13:31:55.158323+0000","last_peered":"2024-05-11T13:31:55.158323+0000","last_clean":"2024-05-11T13:31:55.158323+0000","last_became_active":"2024-05-11T13:28:30.429269+0000","last_became_peered":"2024-05-11T13:28:30.429269+0000","last_unstale":"2024-05-11T13:31:55.158323+0000","last_undegraded":"2024-05-11T13:31:55.158323+0000","last_fullsized":"2024-05-11T13:31:55.158323+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:27:15.620717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":245,"num_read_kb":184,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.524649+0000","last_change":"2024-05-11T13:28:35.210882+0000","last_active":"2024-05-11T13:31:39.524649+0000","last_peered":"2024-05-11T13:31:39.524649+0000","last_clean":"2024-05-11T13:31:39.524649+0000","last_became_active":"2024-05-11T13:28:35.210506+0000","last_became_peered":"2024-05-11T13:28:35.210506+0000","last_unstale":"2024-05-11T13:31:39.524649+0000","last_undegraded":"2024-05-11T13:31:39.524649+0000","last_fullsized":"2024-05-11T13:31:39.524649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:45.752666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017111+0000","last_change":"2024-05-11T13:28:49.693254+0000","last_active":"2024-05-11T13:30:41.017111+0000","last_peered":"2024-05-11T13:30:41.017111+0000","last_clean":"2024-05-11T13:30:41.017111+0000","last_became_active":"2024-05-11T13:28:49.690307+0000","last_became_peered":"2024-05-11T13:28:49.690307+0000","last_unstale":"2024-05-11T13:30:41.017111+0000","last_undegraded":"2024-05-11T13:30:41.017111+0000","last_fullsized":"2024-05-11T13:30:41.017111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:41:31.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.019481+0000","last_change":"2024-05-11T13:28:30.429421+0000","last_active":"2024-05-11T13:30:41.019481+0000","last_peered":"2024-05-11T13:30:41.019481+0000","last_clean":"2024-05-11T13:30:41.019481+0000","last_became_active":"2024-05-11T13:28:30.429032+0000","last_became_peered":"2024-05-11T13:28:30.429032+0000","last_unstale":"2024-05-11T13:30:41.019481+0000","last_undegraded":"2024-05-11T13:30:41.019481+0000","last_fullsized":"2024-05-11T13:30:41.019481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:06.234650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.538071+0000","last_change":"2024-05-11T13:28:35.207709+0000","last_active":"2024-05-11T13:31:39.538071+0000","last_peered":"2024-05-11T13:31:39.538071+0000","last_clean":"2024-05-11T13:31:39.538071+0000","last_became_active":"2024-05-11T13:28:35.207222+0000","last_became_peered":"2024-05-11T13:28:35.207222+0000","last_unstale":"2024-05-11T13:31:39.538071+0000","last_undegraded":"2024-05-11T13:31:39.538071+0000","last_fullsized":"2024-05-11T13:31:39.538071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:20.023565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.913230+0000","last_change":"2024-05-11T13:28:49.692046+0000","last_active":"2024-05-11T13:30:41.913230+0000","last_peered":"2024-05-11T13:30:41.913230+0000","last_clean":"2024-05-11T13:30:41.913230+0000","last_became_active":"2024-05-11T13:28:49.691618+0000","last_became_peered":"2024-05-11T13:28:49.691618+0000","last_unstale":"2024-05-11T13:30:41.913230+0000","last_undegraded":"2024-05-11T13:30:41.913230+0000","last_fullsized":"2024-05-11T13:30:41.913230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:05.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017218+0000","last_change":"2024-05-11T13:28:30.435459+0000","last_active":"2024-05-11T13:30:41.017218+0000","last_peered":"2024-05-11T13:30:41.017218+0000","last_clean":"2024-05-11T13:30:41.017218+0000","last_became_active":"2024-05-11T13:28:30.434997+0000","last_became_peered":"2024-05-11T13:28:30.434997+0000","last_unstale":"2024-05-11T13:30:41.017218+0000","last_undegraded":"2024-05-11T13:30:41.017218+0000","last_fullsized":"2024-05-11T13:30:41.017218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:36.280546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548457+0000","last_change":"2024-05-11T13:28:35.211214+0000","last_active":"2024-05-11T13:31:39.548457+0000","last_peered":"2024-05-11T13:31:39.548457+0000","last_clean":"2024-05-11T13:31:39.548457+0000","last_became_active":"2024-05-11T13:28:35.210777+0000","last_became_peered":"2024-05-11T13:28:35.210777+0000","last_unstale":"2024-05-11T13:31:39.548457+0000","last_undegraded":"2024-05-11T13:31:39.548457+0000","last_fullsized":"2024-05-11T13:31:39.548457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:53.238549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.767860+0000","last_change":"2024-05-11T13:28:49.682336+0000","last_active":"2024-05-11T13:30:41.767860+0000","last_peered":"2024-05-11T13:30:41.767860+0000","last_clean":"2024-05-11T13:30:41.767860+0000","last_became_active":"2024-05-11T13:28:49.679081+0000","last_became_peered":"2024-05-11T13:28:49.679081+0000","last_unstale":"2024-05-11T13:30:41.767860+0000","last_undegraded":"2024-05-11T13:30:41.767860+0000","last_fullsized":"2024-05-11T13:30:41.767860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:00:21.149262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:52.150468+0000","last_change":"2024-05-11T13:28:30.432165+0000","last_active":"2024-05-11T13:31:52.150468+0000","last_peered":"2024-05-11T13:31:52.150468+0000","last_clean":"2024-05-11T13:31:52.150468+0000","last_became_active":"2024-05-11T13:28:30.431181+0000","last_became_peered":"2024-05-11T13:28:30.431181+0000","last_unstale":"2024-05-11T13:31:52.150468+0000","last_undegraded":"2024-05-11T13:31:52.150468+0000","last_fullsized":"2024-05-11T13:31:52.150468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:26:50.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.536072+0000","last_change":"2024-05-11T13:28:35.205662+0000","last_active":"2024-05-11T13:31:39.536072+0000","last_peered":"2024-05-11T13:31:39.536072+0000","last_clean":"2024-05-11T13:31:39.536072+0000","last_became_active":"2024-05-11T13:28:35.205276+0000","last_became_peered":"2024-05-11T13:28:35.205276+0000","last_unstale":"2024-05-11T13:31:39.536072+0000","last_undegraded":"2024-05-11T13:31:39.536072+0000","last_fullsized":"2024-05-11T13:31:39.536072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:36.308402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017145+0000","last_change":"2024-05-11T13:28:49.689705+0000","last_active":"2024-05-11T13:30:41.017145+0000","last_peered":"2024-05-11T13:30:41.017145+0000","last_clean":"2024-05-11T13:30:41.017145+0000","last_became_active":"2024-05-11T13:28:49.689223+0000","last_became_peered":"2024-05-11T13:28:49.689223+0000","last_unstale":"2024-05-11T13:30:41.017145+0000","last_undegraded":"2024-05-11T13:30:41.017145+0000","last_fullsized":"2024-05-11T13:30:41.017145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:37:20.727773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921035+0000","last_change":"2024-05-11T13:28:30.709014+0000","last_active":"2024-05-11T13:30:41.921035+0000","last_peered":"2024-05-11T13:30:41.921035+0000","last_clean":"2024-05-11T13:30:41.921035+0000","last_became_active":"2024-05-11T13:28:30.708000+0000","last_became_peered":"2024-05-11T13:28:30.708000+0000","last_unstale":"2024-05-11T13:30:41.921035+0000","last_undegraded":"2024-05-11T13:30:41.921035+0000","last_fullsized":"2024-05-11T13:30:41.921035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:00.813415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.544748+0000","last_change":"2024-05-11T13:28:35.215149+0000","last_active":"2024-05-11T13:31:39.544748+0000","last_peered":"2024-05-11T13:31:39.544748+0000","last_clean":"2024-05-11T13:31:39.544748+0000","last_became_active":"2024-05-11T13:28:35.214792+0000","last_became_peered":"2024-05-11T13:28:35.214792+0000","last_unstale":"2024-05-11T13:31:39.544748+0000","last_undegraded":"2024-05-11T13:31:39.544748+0000","last_fullsized":"2024-05-11T13:31:39.544748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016602+0000","last_change":"2024-05-11T13:28:49.686347+0000","last_active":"2024-05-11T13:30:41.016602+0000","last_peered":"2024-05-11T13:30:41.016602+0000","last_clean":"2024-05-11T13:30:41.016602+0000","last_became_active":"2024-05-11T13:28:49.685391+0000","last_became_peered":"2024-05-11T13:28:49.685391+0000","last_unstale":"2024-05-11T13:30:41.016602+0000","last_undegraded":"2024-05-11T13:30:41.016602+0000","last_fullsized":"2024-05-11T13:30:41.016602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:24:50.347627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.914009+0000","last_change":"2024-05-11T13:29:40.560815+0000","last_active":"2024-05-11T13:30:41.914009+0000","last_peered":"2024-05-11T13:30:41.914009+0000","last_clean":"2024-05-11T13:30:41.914009+0000","last_became_active":"2024-05-11T13:29:40.553031+0000","last_became_peered":"2024-05-11T13:29:40.553031+0000","last_unstale":"2024-05-11T13:30:41.914009+0000","last_undegraded":"2024-05-11T13:30:41.914009+0000","last_fullsized":"2024-05-11T13:30:41.914009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:27.437654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.531554+0000","last_change":"2024-05-11T13:28:35.218377+0000","last_active":"2024-05-11T13:31:39.531554+0000","last_peered":"2024-05-11T13:31:39.531554+0000","last_clean":"2024-05-11T13:31:39.531554+0000","last_became_active":"2024-05-11T13:28:35.217922+0000","last_became_peered":"2024-05-11T13:28:35.217922+0000","last_unstale":"2024-05-11T13:31:39.531554+0000","last_undegraded":"2024-05-11T13:31:39.531554+0000","last_fullsized":"2024-05-11T13:31:39.531554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:58.383381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.920277+0000","last_change":"2024-05-11T13:28:49.723400+0000","last_active":"2024-05-11T13:30:41.920277+0000","last_peered":"2024-05-11T13:30:41.920277+0000","last_clean":"2024-05-11T13:30:41.920277+0000","last_became_active":"2024-05-11T13:28:49.722675+0000","last_became_peered":"2024-05-11T13:28:49.722675+0000","last_unstale":"2024-05-11T13:30:41.920277+0000","last_undegraded":"2024-05-11T13:30:41.920277+0000","last_fullsized":"2024-05-11T13:30:41.920277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:14:44.810541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.768834+0000","last_change":"2024-05-11T13:28:30.710667+0000","last_active":"2024-05-11T13:30:41.768834+0000","last_peered":"2024-05-11T13:30:41.768834+0000","last_clean":"2024-05-11T13:30:41.768834+0000","last_became_active":"2024-05-11T13:28:30.709933+0000","last_became_peered":"2024-05-11T13:28:30.709933+0000","last_unstale":"2024-05-11T13:30:41.768834+0000","last_undegraded":"2024-05-11T13:30:41.768834+0000","last_fullsized":"2024-05-11T13:30:41.768834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:34.942946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.533224+0000","last_change":"2024-05-11T13:28:35.215691+0000","last_active":"2024-05-11T13:31:39.533224+0000","last_peered":"2024-05-11T13:31:39.533224+0000","last_clean":"2024-05-11T13:31:39.533224+0000","last_became_active":"2024-05-11T13:28:35.209873+0000","last_became_peered":"2024-05-11T13:28:35.209873+0000","last_unstale":"2024-05-11T13:31:39.533224+0000","last_undegraded":"2024-05-11T13:31:39.533224+0000","last_fullsized":"2024-05-11T13:31:39.533224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:42:12.935570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016763+0000","last_change":"2024-05-11T13:28:49.694322+0000","last_active":"2024-05-11T13:30:41.016763+0000","last_peered":"2024-05-11T13:30:41.016763+0000","last_clean":"2024-05-11T13:30:41.016763+0000","last_became_active":"2024-05-11T13:28:49.690900+0000","last_became_peered":"2024-05-11T13:28:49.690900+0000","last_unstale":"2024-05-11T13:30:41.016763+0000","last_undegraded":"2024-05-11T13:30:41.016763+0000","last_fullsized":"2024-05-11T13:30:41.016763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:00:07.433371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":169,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:52.112066+0000","last_change":"2024-05-11T13:28:30.432529+0000","last_active":"2024-05-11T13:31:52.112066+0000","last_peered":"2024-05-11T13:31:52.112066+0000","last_clean":"2024-05-11T13:31:52.112066+0000","last_became_active":"2024-05-11T13:28:30.428077+0000","last_became_peered":"2024-05-11T13:28:30.428077+0000","last_unstale":"2024-05-11T13:31:52.112066+0000","last_undegraded":"2024-05-11T13:31:52.112066+0000","last_fullsized":"2024-05-11T13:31:52.112066+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:42.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.539136+0000","last_change":"2024-05-11T13:28:35.219160+0000","last_active":"2024-05-11T13:31:39.539136+0000","last_peered":"2024-05-11T13:31:39.539136+0000","last_clean":"2024-05-11T13:31:39.539136+0000","last_became_active":"2024-05-11T13:28:35.216810+0000","last_became_peered":"2024-05-11T13:28:35.216810+0000","last_unstale":"2024-05-11T13:31:39.539136+0000","last_undegraded":"2024-05-11T13:31:39.539136+0000","last_fullsized":"2024-05-11T13:31:39.539136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:35:54.465030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017120+0000","last_change":"2024-05-11T13:28:49.725190+0000","last_active":"2024-05-11T13:30:41.017120+0000","last_peered":"2024-05-11T13:30:41.017120+0000","last_clean":"2024-05-11T13:30:41.017120+0000","last_became_active":"2024-05-11T13:28:49.724193+0000","last_became_peered":"2024-05-11T13:28:49.724193+0000","last_unstale":"2024-05-11T13:30:41.017120+0000","last_undegraded":"2024-05-11T13:30:41.017120+0000","last_fullsized":"2024-05-11T13:30:41.017120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:47:41.181606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530900+0000","last_change":"2024-05-11T13:28:35.211262+0000","last_active":"2024-05-11T13:31:39.530900+0000","last_peered":"2024-05-11T13:31:39.530900+0000","last_clean":"2024-05-11T13:31:39.530900+0000","last_became_active":"2024-05-11T13:28:35.210859+0000","last_became_peered":"2024-05-11T13:28:35.210859+0000","last_unstale":"2024-05-11T13:31:39.530900+0000","last_undegraded":"2024-05-11T13:31:39.530900+0000","last_fullsized":"2024-05-11T13:31:39.530900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:40:33.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":171,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:56.160407+0000","last_change":"2024-05-11T13:28:30.707326+0000","last_active":"2024-05-11T13:31:56.160407+0000","last_peered":"2024-05-11T13:31:56.160407+0000","last_clean":"2024-05-11T13:31:56.160407+0000","last_became_active":"2024-05-11T13:28:30.705496+0000","last_became_peered":"2024-05-11T13:28:30.705496+0000","last_unstale":"2024-05-11T13:31:56.160407+0000","last_undegraded":"2024-05-11T13:31:56.160407+0000","last_fullsized":"2024-05-11T13:31:56.160407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:48.117551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.014636+0000","last_change":"2024-05-11T13:28:49.687260+0000","last_active":"2024-05-11T13:30:41.014636+0000","last_peered":"2024-05-11T13:30:41.014636+0000","last_clean":"2024-05-11T13:30:41.014636+0000","last_became_active":"2024-05-11T13:28:49.686593+0000","last_became_peered":"2024-05-11T13:28:49.686593+0000","last_unstale":"2024-05-11T13:30:41.014636+0000","last_undegraded":"2024-05-11T13:30:41.014636+0000","last_fullsized":"2024-05-11T13:30:41.014636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:48.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545582+0000","last_change":"2024-05-11T13:28:35.216643+0000","last_active":"2024-05-11T13:31:39.545582+0000","last_peered":"2024-05-11T13:31:39.545582+0000","last_clean":"2024-05-11T13:31:39.545582+0000","last_became_active":"2024-05-11T13:28:35.216300+0000","last_became_peered":"2024-05-11T13:28:35.216300+0000","last_unstale":"2024-05-11T13:31:39.545582+0000","last_undegraded":"2024-05-11T13:31:39.545582+0000","last_fullsized":"2024-05-11T13:31:39.545582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:20:01.351678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939899+0000","last_change":"2024-05-11T13:28:30.427465+0000","last_active":"2024-05-11T13:30:41.939899+0000","last_peered":"2024-05-11T13:30:41.939899+0000","last_clean":"2024-05-11T13:30:41.939899+0000","last_became_active":"2024-05-11T13:28:30.426968+0000","last_became_peered":"2024-05-11T13:28:30.426968+0000","last_unstale":"2024-05-11T13:30:41.939899+0000","last_undegraded":"2024-05-11T13:30:41.939899+0000","last_fullsized":"2024-05-11T13:30:41.939899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:02:44.969069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.013859+0000","last_change":"2024-05-11T13:26:37.738725+0000","last_active":"2024-05-11T13:30:41.013859+0000","last_peered":"2024-05-11T13:30:41.013859+0000","last_clean":"2024-05-11T13:30:41.013859+0000","last_became_active":"2024-05-11T13:26:37.724302+0000","last_became_peered":"2024-05-11T13:26:37.724302+0000","last_unstale":"2024-05-11T13:30:41.013859+0000","last_undegraded":"2024-05-11T13:30:41.013859+0000","last_fullsized":"2024-05-11T13:30:41.013859+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_clean_scrub_stamp":"2024-05-11T13:21:55.918148+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:32:56.277780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016431+0000","last_change":"2024-05-11T13:28:49.723829+0000","last_active":"2024-05-11T13:30:41.016431+0000","last_peered":"2024-05-11T13:30:41.016431+0000","last_clean":"2024-05-11T13:30:41.016431+0000","last_became_active":"2024-05-11T13:28:49.723190+0000","last_became_peered":"2024-05-11T13:28:49.723190+0000","last_unstale":"2024-05-11T13:30:41.016431+0000","last_undegraded":"2024-05-11T13:30:41.016431+0000","last_fullsized":"2024-05-11T13:30:41.016431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.742374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.525461+0000","last_change":"2024-05-11T13:28:35.213486+0000","last_active":"2024-05-11T13:31:39.525461+0000","last_peered":"2024-05-11T13:31:39.525461+0000","last_clean":"2024-05-11T13:31:39.525461+0000","last_became_active":"2024-05-11T13:28:35.211962+0000","last_became_peered":"2024-05-11T13:28:35.211962+0000","last_unstale":"2024-05-11T13:31:39.525461+0000","last_undegraded":"2024-05-11T13:31:39.525461+0000","last_fullsized":"2024-05-11T13:31:39.525461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:28:34.046898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.013954+0000","last_change":"2024-05-11T13:28:30.431023+0000","last_active":"2024-05-11T13:30:41.013954+0000","last_peered":"2024-05-11T13:30:41.013954+0000","last_clean":"2024-05-11T13:30:41.013954+0000","last_became_active":"2024-05-11T13:28:30.430609+0000","last_became_peered":"2024-05-11T13:28:30.430609+0000","last_unstale":"2024-05-11T13:30:41.013954+0000","last_undegraded":"2024-05-11T13:30:41.013954+0000","last_fullsized":"2024-05-11T13:30:41.013954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:50.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.913253+0000","last_change":"2024-05-11T13:28:49.695550+0000","last_active":"2024-05-11T13:30:41.913253+0000","last_peered":"2024-05-11T13:30:41.913253+0000","last_clean":"2024-05-11T13:30:41.913253+0000","last_became_active":"2024-05-11T13:28:49.694350+0000","last_became_peered":"2024-05-11T13:28:49.694350+0000","last_unstale":"2024-05-11T13:30:41.913253+0000","last_undegraded":"2024-05-11T13:30:41.913253+0000","last_fullsized":"2024-05-11T13:30:41.913253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:49.617673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921612+0000","last_change":"2024-05-11T13:28:30.710826+0000","last_active":"2024-05-11T13:30:41.921612+0000","last_peered":"2024-05-11T13:30:41.921612+0000","last_clean":"2024-05-11T13:30:41.921612+0000","last_became_active":"2024-05-11T13:28:30.709854+0000","last_became_peered":"2024-05-11T13:28:30.709854+0000","last_unstale":"2024-05-11T13:30:41.921612+0000","last_undegraded":"2024-05-11T13:30:41.921612+0000","last_fullsized":"2024-05-11T13:30:41.921612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:35:16.384559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.527286+0000","last_change":"2024-05-11T13:28:35.205521+0000","last_active":"2024-05-11T13:31:39.527286+0000","last_peered":"2024-05-11T13:31:39.527286+0000","last_clean":"2024-05-11T13:31:39.527286+0000","last_became_active":"2024-05-11T13:28:35.205167+0000","last_became_peered":"2024-05-11T13:28:35.205167+0000","last_unstale":"2024-05-11T13:31:39.527286+0000","last_undegraded":"2024-05-11T13:31:39.527286+0000","last_fullsized":"2024-05-11T13:31:39.527286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:10:27.473991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.532268+0000","last_change":"2024-05-11T13:28:49.723652+0000","last_active":"2024-05-11T13:30:41.532268+0000","last_peered":"2024-05-11T13:30:41.532268+0000","last_clean":"2024-05-11T13:30:41.532268+0000","last_became_active":"2024-05-11T13:28:49.723135+0000","last_became_peered":"2024-05-11T13:28:49.723135+0000","last_unstale":"2024-05-11T13:30:41.532268+0000","last_undegraded":"2024-05-11T13:30:41.532268+0000","last_fullsized":"2024-05-11T13:30:41.532268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:21:18.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.768826+0000","last_change":"2024-05-11T13:28:30.428197+0000","last_active":"2024-05-11T13:30:41.768826+0000","last_peered":"2024-05-11T13:30:41.768826+0000","last_clean":"2024-05-11T13:30:41.768826+0000","last_became_active":"2024-05-11T13:28:30.427753+0000","last_became_peered":"2024-05-11T13:28:30.427753+0000","last_unstale":"2024-05-11T13:30:41.768826+0000","last_undegraded":"2024-05-11T13:30:41.768826+0000","last_fullsized":"2024-05-11T13:30:41.768826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:52:54.165993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.538630+0000","last_change":"2024-05-11T13:29:40.541861+0000","last_active":"2024-05-11T13:31:39.538630+0000","last_peered":"2024-05-11T13:31:39.538630+0000","last_clean":"2024-05-11T13:31:39.538630+0000","last_became_active":"2024-05-11T13:29:40.541395+0000","last_became_peered":"2024-05-11T13:29:40.541395+0000","last_unstale":"2024-05-11T13:31:39.538630+0000","last_undegraded":"2024-05-11T13:31:39.538630+0000","last_fullsized":"2024-05-11T13:31:39.538630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:50:39.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.018357+0000","last_change":"2024-05-11T13:28:49.677549+0000","last_active":"2024-05-11T13:30:41.018357+0000","last_peered":"2024-05-11T13:30:41.018357+0000","last_clean":"2024-05-11T13:30:41.018357+0000","last_became_active":"2024-05-11T13:28:49.676625+0000","last_became_peered":"2024-05-11T13:28:49.676625+0000","last_unstale":"2024-05-11T13:30:41.018357+0000","last_undegraded":"2024-05-11T13:30:41.018357+0000","last_fullsized":"2024-05-11T13:30:41.018357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:44.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939670+0000","last_change":"2024-05-11T13:28:30.440945+0000","last_active":"2024-05-11T13:30:41.939670+0000","last_peered":"2024-05-11T13:30:41.939670+0000","last_clean":"2024-05-11T13:30:41.939670+0000","last_became_active":"2024-05-11T13:28:30.440595+0000","last_became_peered":"2024-05-11T13:28:30.440595+0000","last_unstale":"2024-05-11T13:30:41.939670+0000","last_undegraded":"2024-05-11T13:30:41.939670+0000","last_fullsized":"2024-05-11T13:30:41.939670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:51:10.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549451+0000","last_change":"2024-05-11T13:28:35.214207+0000","last_active":"2024-05-11T13:31:39.549451+0000","last_peered":"2024-05-11T13:31:39.549451+0000","last_clean":"2024-05-11T13:31:39.549451+0000","last_became_active":"2024-05-11T13:28:35.213844+0000","last_became_peered":"2024-05-11T13:28:35.213844+0000","last_unstale":"2024-05-11T13:31:39.549451+0000","last_undegraded":"2024-05-11T13:31:39.549451+0000","last_fullsized":"2024-05-11T13:31:39.549451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:53.063329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939373+0000","last_change":"2024-05-11T13:28:49.725505+0000","last_active":"2024-05-11T13:30:41.939373+0000","last_peered":"2024-05-11T13:30:41.939373+0000","last_clean":"2024-05-11T13:30:41.939373+0000","last_became_active":"2024-05-11T13:28:49.723613+0000","last_became_peered":"2024-05-11T13:28:49.723613+0000","last_unstale":"2024-05-11T13:30:41.939373+0000","last_undegraded":"2024-05-11T13:30:41.939373+0000","last_fullsized":"2024-05-11T13:30:41.939373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:54:30.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:42.067588+0000","last_change":"2024-05-11T13:28:30.707025+0000","last_active":"2024-05-11T13:30:42.067588+0000","last_peered":"2024-05-11T13:30:42.067588+0000","last_clean":"2024-05-11T13:30:42.067588+0000","last_became_active":"2024-05-11T13:28:30.705853+0000","last_became_peered":"2024-05-11T13:28:30.705853+0000","last_unstale":"2024-05-11T13:30:42.067588+0000","last_undegraded":"2024-05-11T13:30:42.067588+0000","last_fullsized":"2024-05-11T13:30:42.067588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:15:04.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.483892+0000","last_change":"2024-05-11T13:28:35.214682+0000","last_active":"2024-05-11T13:31:39.483892+0000","last_peered":"2024-05-11T13:31:39.483892+0000","last_clean":"2024-05-11T13:31:39.483892+0000","last_became_active":"2024-05-11T13:28:35.211581+0000","last_became_peered":"2024-05-11T13:28:35.211581+0000","last_unstale":"2024-05-11T13:31:39.483892+0000","last_undegraded":"2024-05-11T13:31:39.483892+0000","last_fullsized":"2024-05-11T13:31:39.483892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:19.231675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.536065+0000","last_change":"2024-05-11T13:30:41.536065+0000","last_active":"2024-05-11T13:30:41.536065+0000","last_peered":"2024-05-11T13:30:41.536065+0000","last_clean":"2024-05-11T13:30:41.536065+0000","last_became_active":"2024-05-11T13:30:41.535466+0000","last_became_peered":"2024-05-11T13:30:41.535466+0000","last_unstale":"2024-05-11T13:30:41.536065+0000","last_undegraded":"2024-05-11T13:30:41.536065+0000","last_fullsized":"2024-05-11T13:30:41.536065+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:18.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.533479+0000","last_change":"2024-05-11T13:28:30.419444+0000","last_active":"2024-05-11T13:30:41.533479+0000","last_peered":"2024-05-11T13:30:41.533479+0000","last_clean":"2024-05-11T13:30:41.533479+0000","last_became_active":"2024-05-11T13:28:30.419064+0000","last_became_peered":"2024-05-11T13:28:30.419064+0000","last_unstale":"2024-05-11T13:30:41.533479+0000","last_undegraded":"2024-05-11T13:30:41.533479+0000","last_fullsized":"2024-05-11T13:30:41.533479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:02.567403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546675+0000","last_change":"2024-05-11T13:28:35.209347+0000","last_active":"2024-05-11T13:31:39.546675+0000","last_peered":"2024-05-11T13:31:39.546675+0000","last_clean":"2024-05-11T13:31:39.546675+0000","last_became_active":"2024-05-11T13:28:35.208912+0000","last_became_peered":"2024-05-11T13:28:35.208912+0000","last_unstale":"2024-05-11T13:31:39.546675+0000","last_undegraded":"2024-05-11T13:31:39.546675+0000","last_fullsized":"2024-05-11T13:31:39.546675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:08:30.832941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939832+0000","last_change":"2024-05-11T13:28:49.677398+0000","last_active":"2024-05-11T13:30:41.939832+0000","last_peered":"2024-05-11T13:30:41.939832+0000","last_clean":"2024-05-11T13:30:41.939832+0000","last_became_active":"2024-05-11T13:28:49.676725+0000","last_became_peered":"2024-05-11T13:28:49.676725+0000","last_unstale":"2024-05-11T13:30:41.939832+0000","last_undegraded":"2024-05-11T13:30:41.939832+0000","last_fullsized":"2024-05-11T13:30:41.939832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:32.476049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.019045+0000","last_change":"2024-05-11T13:28:30.428782+0000","last_active":"2024-05-11T13:30:41.019045+0000","last_peered":"2024-05-11T13:30:41.019045+0000","last_clean":"2024-05-11T13:30:41.019045+0000","last_became_active":"2024-05-11T13:28:30.427819+0000","last_became_peered":"2024-05-11T13:28:30.427819+0000","last_unstale":"2024-05-11T13:30:41.019045+0000","last_undegraded":"2024-05-11T13:30:41.019045+0000","last_fullsized":"2024-05-11T13:30:41.019045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:23:24.533765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545234+0000","last_change":"2024-05-11T13:28:35.210095+0000","last_active":"2024-05-11T13:31:39.545234+0000","last_peered":"2024-05-11T13:31:39.545234+0000","last_clean":"2024-05-11T13:31:39.545234+0000","last_became_active":"2024-05-11T13:28:35.208831+0000","last_became_peered":"2024-05-11T13:28:35.208831+0000","last_unstale":"2024-05-11T13:31:39.545234+0000","last_undegraded":"2024-05-11T13:31:39.545234+0000","last_fullsized":"2024-05-11T13:31:39.545234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:06.957821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939164+0000","last_change":"2024-05-11T13:28:49.724791+0000","last_active":"2024-05-11T13:30:41.939164+0000","last_peered":"2024-05-11T13:30:41.939164+0000","last_clean":"2024-05-11T13:30:41.939164+0000","last_became_active":"2024-05-11T13:28:49.723855+0000","last_became_peered":"2024-05-11T13:28:49.723855+0000","last_unstale":"2024-05-11T13:30:41.939164+0000","last_undegraded":"2024-05-11T13:30:41.939164+0000","last_fullsized":"2024-05-11T13:30:41.939164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:34.705331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921294+0000","last_change":"2024-05-11T13:28:30.431206+0000","last_active":"2024-05-11T13:30:41.921294+0000","last_peered":"2024-05-11T13:30:41.921294+0000","last_clean":"2024-05-11T13:30:41.921294+0000","last_became_active":"2024-05-11T13:28:30.430714+0000","last_became_peered":"2024-05-11T13:28:30.430714+0000","last_unstale":"2024-05-11T13:30:41.921294+0000","last_undegraded":"2024-05-11T13:30:41.921294+0000","last_fullsized":"2024-05-11T13:30:41.921294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:29:21.070046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.543830+0000","last_change":"2024-05-11T13:28:35.222254+0000","last_active":"2024-05-11T13:31:39.543830+0000","last_peered":"2024-05-11T13:31:39.543830+0000","last_clean":"2024-05-11T13:31:39.543830+0000","last_became_active":"2024-05-11T13:28:35.219539+0000","last_became_peered":"2024-05-11T13:28:35.219539+0000","last_unstale":"2024-05-11T13:31:39.543830+0000","last_undegraded":"2024-05-11T13:31:39.543830+0000","last_fullsized":"2024-05-11T13:31:39.543830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:55.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:42.067427+0000","last_change":"2024-05-11T13:28:49.692168+0000","last_active":"2024-05-11T13:30:42.067427+0000","last_peered":"2024-05-11T13:30:42.067427+0000","last_clean":"2024-05-11T13:30:42.067427+0000","last_became_active":"2024-05-11T13:28:49.691730+0000","last_became_peered":"2024-05-11T13:28:49.691730+0000","last_unstale":"2024-05-11T13:30:42.067427+0000","last_undegraded":"2024-05-11T13:30:42.067427+0000","last_fullsized":"2024-05-11T13:30:42.067427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:02.935554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":166,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:52.126657+0000","last_change":"2024-05-11T13:28:30.712115+0000","last_active":"2024-05-11T13:31:52.126657+0000","last_peered":"2024-05-11T13:31:52.126657+0000","last_clean":"2024-05-11T13:31:52.126657+0000","last_became_active":"2024-05-11T13:28:30.710613+0000","last_became_peered":"2024-05-11T13:28:30.710613+0000","last_unstale":"2024-05-11T13:31:52.126657+0000","last_undegraded":"2024-05-11T13:31:52.126657+0000","last_fullsized":"2024-05-11T13:31:52.126657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:00.521915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546340+0000","last_change":"2024-05-11T13:28:35.221340+0000","last_active":"2024-05-11T13:31:39.546340+0000","last_peered":"2024-05-11T13:31:39.546340+0000","last_clean":"2024-05-11T13:31:39.546340+0000","last_became_active":"2024-05-11T13:28:35.220702+0000","last_became_peered":"2024-05-11T13:28:35.220702+0000","last_unstale":"2024-05-11T13:31:39.546340+0000","last_undegraded":"2024-05-11T13:31:39.546340+0000","last_fullsized":"2024-05-11T13:31:39.546340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:59:37.313209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.827356+0000","last_change":"2024-05-11T13:28:49.682917+0000","last_active":"2024-05-11T13:30:41.827356+0000","last_peered":"2024-05-11T13:30:41.827356+0000","last_clean":"2024-05-11T13:30:41.827356+0000","last_became_active":"2024-05-11T13:28:49.681905+0000","last_became_peered":"2024-05-11T13:28:49.681905+0000","last_unstale":"2024-05-11T13:30:41.827356+0000","last_undegraded":"2024-05-11T13:30:41.827356+0000","last_fullsized":"2024-05-11T13:30:41.827356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:09:26.191565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016879+0000","last_change":"2024-05-11T13:28:30.437356+0000","last_active":"2024-05-11T13:30:41.016879+0000","last_peered":"2024-05-11T13:30:41.016879+0000","last_clean":"2024-05-11T13:30:41.016879+0000","last_became_active":"2024-05-11T13:28:30.437013+0000","last_became_peered":"2024-05-11T13:28:30.437013+0000","last_unstale":"2024-05-11T13:30:41.016879+0000","last_undegraded":"2024-05-11T13:30:41.016879+0000","last_fullsized":"2024-05-11T13:30:41.016879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:58.756156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.539209+0000","last_change":"2024-05-11T13:28:35.206105+0000","last_active":"2024-05-11T13:31:39.539209+0000","last_peered":"2024-05-11T13:31:39.539209+0000","last_clean":"2024-05-11T13:31:39.539209+0000","last_became_active":"2024-05-11T13:28:35.205684+0000","last_became_peered":"2024-05-11T13:28:35.205684+0000","last_unstale":"2024-05-11T13:31:39.539209+0000","last_undegraded":"2024-05-11T13:31:39.539209+0000","last_fullsized":"2024-05-11T13:31:39.539209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:12.326574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.015157+0000","last_change":"2024-05-11T13:28:49.689545+0000","last_active":"2024-05-11T13:30:41.015157+0000","last_peered":"2024-05-11T13:30:41.015157+0000","last_clean":"2024-05-11T13:30:41.015157+0000","last_became_active":"2024-05-11T13:28:49.688528+0000","last_became_peered":"2024-05-11T13:28:49.688528+0000","last_unstale":"2024-05-11T13:30:41.015157+0000","last_undegraded":"2024-05-11T13:30:41.015157+0000","last_fullsized":"2024-05-11T13:30:41.015157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:35.418873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017223+0000","last_change":"2024-05-11T13:29:40.560835+0000","last_active":"2024-05-11T13:30:41.017223+0000","last_peered":"2024-05-11T13:30:41.017223+0000","last_clean":"2024-05-11T13:30:41.017223+0000","last_became_active":"2024-05-11T13:29:40.554310+0000","last_became_peered":"2024-05-11T13:29:40.554310+0000","last_unstale":"2024-05-11T13:30:41.017223+0000","last_undegraded":"2024-05-11T13:30:41.017223+0000","last_fullsized":"2024-05-11T13:30:41.017223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:19:26.805261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.539695+0000","last_change":"2024-05-11T13:28:35.220001+0000","last_active":"2024-05-11T13:31:39.539695+0000","last_peered":"2024-05-11T13:31:39.539695+0000","last_clean":"2024-05-11T13:31:39.539695+0000","last_became_active":"2024-05-11T13:28:35.216768+0000","last_became_peered":"2024-05-11T13:28:35.216768+0000","last_unstale":"2024-05-11T13:31:39.539695+0000","last_undegraded":"2024-05-11T13:31:39.539695+0000","last_fullsized":"2024-05-11T13:31:39.539695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:54:23.547689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.531701+0000","last_change":"2024-05-11T13:28:35.213481+0000","last_active":"2024-05-11T13:31:39.531701+0000","last_peered":"2024-05-11T13:31:39.531701+0000","last_clean":"2024-05-11T13:31:39.531701+0000","last_became_active":"2024-05-11T13:28:35.212956+0000","last_became_peered":"2024-05-11T13:28:35.212956+0000","last_unstale":"2024-05-11T13:31:39.531701+0000","last_undegraded":"2024-05-11T13:31:39.531701+0000","last_fullsized":"2024-05-11T13:31:39.531701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:14.814155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530898+0000","last_change":"2024-05-11T13:29:40.554654+0000","last_active":"2024-05-11T13:31:39.530898+0000","last_peered":"2024-05-11T13:31:39.530898+0000","last_clean":"2024-05-11T13:31:39.530898+0000","last_became_active":"2024-05-11T13:29:40.551836+0000","last_became_peered":"2024-05-11T13:29:40.551836+0000","last_unstale":"2024-05-11T13:31:39.530898+0000","last_undegraded":"2024-05-11T13:31:39.530898+0000","last_fullsized":"2024-05-11T13:31:39.530898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:49:44.566051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548974+0000","last_change":"2024-05-11T13:28:35.208925+0000","last_active":"2024-05-11T13:31:39.548974+0000","last_peered":"2024-05-11T13:31:39.548974+0000","last_clean":"2024-05-11T13:31:39.548974+0000","last_became_active":"2024-05-11T13:28:35.208055+0000","last_became_peered":"2024-05-11T13:28:35.208055+0000","last_unstale":"2024-05-11T13:31:39.548974+0000","last_undegraded":"2024-05-11T13:31:39.548974+0000","last_fullsized":"2024-05-11T13:31:39.548974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:20.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549052+0000","last_change":"2024-05-11T13:28:35.213226+0000","last_active":"2024-05-11T13:31:39.549052+0000","last_peered":"2024-05-11T13:31:39.549052+0000","last_clean":"2024-05-11T13:31:39.549052+0000","last_became_active":"2024-05-11T13:28:35.212839+0000","last_became_peered":"2024-05-11T13:28:35.212839+0000","last_unstale":"2024-05-11T13:31:39.549052+0000","last_undegraded":"2024-05-11T13:31:39.549052+0000","last_fullsized":"2024-05-11T13:31:39.549052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:37:05.312505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549428+0000","last_change":"2024-05-11T13:29:40.848092+0000","last_active":"2024-05-11T13:31:39.549428+0000","last_peered":"2024-05-11T13:31:39.549428+0000","last_clean":"2024-05-11T13:31:39.549428+0000","last_became_active":"2024-05-11T13:29:40.846950+0000","last_became_peered":"2024-05-11T13:29:40.846950+0000","last_unstale":"2024-05-11T13:31:39.549428+0000","last_undegraded":"2024-05-11T13:31:39.549428+0000","last_fullsized":"2024-05-11T13:31:39.549428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:18:06.675351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530970+0000","last_change":"2024-05-11T13:28:35.217529+0000","last_active":"2024-05-11T13:31:39.530970+0000","last_peered":"2024-05-11T13:31:39.530970+0000","last_clean":"2024-05-11T13:31:39.530970+0000","last_became_active":"2024-05-11T13:28:35.217062+0000","last_became_peered":"2024-05-11T13:28:35.217062+0000","last_unstale":"2024-05-11T13:31:39.530970+0000","last_undegraded":"2024-05-11T13:31:39.530970+0000","last_fullsized":"2024-05-11T13:31:39.530970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:24:33.955794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.550463+0000","last_change":"2024-05-11T13:28:35.217777+0000","last_active":"2024-05-11T13:31:39.550463+0000","last_peered":"2024-05-11T13:31:39.550463+0000","last_clean":"2024-05-11T13:31:39.550463+0000","last_became_active":"2024-05-11T13:28:35.217381+0000","last_became_peered":"2024-05-11T13:28:35.217381+0000","last_unstale":"2024-05-11T13:31:39.550463+0000","last_undegraded":"2024-05-11T13:31:39.550463+0000","last_fullsized":"2024-05-11T13:31:39.550463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:43.973192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.524878+0000","last_change":"2024-05-11T13:28:35.209826+0000","last_active":"2024-05-11T13:31:39.524878+0000","last_peered":"2024-05-11T13:31:39.524878+0000","last_clean":"2024-05-11T13:31:39.524878+0000","last_became_active":"2024-05-11T13:28:35.209461+0000","last_became_peered":"2024-05-11T13:28:35.209461+0000","last_unstale":"2024-05-11T13:31:39.524878+0000","last_undegraded":"2024-05-11T13:31:39.524878+0000","last_fullsized":"2024-05-11T13:31:39.524878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:03.323045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535561+0000","last_change":"2024-05-11T13:28:35.205252+0000","last_active":"2024-05-11T13:31:39.535561+0000","last_peered":"2024-05-11T13:31:39.535561+0000","last_clean":"2024-05-11T13:31:39.535561+0000","last_became_active":"2024-05-11T13:28:35.204835+0000","last_became_peered":"2024-05-11T13:28:35.204835+0000","last_unstale":"2024-05-11T13:31:39.535561+0000","last_undegraded":"2024-05-11T13:31:39.535561+0000","last_fullsized":"2024-05-11T13:31:39.535561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:10.032439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535006+0000","last_change":"2024-05-11T13:28:35.219718+0000","last_active":"2024-05-11T13:31:39.535006+0000","last_peered":"2024-05-11T13:31:39.535006+0000","last_clean":"2024-05-11T13:31:39.535006+0000","last_became_active":"2024-05-11T13:28:35.219032+0000","last_became_peered":"2024-05-11T13:28:35.219032+0000","last_unstale":"2024-05-11T13:31:39.535006+0000","last_undegraded":"2024-05-11T13:31:39.535006+0000","last_fullsized":"2024-05-11T13:31:39.535006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:03:58.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.547789+0000","last_change":"2024-05-11T13:28:35.217820+0000","last_active":"2024-05-11T13:31:39.547789+0000","last_peered":"2024-05-11T13:31:39.547789+0000","last_clean":"2024-05-11T13:31:39.547789+0000","last_became_active":"2024-05-11T13:28:35.216260+0000","last_became_peered":"2024-05-11T13:28:35.216260+0000","last_unstale":"2024-05-11T13:31:39.547789+0000","last_undegraded":"2024-05-11T13:31:39.547789+0000","last_fullsized":"2024-05-11T13:31:39.547789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:20:47.188238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549407+0000","last_change":"2024-05-11T13:28:35.209762+0000","last_active":"2024-05-11T13:31:39.549407+0000","last_peered":"2024-05-11T13:31:39.549407+0000","last_clean":"2024-05-11T13:31:39.549407+0000","last_became_active":"2024-05-11T13:28:35.207404+0000","last_became_peered":"2024-05-11T13:28:35.207404+0000","last_unstale":"2024-05-11T13:31:39.549407+0000","last_undegraded":"2024-05-11T13:31:39.549407+0000","last_fullsized":"2024-05-11T13:31:39.549407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:41:20.808009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.531809+0000","last_change":"2024-05-11T13:28:35.215137+0000","last_active":"2024-05-11T13:31:39.531809+0000","last_peered":"2024-05-11T13:31:39.531809+0000","last_clean":"2024-05-11T13:31:39.531809+0000","last_became_active":"2024-05-11T13:28:35.214745+0000","last_became_peered":"2024-05-11T13:28:35.214745+0000","last_unstale":"2024-05-11T13:31:39.531809+0000","last_undegraded":"2024-05-11T13:31:39.531809+0000","last_fullsized":"2024-05-11T13:31:39.531809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:44.381501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70392,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1213,"num_read_kb":688,"num_write":257,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678076,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":519366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874306,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32872,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712280,"statfs":{"total":95995035648,"available":95961374720,"internally_reserved":0,"allocated":1736704,"data_stored":1169016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":522788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234065,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":518466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430296,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1116240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593824,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":514728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574978,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":578664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738504,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":515652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":577730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149813,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":513904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953582,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":519424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790055,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1110758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T13:31:58.998 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T13:31:59.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:58 smithi063 ceph-mon[29809]: pgmap v479: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:59.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:58 smithi071 ceph-mon[33293]: pgmap v479: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:58 smithi165 ceph-mon[33235]: pgmap v479: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:31:59.513 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:00.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:31:59 smithi071 ceph-mon[33293]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:32:00.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:31:59 smithi165 ceph-mon[33235]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:32:00.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:31:59 smithi063 ceph-mon[29809]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:32:01.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:00 smithi063 ceph-mon[29809]: pgmap v480: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:01.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:00 smithi165 ceph-mon[33235]: pgmap v480: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:01.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:00 smithi071 ceph-mon[33293]: pgmap v480: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 -- 172.21.15.63:0/1905289370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0114490 msgr2=0x7f54c0116880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 --2- 172.21.15.63:0/1905289370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0114490 0x7f54c0116880 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54b800b0a0 tx=0x7f54b800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:32:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 -- 172.21.15.63:0/1905289370 shutdown_connections 2024-05-11T13:32:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 --2- 172.21.15.63:0/1905289370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0114490 0x7f54c0116880 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 --2- 172.21.15.63:0/1905289370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c00758f0 0x7f54c0075cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 --2- 172.21.15.63:0/1905289370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54c0076210 0x7f54c0113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 -- 172.21.15.63:0/1905289370 >> 172.21.15.63:0/1905289370 conn(0x7f54c00fefd0 msgr2=0x7f54c01013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 -- 172.21.15.63:0/1905289370 shutdown_connections 2024-05-11T13:32:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.879+0000 7f54c4a00700 1 -- 172.21.15.63:0/1905289370 wait complete. 2024-05-11T13:32:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.880+0000 7f54c4a00700 1 Processor -- start 2024-05-11T13:32:01.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.880+0000 7f54c4a00700 1 -- start start 2024-05-11T13:32:01.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.880+0000 7f54c4a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54c00758f0 0x7f54c01a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:01.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54c4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0076210 0x7f54c01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54c4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c0114490 0x7f54c01a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54c4a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0119bb0 con 0x7f54c0076210 2024-05-11T13:32:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54c4a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0119a30 con 0x7f54c0114490 2024-05-11T13:32:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54c4a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0119d30 con 0x7f54c00758f0 2024-05-11T13:32:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0076210 0x7f54c01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0076210 0x7f54c01a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53486/0 (socket says 172.21.15.63:53486) 2024-05-11T13:32:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 -- 172.21.15.63:0/3731400275 learned_addr learned my addr 172.21.15.63:0/3731400275 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bf200700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c0114490 0x7f54c01a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54be800700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54c00758f0 0x7f54c01a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 -- 172.21.15.63:0/3731400275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54c00758f0 msgr2=0x7f54c01a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54c00758f0 0x7f54c01a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 -- 172.21.15.63:0/3731400275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c0114490 msgr2=0x7f54c01a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c0114490 0x7f54c01a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.881+0000 7f54bde00700 1 -- 172.21.15.63:0/3731400275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01a50e0 con 0x7f54c0076210 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.882+0000 7f54bde00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0076210 0x7f54c01a0660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00b530 tx=0x7f54ac00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.882+0000 7f54b7400700 1 -- 172.21.15.63:0/3731400275 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac015020 con 0x7f54c0076210 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.882+0000 7f54b7400700 1 -- 172.21.15.63:0/3731400275 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54ac011cc0 con 0x7f54c0076210 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.882+0000 7f54b7400700 1 -- 172.21.15.63:0/3731400275 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac010d30 con 0x7f54c0076210 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.882+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01a53d0 con 0x7f54c0076210 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.882+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01acc50 con 0x7f54c0076210 2024-05-11T13:32:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.884+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c006a240 con 0x7f54c0076210 2024-05-11T13:32:01.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.886+0000 7f54b7400700 1 -- 172.21.15.63:0/3731400275 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f54ac018020 con 0x7f54c0076210 2024-05-11T13:32:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.887+0000 7f54b7400700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f54a40b2900 0x7f54a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.888+0000 7f54b7400700 1 -- 172.21.15.63:0/3731400275 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f54ac0e1ea0 con 0x7f54c0076210 2024-05-11T13:32:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.888+0000 7f54be800700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f54a40b2900 0x7f54a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.888+0000 7f54be800700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f54a40b2900 0x7f54a40b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54b0005950 tx=0x7f54b00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:01.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:01.892+0000 7f54b7400700 1 -- 172.21.15.63:0/3731400275 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54ac01f020 con 0x7f54c0076210 2024-05-11T13:32:02.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.126+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54c0063990 con 0x7f54a40b2900 2024-05-11T13:32:02.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.138+0000 7f54b7400700 1 -- 172.21.15.63:0/3731400275 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340517 (secure 0 0 0) 0x7f54c0063990 con 0x7f54a40b2900 2024-05-11T13:32:02.146 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f54a40b2900 msgr2=0x7f54a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f54a40b2900 0x7f54a40b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54b0005950 tx=0x7f54b00058e0 comp rx=0 tx=0).stop 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0076210 msgr2=0x7f54c01a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0076210 0x7f54c01a0660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00b530 tx=0x7f54ac00b8f0 comp rx=0 tx=0).stop 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 shutdown_connections 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f54a40b2900 0x7f54a40b4dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c0114490 0x7f54c01a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54c00758f0 0x7f54c01a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:02.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 --2- 172.21.15.63:0/3731400275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54c0076210 0x7f54c01a0660 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:02.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.146+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 >> 172.21.15.63:0/3731400275 conn(0x7f54c00fefd0 msgr2=0x7f54c01008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:02.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.147+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 shutdown_connections 2024-05-11T13:32:02.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:02.147+0000 7f54c4a00700 1 -- 172.21.15.63:0/3731400275 wait complete. 2024-05-11T13:32:02.150 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-11T13:32:03.049 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-05-11T13:32:01.638211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660994,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1334,"num_read_kb":777,"num_write":382,"num_write_kb":2301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380128,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561696,"statfs":{"total":1151940427776,"available":1151551176704,"internally_reserved":0,"allocated":14680064,"data_stored":8176736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005868"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546529+0000","last_change":"2024-05-11T13:28:35.223417+0000","last_active":"2024-05-11T13:31:39.546529+0000","last_peered":"2024-05-11T13:31:39.546529+0000","last_clean":"2024-05-11T13:31:39.546529+0000","last_became_active":"2024-05-11T13:28:35.222862+0000","last_became_peered":"2024-05-11T13:28:35.222862+0000","last_unstale":"2024-05-11T13:31:39.546529+0000","last_undegraded":"2024-05-11T13:31:39.546529+0000","last_fullsized":"2024-05-11T13:31:39.546529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:20.259043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.525965+0000","last_change":"2024-05-11T13:28:35.222289+0000","last_active":"2024-05-11T13:31:39.525965+0000","last_peered":"2024-05-11T13:31:39.525965+0000","last_clean":"2024-05-11T13:31:39.525965+0000","last_became_active":"2024-05-11T13:28:35.221366+0000","last_became_peered":"2024-05-11T13:28:35.221366+0000","last_unstale":"2024-05-11T13:31:39.525965+0000","last_undegraded":"2024-05-11T13:31:39.525965+0000","last_fullsized":"2024-05-11T13:31:39.525965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:02.648140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549679+0000","last_change":"2024-05-11T13:28:35.221271+0000","last_active":"2024-05-11T13:31:39.549679+0000","last_peered":"2024-05-11T13:31:39.549679+0000","last_clean":"2024-05-11T13:31:39.549679+0000","last_became_active":"2024-05-11T13:28:35.220294+0000","last_became_peered":"2024-05-11T13:28:35.220294+0000","last_unstale":"2024-05-11T13:31:39.549679+0000","last_undegraded":"2024-05-11T13:31:39.549679+0000","last_fullsized":"2024-05-11T13:31:39.549679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:04.608042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.520852+0000","last_change":"2024-05-11T13:28:35.197643+0000","last_active":"2024-05-11T13:31:39.520852+0000","last_peered":"2024-05-11T13:31:39.520852+0000","last_clean":"2024-05-11T13:31:39.520852+0000","last_became_active":"2024-05-11T13:28:35.196528+0000","last_became_peered":"2024-05-11T13:28:35.196528+0000","last_unstale":"2024-05-11T13:31:39.520852+0000","last_undegraded":"2024-05-11T13:31:39.520852+0000","last_fullsized":"2024-05-11T13:31:39.520852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:02:03.088456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.518934+0000","last_change":"2024-05-11T13:28:35.206833+0000","last_active":"2024-05-11T13:31:39.518934+0000","last_peered":"2024-05-11T13:31:39.518934+0000","last_clean":"2024-05-11T13:31:39.518934+0000","last_became_active":"2024-05-11T13:28:35.206287+0000","last_became_peered":"2024-05-11T13:28:35.206287+0000","last_unstale":"2024-05-11T13:31:39.518934+0000","last_undegraded":"2024-05-11T13:31:39.518934+0000","last_fullsized":"2024-05-11T13:31:39.518934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:25:20.576464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.529275+0000","last_change":"2024-05-11T13:28:35.218950+0000","last_active":"2024-05-11T13:31:39.529275+0000","last_peered":"2024-05-11T13:31:39.529275+0000","last_clean":"2024-05-11T13:31:39.529275+0000","last_became_active":"2024-05-11T13:28:35.218597+0000","last_became_peered":"2024-05-11T13:28:35.218597+0000","last_unstale":"2024-05-11T13:31:39.529275+0000","last_undegraded":"2024-05-11T13:31:39.529275+0000","last_fullsized":"2024-05-11T13:31:39.529275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:08:42.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.520884+0000","last_change":"2024-05-11T13:29:40.541106+0000","last_active":"2024-05-11T13:31:39.520884+0000","last_peered":"2024-05-11T13:31:39.520884+0000","last_clean":"2024-05-11T13:31:39.520884+0000","last_became_active":"2024-05-11T13:29:40.540007+0000","last_became_peered":"2024-05-11T13:29:40.540007+0000","last_unstale":"2024-05-11T13:31:39.520884+0000","last_undegraded":"2024-05-11T13:31:39.520884+0000","last_fullsized":"2024-05-11T13:31:39.520884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:00.596580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.534232+0000","last_change":"2024-05-11T13:28:35.205483+0000","last_active":"2024-05-11T13:31:39.534232+0000","last_peered":"2024-05-11T13:31:39.534232+0000","last_clean":"2024-05-11T13:31:39.534232+0000","last_became_active":"2024-05-11T13:28:35.205101+0000","last_became_peered":"2024-05-11T13:28:35.205101+0000","last_unstale":"2024-05-11T13:31:39.534232+0000","last_undegraded":"2024-05-11T13:31:39.534232+0000","last_fullsized":"2024-05-11T13:31:39.534232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:37.917464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.547438+0000","last_change":"2024-05-11T13:28:35.217372+0000","last_active":"2024-05-11T13:31:39.547438+0000","last_peered":"2024-05-11T13:31:39.547438+0000","last_clean":"2024-05-11T13:31:39.547438+0000","last_became_active":"2024-05-11T13:28:35.217021+0000","last_became_peered":"2024-05-11T13:28:35.217021+0000","last_unstale":"2024-05-11T13:31:39.547438+0000","last_undegraded":"2024-05-11T13:31:39.547438+0000","last_fullsized":"2024-05-11T13:31:39.547438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:24.273041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.532583+0000","last_change":"2024-05-11T13:29:40.548634+0000","last_active":"2024-05-11T13:31:39.532583+0000","last_peered":"2024-05-11T13:31:39.532583+0000","last_clean":"2024-05-11T13:31:39.532583+0000","last_became_active":"2024-05-11T13:29:40.547536+0000","last_became_peered":"2024-05-11T13:29:40.547536+0000","last_unstale":"2024-05-11T13:31:39.532583+0000","last_undegraded":"2024-05-11T13:31:39.532583+0000","last_fullsized":"2024-05-11T13:31:39.532583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:15.362317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545128+0000","last_change":"2024-05-11T13:28:35.214091+0000","last_active":"2024-05-11T13:31:39.545128+0000","last_peered":"2024-05-11T13:31:39.545128+0000","last_clean":"2024-05-11T13:31:39.545128+0000","last_became_active":"2024-05-11T13:28:35.213668+0000","last_became_peered":"2024-05-11T13:28:35.213668+0000","last_unstale":"2024-05-11T13:31:39.545128+0000","last_undegraded":"2024-05-11T13:31:39.545128+0000","last_fullsized":"2024-05-11T13:31:39.545128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:19:55.768203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.527465+0000","last_change":"2024-05-11T13:28:35.211874+0000","last_active":"2024-05-11T13:31:39.527465+0000","last_peered":"2024-05-11T13:31:39.527465+0000","last_clean":"2024-05-11T13:31:39.527465+0000","last_became_active":"2024-05-11T13:28:35.211503+0000","last_became_peered":"2024-05-11T13:28:35.211503+0000","last_unstale":"2024-05-11T13:31:39.527465+0000","last_undegraded":"2024-05-11T13:31:39.527465+0000","last_fullsized":"2024-05-11T13:31:39.527465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:46.884110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530668+0000","last_change":"2024-05-11T13:28:35.209308+0000","last_active":"2024-05-11T13:31:39.530668+0000","last_peered":"2024-05-11T13:31:39.530668+0000","last_clean":"2024-05-11T13:31:39.530668+0000","last_became_active":"2024-05-11T13:28:35.207927+0000","last_became_peered":"2024-05-11T13:28:35.207927+0000","last_unstale":"2024-05-11T13:31:39.530668+0000","last_undegraded":"2024-05-11T13:31:39.530668+0000","last_fullsized":"2024-05-11T13:31:39.530668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:04.541849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.541562+0000","last_change":"2024-05-11T13:28:35.208524+0000","last_active":"2024-05-11T13:31:39.541562+0000","last_peered":"2024-05-11T13:31:39.541562+0000","last_clean":"2024-05-11T13:31:39.541562+0000","last_became_active":"2024-05-11T13:28:35.207875+0000","last_became_peered":"2024-05-11T13:28:35.207875+0000","last_unstale":"2024-05-11T13:31:39.541562+0000","last_undegraded":"2024-05-11T13:31:39.541562+0000","last_fullsized":"2024-05-11T13:31:39.541562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:23:38.896916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548589+0000","last_change":"2024-05-11T13:28:35.216418+0000","last_active":"2024-05-11T13:31:39.548589+0000","last_peered":"2024-05-11T13:31:39.548589+0000","last_clean":"2024-05-11T13:31:39.548589+0000","last_became_active":"2024-05-11T13:28:35.215984+0000","last_became_peered":"2024-05-11T13:28:35.215984+0000","last_unstale":"2024-05-11T13:31:39.548589+0000","last_undegraded":"2024-05-11T13:31:39.548589+0000","last_fullsized":"2024-05-11T13:31:39.548589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:15:13.650821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.547193+0000","last_change":"2024-05-11T13:28:35.217701+0000","last_active":"2024-05-11T13:31:39.547193+0000","last_peered":"2024-05-11T13:31:39.547193+0000","last_clean":"2024-05-11T13:31:39.547193+0000","last_became_active":"2024-05-11T13:28:35.217050+0000","last_became_peered":"2024-05-11T13:28:35.217050+0000","last_unstale":"2024-05-11T13:31:39.547193+0000","last_undegraded":"2024-05-11T13:31:39.547193+0000","last_fullsized":"2024-05-11T13:31:39.547193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:29:38.274540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546742+0000","last_change":"2024-05-11T13:28:35.223794+0000","last_active":"2024-05-11T13:31:39.546742+0000","last_peered":"2024-05-11T13:31:39.546742+0000","last_clean":"2024-05-11T13:31:39.546742+0000","last_became_active":"2024-05-11T13:28:35.223388+0000","last_became_peered":"2024-05-11T13:28:35.223388+0000","last_unstale":"2024-05-11T13:31:39.546742+0000","last_undegraded":"2024-05-11T13:31:39.546742+0000","last_fullsized":"2024-05-11T13:31:39.546742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:56:13.272357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545385+0000","last_change":"2024-05-11T13:28:35.216452+0000","last_active":"2024-05-11T13:31:39.545385+0000","last_peered":"2024-05-11T13:31:39.545385+0000","last_clean":"2024-05-11T13:31:39.545385+0000","last_became_active":"2024-05-11T13:28:35.216075+0000","last_became_peered":"2024-05-11T13:28:35.216075+0000","last_unstale":"2024-05-11T13:31:39.545385+0000","last_undegraded":"2024-05-11T13:31:39.545385+0000","last_fullsized":"2024-05-11T13:31:39.545385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:23.012302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.524485+0000","last_change":"2024-05-11T13:28:35.206528+0000","last_active":"2024-05-11T13:31:39.524485+0000","last_peered":"2024-05-11T13:31:39.524485+0000","last_clean":"2024-05-11T13:31:39.524485+0000","last_became_active":"2024-05-11T13:28:35.206102+0000","last_became_peered":"2024-05-11T13:28:35.206102+0000","last_unstale":"2024-05-11T13:31:39.524485+0000","last_undegraded":"2024-05-11T13:31:39.524485+0000","last_fullsized":"2024-05-11T13:31:39.524485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.685477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.924274+0000","last_change":"2024-05-11T13:30:41.923155+0000","last_active":"2024-05-11T13:30:41.924274+0000","last_peered":"2024-05-11T13:30:41.924274+0000","last_clean":"2024-05-11T13:30:41.924274+0000","last_became_active":"2024-05-11T13:30:41.922484+0000","last_became_peered":"2024-05-11T13:30:41.922484+0000","last_unstale":"2024-05-11T13:30:41.924274+0000","last_undegraded":"2024-05-11T13:30:41.924274+0000","last_fullsized":"2024-05-11T13:30:41.924274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:52.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":181,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:59.127927+0000","last_change":"2024-05-11T13:28:30.428076+0000","last_active":"2024-05-11T13:31:59.127927+0000","last_peered":"2024-05-11T13:31:59.127927+0000","last_clean":"2024-05-11T13:31:59.127927+0000","last_became_active":"2024-05-11T13:28:30.427519+0000","last_became_peered":"2024-05-11T13:28:30.427519+0000","last_unstale":"2024-05-11T13:31:59.127927+0000","last_undegraded":"2024-05-11T13:31:59.127927+0000","last_fullsized":"2024-05-11T13:31:59.127927+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:52.808051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.533839+0000","last_change":"2024-05-11T13:28:35.207587+0000","last_active":"2024-05-11T13:31:39.533839+0000","last_peered":"2024-05-11T13:31:39.533839+0000","last_clean":"2024-05-11T13:31:39.533839+0000","last_became_active":"2024-05-11T13:28:35.206692+0000","last_became_peered":"2024-05-11T13:28:35.206692+0000","last_unstale":"2024-05-11T13:31:39.533839+0000","last_undegraded":"2024-05-11T13:31:39.533839+0000","last_fullsized":"2024-05-11T13:31:39.533839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:25.171342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939018+0000","last_change":"2024-05-11T13:28:49.682649+0000","last_active":"2024-05-11T13:30:41.939018+0000","last_peered":"2024-05-11T13:30:41.939018+0000","last_clean":"2024-05-11T13:30:41.939018+0000","last_became_active":"2024-05-11T13:28:49.682192+0000","last_became_peered":"2024-05-11T13:28:49.682192+0000","last_unstale":"2024-05-11T13:30:41.939018+0000","last_undegraded":"2024-05-11T13:30:41.939018+0000","last_fullsized":"2024-05-11T13:30:41.939018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:01.728705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.534308+0000","last_change":"2024-05-11T13:28:30.707050+0000","last_active":"2024-05-11T13:30:41.534308+0000","last_peered":"2024-05-11T13:30:41.534308+0000","last_clean":"2024-05-11T13:30:41.534308+0000","last_became_active":"2024-05-11T13:28:30.706279+0000","last_became_peered":"2024-05-11T13:28:30.706279+0000","last_unstale":"2024-05-11T13:30:41.534308+0000","last_undegraded":"2024-05-11T13:30:41.534308+0000","last_fullsized":"2024-05-11T13:30:41.534308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:14.135734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.550636+0000","last_change":"2024-05-11T13:28:35.219622+0000","last_active":"2024-05-11T13:31:39.550636+0000","last_peered":"2024-05-11T13:31:39.550636+0000","last_clean":"2024-05-11T13:31:39.550636+0000","last_became_active":"2024-05-11T13:28:35.218687+0000","last_became_peered":"2024-05-11T13:28:35.218687+0000","last_unstale":"2024-05-11T13:31:39.550636+0000","last_undegraded":"2024-05-11T13:31:39.550636+0000","last_fullsized":"2024-05-11T13:31:39.550636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:53.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939066+0000","last_change":"2024-05-11T13:28:49.685770+0000","last_active":"2024-05-11T13:30:41.939066+0000","last_peered":"2024-05-11T13:30:41.939066+0000","last_clean":"2024-05-11T13:30:41.939066+0000","last_became_active":"2024-05-11T13:28:49.685091+0000","last_became_peered":"2024-05-11T13:28:49.685091+0000","last_unstale":"2024-05-11T13:30:41.939066+0000","last_undegraded":"2024-05-11T13:30:41.939066+0000","last_fullsized":"2024-05-11T13:30:41.939066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:42.458310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921649+0000","last_change":"2024-05-11T13:28:30.435304+0000","last_active":"2024-05-11T13:30:41.921649+0000","last_peered":"2024-05-11T13:30:41.921649+0000","last_clean":"2024-05-11T13:30:41.921649+0000","last_became_active":"2024-05-11T13:28:30.434648+0000","last_became_peered":"2024-05-11T13:28:30.434648+0000","last_unstale":"2024-05-11T13:30:41.921649+0000","last_undegraded":"2024-05-11T13:30:41.921649+0000","last_fullsized":"2024-05-11T13:30:41.921649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:21.553376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549398+0000","last_change":"2024-05-11T13:28:35.210877+0000","last_active":"2024-05-11T13:31:39.549398+0000","last_peered":"2024-05-11T13:31:39.549398+0000","last_clean":"2024-05-11T13:31:39.549398+0000","last_became_active":"2024-05-11T13:28:35.207069+0000","last_became_peered":"2024-05-11T13:28:35.207069+0000","last_unstale":"2024-05-11T13:31:39.549398+0000","last_undegraded":"2024-05-11T13:31:39.549398+0000","last_fullsized":"2024-05-11T13:31:39.549398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:59.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939019+0000","last_change":"2024-05-11T13:28:49.689070+0000","last_active":"2024-05-11T13:30:41.939019+0000","last_peered":"2024-05-11T13:30:41.939019+0000","last_clean":"2024-05-11T13:30:41.939019+0000","last_became_active":"2024-05-11T13:28:49.688502+0000","last_became_peered":"2024-05-11T13:28:49.688502+0000","last_unstale":"2024-05-11T13:30:41.939019+0000","last_undegraded":"2024-05-11T13:30:41.939019+0000","last_fullsized":"2024-05-11T13:30:41.939019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:21.516074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.913426+0000","last_change":"2024-05-11T13:29:40.564494+0000","last_active":"2024-05-11T13:30:41.913426+0000","last_peered":"2024-05-11T13:30:41.913426+0000","last_clean":"2024-05-11T13:30:41.913426+0000","last_became_active":"2024-05-11T13:29:40.555386+0000","last_became_peered":"2024-05-11T13:29:40.555386+0000","last_unstale":"2024-05-11T13:30:41.913426+0000","last_undegraded":"2024-05-11T13:30:41.913426+0000","last_fullsized":"2024-05-11T13:30:41.913426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:32:50.641372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549880+0000","last_change":"2024-05-11T13:28:35.217950+0000","last_active":"2024-05-11T13:31:39.549880+0000","last_peered":"2024-05-11T13:31:39.549880+0000","last_clean":"2024-05-11T13:31:39.549880+0000","last_became_active":"2024-05-11T13:28:35.216698+0000","last_became_peered":"2024-05-11T13:28:35.216698+0000","last_unstale":"2024-05-11T13:31:39.549880+0000","last_undegraded":"2024-05-11T13:31:39.549880+0000","last_fullsized":"2024-05-11T13:31:39.549880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:01.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.924187+0000","last_change":"2024-05-11T13:30:41.923049+0000","last_active":"2024-05-11T13:30:41.924187+0000","last_peered":"2024-05-11T13:30:41.924187+0000","last_clean":"2024-05-11T13:30:41.924187+0000","last_became_active":"2024-05-11T13:30:41.922585+0000","last_became_peered":"2024-05-11T13:30:41.922585+0000","last_unstale":"2024-05-11T13:30:41.924187+0000","last_undegraded":"2024-05-11T13:30:41.924187+0000","last_fullsized":"2024-05-11T13:30:41.924187+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:34:03.716885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:56.139302+0000","last_change":"2024-05-11T13:29:40.555238+0000","last_active":"2024-05-11T13:31:56.139302+0000","last_peered":"2024-05-11T13:31:56.139302+0000","last_clean":"2024-05-11T13:31:56.139302+0000","last_became_active":"2024-05-11T13:29:40.554661+0000","last_became_peered":"2024-05-11T13:29:40.554661+0000","last_unstale":"2024-05-11T13:31:56.139302+0000","last_undegraded":"2024-05-11T13:31:56.139302+0000","last_fullsized":"2024-05-11T13:31:56.139302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:22.983649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530851+0000","last_change":"2024-05-11T13:28:35.209163+0000","last_active":"2024-05-11T13:31:39.530851+0000","last_peered":"2024-05-11T13:31:39.530851+0000","last_clean":"2024-05-11T13:31:39.530851+0000","last_became_active":"2024-05-11T13:28:35.204014+0000","last_became_peered":"2024-05-11T13:28:35.204014+0000","last_unstale":"2024-05-11T13:31:39.530851+0000","last_undegraded":"2024-05-11T13:31:39.530851+0000","last_fullsized":"2024-05-11T13:31:39.530851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:17.570863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.019977+0000","last_change":"2024-05-11T13:29:40.555520+0000","last_active":"2024-05-11T13:30:41.019977+0000","last_peered":"2024-05-11T13:30:41.019977+0000","last_clean":"2024-05-11T13:30:41.019977+0000","last_became_active":"2024-05-11T13:29:40.553952+0000","last_became_peered":"2024-05-11T13:29:40.553952+0000","last_unstale":"2024-05-11T13:30:41.019977+0000","last_undegraded":"2024-05-11T13:30:41.019977+0000","last_fullsized":"2024-05-11T13:30:41.019977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.035691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.018438+0000","last_change":"2024-05-11T13:28:30.429829+0000","last_active":"2024-05-11T13:30:41.018438+0000","last_peered":"2024-05-11T13:30:41.018438+0000","last_clean":"2024-05-11T13:30:41.018438+0000","last_became_active":"2024-05-11T13:28:30.428818+0000","last_became_peered":"2024-05-11T13:28:30.428818+0000","last_unstale":"2024-05-11T13:30:41.018438+0000","last_undegraded":"2024-05-11T13:30:41.018438+0000","last_fullsized":"2024-05-11T13:30:41.018438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:59:25.653181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535645+0000","last_change":"2024-05-11T13:28:35.214153+0000","last_active":"2024-05-11T13:31:39.535645+0000","last_peered":"2024-05-11T13:31:39.535645+0000","last_clean":"2024-05-11T13:31:39.535645+0000","last_became_active":"2024-05-11T13:28:35.213800+0000","last_became_peered":"2024-05-11T13:28:35.213800+0000","last_unstale":"2024-05-11T13:31:39.535645+0000","last_undegraded":"2024-05-11T13:31:39.535645+0000","last_fullsized":"2024-05-11T13:31:39.535645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:12.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.914038+0000","last_change":"2024-05-11T13:28:49.697320+0000","last_active":"2024-05-11T13:30:41.914038+0000","last_peered":"2024-05-11T13:30:41.914038+0000","last_clean":"2024-05-11T13:30:41.914038+0000","last_became_active":"2024-05-11T13:28:49.696751+0000","last_became_peered":"2024-05-11T13:28:49.696751+0000","last_unstale":"2024-05-11T13:30:41.914038+0000","last_undegraded":"2024-05-11T13:30:41.914038+0000","last_fullsized":"2024-05-11T13:30:41.914038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:24.617241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.015202+0000","last_change":"2024-05-11T13:28:30.423943+0000","last_active":"2024-05-11T13:30:41.015202+0000","last_peered":"2024-05-11T13:30:41.015202+0000","last_clean":"2024-05-11T13:30:41.015202+0000","last_became_active":"2024-05-11T13:28:30.423454+0000","last_became_peered":"2024-05-11T13:28:30.423454+0000","last_unstale":"2024-05-11T13:30:41.015202+0000","last_undegraded":"2024-05-11T13:30:41.015202+0000","last_fullsized":"2024-05-11T13:30:41.015202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:23:10.967537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535645+0000","last_change":"2024-05-11T13:28:35.220479+0000","last_active":"2024-05-11T13:31:39.535645+0000","last_peered":"2024-05-11T13:31:39.535645+0000","last_clean":"2024-05-11T13:31:39.535645+0000","last_became_active":"2024-05-11T13:28:35.214541+0000","last_became_peered":"2024-05-11T13:28:35.214541+0000","last_unstale":"2024-05-11T13:31:39.535645+0000","last_undegraded":"2024-05-11T13:31:39.535645+0000","last_fullsized":"2024-05-11T13:31:39.535645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:07.746301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.827235+0000","last_change":"2024-05-11T13:28:49.691867+0000","last_active":"2024-05-11T13:30:41.827235+0000","last_peered":"2024-05-11T13:30:41.827235+0000","last_clean":"2024-05-11T13:30:41.827235+0000","last_became_active":"2024-05-11T13:28:49.690045+0000","last_became_peered":"2024-05-11T13:28:49.690045+0000","last_unstale":"2024-05-11T13:30:41.827235+0000","last_undegraded":"2024-05-11T13:30:41.827235+0000","last_fullsized":"2024-05-11T13:30:41.827235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:16.220334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.015294+0000","last_change":"2024-05-11T13:28:30.436354+0000","last_active":"2024-05-11T13:30:41.015294+0000","last_peered":"2024-05-11T13:30:41.015294+0000","last_clean":"2024-05-11T13:30:41.015294+0000","last_became_active":"2024-05-11T13:28:30.435983+0000","last_became_peered":"2024-05-11T13:28:30.435983+0000","last_unstale":"2024-05-11T13:30:41.015294+0000","last_undegraded":"2024-05-11T13:30:41.015294+0000","last_fullsized":"2024-05-11T13:30:41.015294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:42.442829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.532445+0000","last_change":"2024-05-11T13:28:35.207284+0000","last_active":"2024-05-11T13:31:39.532445+0000","last_peered":"2024-05-11T13:31:39.532445+0000","last_clean":"2024-05-11T13:31:39.532445+0000","last_became_active":"2024-05-11T13:28:35.206945+0000","last_became_peered":"2024-05-11T13:28:35.206945+0000","last_unstale":"2024-05-11T13:31:39.532445+0000","last_undegraded":"2024-05-11T13:31:39.532445+0000","last_fullsized":"2024-05-11T13:31:39.532445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:18.493659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.533467+0000","last_change":"2024-05-11T13:28:49.689558+0000","last_active":"2024-05-11T13:30:41.533467+0000","last_peered":"2024-05-11T13:30:41.533467+0000","last_clean":"2024-05-11T13:30:41.533467+0000","last_became_active":"2024-05-11T13:28:49.689156+0000","last_became_peered":"2024-05-11T13:28:49.689156+0000","last_unstale":"2024-05-11T13:30:41.533467+0000","last_undegraded":"2024-05-11T13:30:41.533467+0000","last_fullsized":"2024-05-11T13:30:41.533467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:21.530111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":293,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:32:00.149695+0000","last_change":"2024-05-11T13:28:30.434534+0000","last_active":"2024-05-11T13:32:00.149695+0000","last_peered":"2024-05-11T13:32:00.149695+0000","last_clean":"2024-05-11T13:32:00.149695+0000","last_became_active":"2024-05-11T13:28:30.433983+0000","last_became_peered":"2024-05-11T13:28:30.433983+0000","last_unstale":"2024-05-11T13:32:00.149695+0000","last_undegraded":"2024-05-11T13:32:00.149695+0000","last_fullsized":"2024-05-11T13:32:00.149695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:16.056662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":244,"num_read_kb":244,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549771+0000","last_change":"2024-05-11T13:28:35.205021+0000","last_active":"2024-05-11T13:31:39.549771+0000","last_peered":"2024-05-11T13:31:39.549771+0000","last_clean":"2024-05-11T13:31:39.549771+0000","last_became_active":"2024-05-11T13:28:35.204243+0000","last_became_peered":"2024-05-11T13:28:35.204243+0000","last_unstale":"2024-05-11T13:31:39.549771+0000","last_undegraded":"2024-05-11T13:31:39.549771+0000","last_fullsized":"2024-05-11T13:31:39.549771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:12.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.767631+0000","last_change":"2024-05-11T13:28:49.677392+0000","last_active":"2024-05-11T13:30:41.767631+0000","last_peered":"2024-05-11T13:30:41.767631+0000","last_clean":"2024-05-11T13:30:41.767631+0000","last_became_active":"2024-05-11T13:28:49.674008+0000","last_became_peered":"2024-05-11T13:28:49.674008+0000","last_unstale":"2024-05-11T13:30:41.767631+0000","last_undegraded":"2024-05-11T13:30:41.767631+0000","last_fullsized":"2024-05-11T13:30:41.767631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:47:14.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.014459+0000","last_change":"2024-05-11T13:28:30.429500+0000","last_active":"2024-05-11T13:30:41.014459+0000","last_peered":"2024-05-11T13:30:41.014459+0000","last_clean":"2024-05-11T13:30:41.014459+0000","last_became_active":"2024-05-11T13:28:30.429049+0000","last_became_peered":"2024-05-11T13:28:30.429049+0000","last_unstale":"2024-05-11T13:30:41.014459+0000","last_undegraded":"2024-05-11T13:30:41.014459+0000","last_fullsized":"2024-05-11T13:30:41.014459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:00.533603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.541794+0000","last_change":"2024-05-11T13:28:35.223764+0000","last_active":"2024-05-11T13:31:39.541794+0000","last_peered":"2024-05-11T13:31:39.541794+0000","last_clean":"2024-05-11T13:31:39.541794+0000","last_became_active":"2024-05-11T13:28:35.223016+0000","last_became_peered":"2024-05-11T13:28:35.223016+0000","last_unstale":"2024-05-11T13:31:39.541794+0000","last_undegraded":"2024-05-11T13:31:39.541794+0000","last_fullsized":"2024-05-11T13:31:39.541794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:42.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.938999+0000","last_change":"2024-05-11T13:28:49.683733+0000","last_active":"2024-05-11T13:30:41.938999+0000","last_peered":"2024-05-11T13:30:41.938999+0000","last_clean":"2024-05-11T13:30:41.938999+0000","last_became_active":"2024-05-11T13:28:49.681245+0000","last_became_peered":"2024-05-11T13:28:49.681245+0000","last_unstale":"2024-05-11T13:30:41.938999+0000","last_undegraded":"2024-05-11T13:30:41.938999+0000","last_fullsized":"2024-05-11T13:30:41.938999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:03.386943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016649+0000","last_change":"2024-05-11T13:28:30.707394+0000","last_active":"2024-05-11T13:30:41.016649+0000","last_peered":"2024-05-11T13:30:41.016649+0000","last_clean":"2024-05-11T13:30:41.016649+0000","last_became_active":"2024-05-11T13:28:30.706114+0000","last_became_peered":"2024-05-11T13:28:30.706114+0000","last_unstale":"2024-05-11T13:30:41.016649+0000","last_undegraded":"2024-05-11T13:30:41.016649+0000","last_fullsized":"2024-05-11T13:30:41.016649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:53:04.599800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548448+0000","last_change":"2024-05-11T13:28:35.219836+0000","last_active":"2024-05-11T13:31:39.548448+0000","last_peered":"2024-05-11T13:31:39.548448+0000","last_clean":"2024-05-11T13:31:39.548448+0000","last_became_active":"2024-05-11T13:28:35.214474+0000","last_became_peered":"2024-05-11T13:28:35.214474+0000","last_unstale":"2024-05-11T13:31:39.548448+0000","last_undegraded":"2024-05-11T13:31:39.548448+0000","last_fullsized":"2024-05-11T13:31:39.548448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:02.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.767625+0000","last_change":"2024-05-11T13:28:49.694677+0000","last_active":"2024-05-11T13:30:41.767625+0000","last_peered":"2024-05-11T13:30:41.767625+0000","last_clean":"2024-05-11T13:30:41.767625+0000","last_became_active":"2024-05-11T13:28:49.694176+0000","last_became_peered":"2024-05-11T13:28:49.694176+0000","last_unstale":"2024-05-11T13:30:41.767625+0000","last_undegraded":"2024-05-11T13:30:41.767625+0000","last_fullsized":"2024-05-11T13:30:41.767625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:35.461937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":338,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:32:00.172308+0000","last_change":"2024-05-11T13:28:30.433202+0000","last_active":"2024-05-11T13:32:00.172308+0000","last_peered":"2024-05-11T13:32:00.172308+0000","last_clean":"2024-05-11T13:32:00.172308+0000","last_became_active":"2024-05-11T13:28:30.429269+0000","last_became_peered":"2024-05-11T13:28:30.429269+0000","last_unstale":"2024-05-11T13:32:00.172308+0000","last_undegraded":"2024-05-11T13:32:00.172308+0000","last_fullsized":"2024-05-11T13:32:00.172308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:27:15.620717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":255,"num_read_kb":189,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.524649+0000","last_change":"2024-05-11T13:28:35.210882+0000","last_active":"2024-05-11T13:31:39.524649+0000","last_peered":"2024-05-11T13:31:39.524649+0000","last_clean":"2024-05-11T13:31:39.524649+0000","last_became_active":"2024-05-11T13:28:35.210506+0000","last_became_peered":"2024-05-11T13:28:35.210506+0000","last_unstale":"2024-05-11T13:31:39.524649+0000","last_undegraded":"2024-05-11T13:31:39.524649+0000","last_fullsized":"2024-05-11T13:31:39.524649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:45.752666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017111+0000","last_change":"2024-05-11T13:28:49.693254+0000","last_active":"2024-05-11T13:30:41.017111+0000","last_peered":"2024-05-11T13:30:41.017111+0000","last_clean":"2024-05-11T13:30:41.017111+0000","last_became_active":"2024-05-11T13:28:49.690307+0000","last_became_peered":"2024-05-11T13:28:49.690307+0000","last_unstale":"2024-05-11T13:30:41.017111+0000","last_undegraded":"2024-05-11T13:30:41.017111+0000","last_fullsized":"2024-05-11T13:30:41.017111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:41:31.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.019481+0000","last_change":"2024-05-11T13:28:30.429421+0000","last_active":"2024-05-11T13:30:41.019481+0000","last_peered":"2024-05-11T13:30:41.019481+0000","last_clean":"2024-05-11T13:30:41.019481+0000","last_became_active":"2024-05-11T13:28:30.429032+0000","last_became_peered":"2024-05-11T13:28:30.429032+0000","last_unstale":"2024-05-11T13:30:41.019481+0000","last_undegraded":"2024-05-11T13:30:41.019481+0000","last_fullsized":"2024-05-11T13:30:41.019481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:06.234650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.538071+0000","last_change":"2024-05-11T13:28:35.207709+0000","last_active":"2024-05-11T13:31:39.538071+0000","last_peered":"2024-05-11T13:31:39.538071+0000","last_clean":"2024-05-11T13:31:39.538071+0000","last_became_active":"2024-05-11T13:28:35.207222+0000","last_became_peered":"2024-05-11T13:28:35.207222+0000","last_unstale":"2024-05-11T13:31:39.538071+0000","last_undegraded":"2024-05-11T13:31:39.538071+0000","last_fullsized":"2024-05-11T13:31:39.538071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:20.023565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.913230+0000","last_change":"2024-05-11T13:28:49.692046+0000","last_active":"2024-05-11T13:30:41.913230+0000","last_peered":"2024-05-11T13:30:41.913230+0000","last_clean":"2024-05-11T13:30:41.913230+0000","last_became_active":"2024-05-11T13:28:49.691618+0000","last_became_peered":"2024-05-11T13:28:49.691618+0000","last_unstale":"2024-05-11T13:30:41.913230+0000","last_undegraded":"2024-05-11T13:30:41.913230+0000","last_fullsized":"2024-05-11T13:30:41.913230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:05.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017218+0000","last_change":"2024-05-11T13:28:30.435459+0000","last_active":"2024-05-11T13:30:41.017218+0000","last_peered":"2024-05-11T13:30:41.017218+0000","last_clean":"2024-05-11T13:30:41.017218+0000","last_became_active":"2024-05-11T13:28:30.434997+0000","last_became_peered":"2024-05-11T13:28:30.434997+0000","last_unstale":"2024-05-11T13:30:41.017218+0000","last_undegraded":"2024-05-11T13:30:41.017218+0000","last_fullsized":"2024-05-11T13:30:41.017218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:36.280546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548457+0000","last_change":"2024-05-11T13:28:35.211214+0000","last_active":"2024-05-11T13:31:39.548457+0000","last_peered":"2024-05-11T13:31:39.548457+0000","last_clean":"2024-05-11T13:31:39.548457+0000","last_became_active":"2024-05-11T13:28:35.210777+0000","last_became_peered":"2024-05-11T13:28:35.210777+0000","last_unstale":"2024-05-11T13:31:39.548457+0000","last_undegraded":"2024-05-11T13:31:39.548457+0000","last_fullsized":"2024-05-11T13:31:39.548457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:53.238549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.767860+0000","last_change":"2024-05-11T13:28:49.682336+0000","last_active":"2024-05-11T13:30:41.767860+0000","last_peered":"2024-05-11T13:30:41.767860+0000","last_clean":"2024-05-11T13:30:41.767860+0000","last_became_active":"2024-05-11T13:28:49.679081+0000","last_became_peered":"2024-05-11T13:28:49.679081+0000","last_unstale":"2024-05-11T13:30:41.767860+0000","last_undegraded":"2024-05-11T13:30:41.767860+0000","last_fullsized":"2024-05-11T13:30:41.767860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:00:21.149262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":175,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:57.164260+0000","last_change":"2024-05-11T13:28:30.432165+0000","last_active":"2024-05-11T13:31:57.164260+0000","last_peered":"2024-05-11T13:31:57.164260+0000","last_clean":"2024-05-11T13:31:57.164260+0000","last_became_active":"2024-05-11T13:28:30.431181+0000","last_became_peered":"2024-05-11T13:28:30.431181+0000","last_unstale":"2024-05-11T13:31:57.164260+0000","last_undegraded":"2024-05-11T13:31:57.164260+0000","last_fullsized":"2024-05-11T13:31:57.164260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:26:50.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.536072+0000","last_change":"2024-05-11T13:28:35.205662+0000","last_active":"2024-05-11T13:31:39.536072+0000","last_peered":"2024-05-11T13:31:39.536072+0000","last_clean":"2024-05-11T13:31:39.536072+0000","last_became_active":"2024-05-11T13:28:35.205276+0000","last_became_peered":"2024-05-11T13:28:35.205276+0000","last_unstale":"2024-05-11T13:31:39.536072+0000","last_undegraded":"2024-05-11T13:31:39.536072+0000","last_fullsized":"2024-05-11T13:31:39.536072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:36.308402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017145+0000","last_change":"2024-05-11T13:28:49.689705+0000","last_active":"2024-05-11T13:30:41.017145+0000","last_peered":"2024-05-11T13:30:41.017145+0000","last_clean":"2024-05-11T13:30:41.017145+0000","last_became_active":"2024-05-11T13:28:49.689223+0000","last_became_peered":"2024-05-11T13:28:49.689223+0000","last_unstale":"2024-05-11T13:30:41.017145+0000","last_undegraded":"2024-05-11T13:30:41.017145+0000","last_fullsized":"2024-05-11T13:30:41.017145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:37:20.727773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921035+0000","last_change":"2024-05-11T13:28:30.709014+0000","last_active":"2024-05-11T13:30:41.921035+0000","last_peered":"2024-05-11T13:30:41.921035+0000","last_clean":"2024-05-11T13:30:41.921035+0000","last_became_active":"2024-05-11T13:28:30.708000+0000","last_became_peered":"2024-05-11T13:28:30.708000+0000","last_unstale":"2024-05-11T13:30:41.921035+0000","last_undegraded":"2024-05-11T13:30:41.921035+0000","last_fullsized":"2024-05-11T13:30:41.921035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:00.813415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.544748+0000","last_change":"2024-05-11T13:28:35.215149+0000","last_active":"2024-05-11T13:31:39.544748+0000","last_peered":"2024-05-11T13:31:39.544748+0000","last_clean":"2024-05-11T13:31:39.544748+0000","last_became_active":"2024-05-11T13:28:35.214792+0000","last_became_peered":"2024-05-11T13:28:35.214792+0000","last_unstale":"2024-05-11T13:31:39.544748+0000","last_undegraded":"2024-05-11T13:31:39.544748+0000","last_fullsized":"2024-05-11T13:31:39.544748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016602+0000","last_change":"2024-05-11T13:28:49.686347+0000","last_active":"2024-05-11T13:30:41.016602+0000","last_peered":"2024-05-11T13:30:41.016602+0000","last_clean":"2024-05-11T13:30:41.016602+0000","last_became_active":"2024-05-11T13:28:49.685391+0000","last_became_peered":"2024-05-11T13:28:49.685391+0000","last_unstale":"2024-05-11T13:30:41.016602+0000","last_undegraded":"2024-05-11T13:30:41.016602+0000","last_fullsized":"2024-05-11T13:30:41.016602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:24:50.347627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.914009+0000","last_change":"2024-05-11T13:29:40.560815+0000","last_active":"2024-05-11T13:30:41.914009+0000","last_peered":"2024-05-11T13:30:41.914009+0000","last_clean":"2024-05-11T13:30:41.914009+0000","last_became_active":"2024-05-11T13:29:40.553031+0000","last_became_peered":"2024-05-11T13:29:40.553031+0000","last_unstale":"2024-05-11T13:30:41.914009+0000","last_undegraded":"2024-05-11T13:30:41.914009+0000","last_fullsized":"2024-05-11T13:30:41.914009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:27.437654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.531554+0000","last_change":"2024-05-11T13:28:35.218377+0000","last_active":"2024-05-11T13:31:39.531554+0000","last_peered":"2024-05-11T13:31:39.531554+0000","last_clean":"2024-05-11T13:31:39.531554+0000","last_became_active":"2024-05-11T13:28:35.217922+0000","last_became_peered":"2024-05-11T13:28:35.217922+0000","last_unstale":"2024-05-11T13:31:39.531554+0000","last_undegraded":"2024-05-11T13:31:39.531554+0000","last_fullsized":"2024-05-11T13:31:39.531554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:58.383381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.920277+0000","last_change":"2024-05-11T13:28:49.723400+0000","last_active":"2024-05-11T13:30:41.920277+0000","last_peered":"2024-05-11T13:30:41.920277+0000","last_clean":"2024-05-11T13:30:41.920277+0000","last_became_active":"2024-05-11T13:28:49.722675+0000","last_became_peered":"2024-05-11T13:28:49.722675+0000","last_unstale":"2024-05-11T13:30:41.920277+0000","last_undegraded":"2024-05-11T13:30:41.920277+0000","last_fullsized":"2024-05-11T13:30:41.920277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:14:44.810541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.768834+0000","last_change":"2024-05-11T13:28:30.710667+0000","last_active":"2024-05-11T13:30:41.768834+0000","last_peered":"2024-05-11T13:30:41.768834+0000","last_clean":"2024-05-11T13:30:41.768834+0000","last_became_active":"2024-05-11T13:28:30.709933+0000","last_became_peered":"2024-05-11T13:28:30.709933+0000","last_unstale":"2024-05-11T13:30:41.768834+0000","last_undegraded":"2024-05-11T13:30:41.768834+0000","last_fullsized":"2024-05-11T13:30:41.768834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:34.942946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.533224+0000","last_change":"2024-05-11T13:28:35.215691+0000","last_active":"2024-05-11T13:31:39.533224+0000","last_peered":"2024-05-11T13:31:39.533224+0000","last_clean":"2024-05-11T13:31:39.533224+0000","last_became_active":"2024-05-11T13:28:35.209873+0000","last_became_peered":"2024-05-11T13:28:35.209873+0000","last_unstale":"2024-05-11T13:31:39.533224+0000","last_undegraded":"2024-05-11T13:31:39.533224+0000","last_fullsized":"2024-05-11T13:31:39.533224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:42:12.935570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016763+0000","last_change":"2024-05-11T13:28:49.694322+0000","last_active":"2024-05-11T13:30:41.016763+0000","last_peered":"2024-05-11T13:30:41.016763+0000","last_clean":"2024-05-11T13:30:41.016763+0000","last_became_active":"2024-05-11T13:28:49.690900+0000","last_became_peered":"2024-05-11T13:28:49.690900+0000","last_unstale":"2024-05-11T13:30:41.016763+0000","last_undegraded":"2024-05-11T13:30:41.016763+0000","last_fullsized":"2024-05-11T13:30:41.016763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:00:07.433371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":174,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:57.124770+0000","last_change":"2024-05-11T13:28:30.432529+0000","last_active":"2024-05-11T13:31:57.124770+0000","last_peered":"2024-05-11T13:31:57.124770+0000","last_clean":"2024-05-11T13:31:57.124770+0000","last_became_active":"2024-05-11T13:28:30.428077+0000","last_became_peered":"2024-05-11T13:28:30.428077+0000","last_unstale":"2024-05-11T13:31:57.124770+0000","last_undegraded":"2024-05-11T13:31:57.124770+0000","last_fullsized":"2024-05-11T13:31:57.124770+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:42.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.539136+0000","last_change":"2024-05-11T13:28:35.219160+0000","last_active":"2024-05-11T13:31:39.539136+0000","last_peered":"2024-05-11T13:31:39.539136+0000","last_clean":"2024-05-11T13:31:39.539136+0000","last_became_active":"2024-05-11T13:28:35.216810+0000","last_became_peered":"2024-05-11T13:28:35.216810+0000","last_unstale":"2024-05-11T13:31:39.539136+0000","last_undegraded":"2024-05-11T13:31:39.539136+0000","last_fullsized":"2024-05-11T13:31:39.539136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:35:54.465030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017120+0000","last_change":"2024-05-11T13:28:49.725190+0000","last_active":"2024-05-11T13:30:41.017120+0000","last_peered":"2024-05-11T13:30:41.017120+0000","last_clean":"2024-05-11T13:30:41.017120+0000","last_became_active":"2024-05-11T13:28:49.724193+0000","last_became_peered":"2024-05-11T13:28:49.724193+0000","last_unstale":"2024-05-11T13:30:41.017120+0000","last_undegraded":"2024-05-11T13:30:41.017120+0000","last_fullsized":"2024-05-11T13:30:41.017120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:47:41.181606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530900+0000","last_change":"2024-05-11T13:28:35.211262+0000","last_active":"2024-05-11T13:31:39.530900+0000","last_peered":"2024-05-11T13:31:39.530900+0000","last_clean":"2024-05-11T13:31:39.530900+0000","last_became_active":"2024-05-11T13:28:35.210859+0000","last_became_peered":"2024-05-11T13:28:35.210859+0000","last_unstale":"2024-05-11T13:31:39.530900+0000","last_undegraded":"2024-05-11T13:31:39.530900+0000","last_fullsized":"2024-05-11T13:31:39.530900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:40:33.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":171,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:56.160407+0000","last_change":"2024-05-11T13:28:30.707326+0000","last_active":"2024-05-11T13:31:56.160407+0000","last_peered":"2024-05-11T13:31:56.160407+0000","last_clean":"2024-05-11T13:31:56.160407+0000","last_became_active":"2024-05-11T13:28:30.705496+0000","last_became_peered":"2024-05-11T13:28:30.705496+0000","last_unstale":"2024-05-11T13:31:56.160407+0000","last_undegraded":"2024-05-11T13:31:56.160407+0000","last_fullsized":"2024-05-11T13:31:56.160407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:48.117551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.014636+0000","last_change":"2024-05-11T13:28:49.687260+0000","last_active":"2024-05-11T13:30:41.014636+0000","last_peered":"2024-05-11T13:30:41.014636+0000","last_clean":"2024-05-11T13:30:41.014636+0000","last_became_active":"2024-05-11T13:28:49.686593+0000","last_became_peered":"2024-05-11T13:28:49.686593+0000","last_unstale":"2024-05-11T13:30:41.014636+0000","last_undegraded":"2024-05-11T13:30:41.014636+0000","last_fullsized":"2024-05-11T13:30:41.014636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:48.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545582+0000","last_change":"2024-05-11T13:28:35.216643+0000","last_active":"2024-05-11T13:31:39.545582+0000","last_peered":"2024-05-11T13:31:39.545582+0000","last_clean":"2024-05-11T13:31:39.545582+0000","last_became_active":"2024-05-11T13:28:35.216300+0000","last_became_peered":"2024-05-11T13:28:35.216300+0000","last_unstale":"2024-05-11T13:31:39.545582+0000","last_undegraded":"2024-05-11T13:31:39.545582+0000","last_fullsized":"2024-05-11T13:31:39.545582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:20:01.351678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939899+0000","last_change":"2024-05-11T13:28:30.427465+0000","last_active":"2024-05-11T13:30:41.939899+0000","last_peered":"2024-05-11T13:30:41.939899+0000","last_clean":"2024-05-11T13:30:41.939899+0000","last_became_active":"2024-05-11T13:28:30.426968+0000","last_became_peered":"2024-05-11T13:28:30.426968+0000","last_unstale":"2024-05-11T13:30:41.939899+0000","last_undegraded":"2024-05-11T13:30:41.939899+0000","last_fullsized":"2024-05-11T13:30:41.939899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:02:44.969069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.013859+0000","last_change":"2024-05-11T13:26:37.738725+0000","last_active":"2024-05-11T13:30:41.013859+0000","last_peered":"2024-05-11T13:30:41.013859+0000","last_clean":"2024-05-11T13:30:41.013859+0000","last_became_active":"2024-05-11T13:26:37.724302+0000","last_became_peered":"2024-05-11T13:26:37.724302+0000","last_unstale":"2024-05-11T13:30:41.013859+0000","last_undegraded":"2024-05-11T13:30:41.013859+0000","last_fullsized":"2024-05-11T13:30:41.013859+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_clean_scrub_stamp":"2024-05-11T13:21:55.918148+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:32:56.277780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016431+0000","last_change":"2024-05-11T13:28:49.723829+0000","last_active":"2024-05-11T13:30:41.016431+0000","last_peered":"2024-05-11T13:30:41.016431+0000","last_clean":"2024-05-11T13:30:41.016431+0000","last_became_active":"2024-05-11T13:28:49.723190+0000","last_became_peered":"2024-05-11T13:28:49.723190+0000","last_unstale":"2024-05-11T13:30:41.016431+0000","last_undegraded":"2024-05-11T13:30:41.016431+0000","last_fullsized":"2024-05-11T13:30:41.016431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.742374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.525461+0000","last_change":"2024-05-11T13:28:35.213486+0000","last_active":"2024-05-11T13:31:39.525461+0000","last_peered":"2024-05-11T13:31:39.525461+0000","last_clean":"2024-05-11T13:31:39.525461+0000","last_became_active":"2024-05-11T13:28:35.211962+0000","last_became_peered":"2024-05-11T13:28:35.211962+0000","last_unstale":"2024-05-11T13:31:39.525461+0000","last_undegraded":"2024-05-11T13:31:39.525461+0000","last_fullsized":"2024-05-11T13:31:39.525461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:28:34.046898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.013954+0000","last_change":"2024-05-11T13:28:30.431023+0000","last_active":"2024-05-11T13:30:41.013954+0000","last_peered":"2024-05-11T13:30:41.013954+0000","last_clean":"2024-05-11T13:30:41.013954+0000","last_became_active":"2024-05-11T13:28:30.430609+0000","last_became_peered":"2024-05-11T13:28:30.430609+0000","last_unstale":"2024-05-11T13:30:41.013954+0000","last_undegraded":"2024-05-11T13:30:41.013954+0000","last_fullsized":"2024-05-11T13:30:41.013954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:50.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.913253+0000","last_change":"2024-05-11T13:28:49.695550+0000","last_active":"2024-05-11T13:30:41.913253+0000","last_peered":"2024-05-11T13:30:41.913253+0000","last_clean":"2024-05-11T13:30:41.913253+0000","last_became_active":"2024-05-11T13:28:49.694350+0000","last_became_peered":"2024-05-11T13:28:49.694350+0000","last_unstale":"2024-05-11T13:30:41.913253+0000","last_undegraded":"2024-05-11T13:30:41.913253+0000","last_fullsized":"2024-05-11T13:30:41.913253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:49.617673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921612+0000","last_change":"2024-05-11T13:28:30.710826+0000","last_active":"2024-05-11T13:30:41.921612+0000","last_peered":"2024-05-11T13:30:41.921612+0000","last_clean":"2024-05-11T13:30:41.921612+0000","last_became_active":"2024-05-11T13:28:30.709854+0000","last_became_peered":"2024-05-11T13:28:30.709854+0000","last_unstale":"2024-05-11T13:30:41.921612+0000","last_undegraded":"2024-05-11T13:30:41.921612+0000","last_fullsized":"2024-05-11T13:30:41.921612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:35:16.384559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.527286+0000","last_change":"2024-05-11T13:28:35.205521+0000","last_active":"2024-05-11T13:31:39.527286+0000","last_peered":"2024-05-11T13:31:39.527286+0000","last_clean":"2024-05-11T13:31:39.527286+0000","last_became_active":"2024-05-11T13:28:35.205167+0000","last_became_peered":"2024-05-11T13:28:35.205167+0000","last_unstale":"2024-05-11T13:31:39.527286+0000","last_undegraded":"2024-05-11T13:31:39.527286+0000","last_fullsized":"2024-05-11T13:31:39.527286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:10:27.473991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.532268+0000","last_change":"2024-05-11T13:28:49.723652+0000","last_active":"2024-05-11T13:30:41.532268+0000","last_peered":"2024-05-11T13:30:41.532268+0000","last_clean":"2024-05-11T13:30:41.532268+0000","last_became_active":"2024-05-11T13:28:49.723135+0000","last_became_peered":"2024-05-11T13:28:49.723135+0000","last_unstale":"2024-05-11T13:30:41.532268+0000","last_undegraded":"2024-05-11T13:30:41.532268+0000","last_fullsized":"2024-05-11T13:30:41.532268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:21:18.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.768826+0000","last_change":"2024-05-11T13:28:30.428197+0000","last_active":"2024-05-11T13:30:41.768826+0000","last_peered":"2024-05-11T13:30:41.768826+0000","last_clean":"2024-05-11T13:30:41.768826+0000","last_became_active":"2024-05-11T13:28:30.427753+0000","last_became_peered":"2024-05-11T13:28:30.427753+0000","last_unstale":"2024-05-11T13:30:41.768826+0000","last_undegraded":"2024-05-11T13:30:41.768826+0000","last_fullsized":"2024-05-11T13:30:41.768826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:52:54.165993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.538630+0000","last_change":"2024-05-11T13:29:40.541861+0000","last_active":"2024-05-11T13:31:39.538630+0000","last_peered":"2024-05-11T13:31:39.538630+0000","last_clean":"2024-05-11T13:31:39.538630+0000","last_became_active":"2024-05-11T13:29:40.541395+0000","last_became_peered":"2024-05-11T13:29:40.541395+0000","last_unstale":"2024-05-11T13:31:39.538630+0000","last_undegraded":"2024-05-11T13:31:39.538630+0000","last_fullsized":"2024-05-11T13:31:39.538630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:50:39.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.018357+0000","last_change":"2024-05-11T13:28:49.677549+0000","last_active":"2024-05-11T13:30:41.018357+0000","last_peered":"2024-05-11T13:30:41.018357+0000","last_clean":"2024-05-11T13:30:41.018357+0000","last_became_active":"2024-05-11T13:28:49.676625+0000","last_became_peered":"2024-05-11T13:28:49.676625+0000","last_unstale":"2024-05-11T13:30:41.018357+0000","last_undegraded":"2024-05-11T13:30:41.018357+0000","last_fullsized":"2024-05-11T13:30:41.018357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:44.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939670+0000","last_change":"2024-05-11T13:28:30.440945+0000","last_active":"2024-05-11T13:30:41.939670+0000","last_peered":"2024-05-11T13:30:41.939670+0000","last_clean":"2024-05-11T13:30:41.939670+0000","last_became_active":"2024-05-11T13:28:30.440595+0000","last_became_peered":"2024-05-11T13:28:30.440595+0000","last_unstale":"2024-05-11T13:30:41.939670+0000","last_undegraded":"2024-05-11T13:30:41.939670+0000","last_fullsized":"2024-05-11T13:30:41.939670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:51:10.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549451+0000","last_change":"2024-05-11T13:28:35.214207+0000","last_active":"2024-05-11T13:31:39.549451+0000","last_peered":"2024-05-11T13:31:39.549451+0000","last_clean":"2024-05-11T13:31:39.549451+0000","last_became_active":"2024-05-11T13:28:35.213844+0000","last_became_peered":"2024-05-11T13:28:35.213844+0000","last_unstale":"2024-05-11T13:31:39.549451+0000","last_undegraded":"2024-05-11T13:31:39.549451+0000","last_fullsized":"2024-05-11T13:31:39.549451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:53.063329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939373+0000","last_change":"2024-05-11T13:28:49.725505+0000","last_active":"2024-05-11T13:30:41.939373+0000","last_peered":"2024-05-11T13:30:41.939373+0000","last_clean":"2024-05-11T13:30:41.939373+0000","last_became_active":"2024-05-11T13:28:49.723613+0000","last_became_peered":"2024-05-11T13:28:49.723613+0000","last_unstale":"2024-05-11T13:30:41.939373+0000","last_undegraded":"2024-05-11T13:30:41.939373+0000","last_fullsized":"2024-05-11T13:30:41.939373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:54:30.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:42.067588+0000","last_change":"2024-05-11T13:28:30.707025+0000","last_active":"2024-05-11T13:30:42.067588+0000","last_peered":"2024-05-11T13:30:42.067588+0000","last_clean":"2024-05-11T13:30:42.067588+0000","last_became_active":"2024-05-11T13:28:30.705853+0000","last_became_peered":"2024-05-11T13:28:30.705853+0000","last_unstale":"2024-05-11T13:30:42.067588+0000","last_undegraded":"2024-05-11T13:30:42.067588+0000","last_fullsized":"2024-05-11T13:30:42.067588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:15:04.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.483892+0000","last_change":"2024-05-11T13:28:35.214682+0000","last_active":"2024-05-11T13:31:39.483892+0000","last_peered":"2024-05-11T13:31:39.483892+0000","last_clean":"2024-05-11T13:31:39.483892+0000","last_became_active":"2024-05-11T13:28:35.211581+0000","last_became_peered":"2024-05-11T13:28:35.211581+0000","last_unstale":"2024-05-11T13:31:39.483892+0000","last_undegraded":"2024-05-11T13:31:39.483892+0000","last_fullsized":"2024-05-11T13:31:39.483892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:19.231675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.536065+0000","last_change":"2024-05-11T13:30:41.536065+0000","last_active":"2024-05-11T13:30:41.536065+0000","last_peered":"2024-05-11T13:30:41.536065+0000","last_clean":"2024-05-11T13:30:41.536065+0000","last_became_active":"2024-05-11T13:30:41.535466+0000","last_became_peered":"2024-05-11T13:30:41.535466+0000","last_unstale":"2024-05-11T13:30:41.536065+0000","last_undegraded":"2024-05-11T13:30:41.536065+0000","last_fullsized":"2024-05-11T13:30:41.536065+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:18.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.533479+0000","last_change":"2024-05-11T13:28:30.419444+0000","last_active":"2024-05-11T13:30:41.533479+0000","last_peered":"2024-05-11T13:30:41.533479+0000","last_clean":"2024-05-11T13:30:41.533479+0000","last_became_active":"2024-05-11T13:28:30.419064+0000","last_became_peered":"2024-05-11T13:28:30.419064+0000","last_unstale":"2024-05-11T13:30:41.533479+0000","last_undegraded":"2024-05-11T13:30:41.533479+0000","last_fullsized":"2024-05-11T13:30:41.533479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:02.567403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546675+0000","last_change":"2024-05-11T13:28:35.209347+0000","last_active":"2024-05-11T13:31:39.546675+0000","last_peered":"2024-05-11T13:31:39.546675+0000","last_clean":"2024-05-11T13:31:39.546675+0000","last_became_active":"2024-05-11T13:28:35.208912+0000","last_became_peered":"2024-05-11T13:28:35.208912+0000","last_unstale":"2024-05-11T13:31:39.546675+0000","last_undegraded":"2024-05-11T13:31:39.546675+0000","last_fullsized":"2024-05-11T13:31:39.546675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:08:30.832941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939832+0000","last_change":"2024-05-11T13:28:49.677398+0000","last_active":"2024-05-11T13:30:41.939832+0000","last_peered":"2024-05-11T13:30:41.939832+0000","last_clean":"2024-05-11T13:30:41.939832+0000","last_became_active":"2024-05-11T13:28:49.676725+0000","last_became_peered":"2024-05-11T13:28:49.676725+0000","last_unstale":"2024-05-11T13:30:41.939832+0000","last_undegraded":"2024-05-11T13:30:41.939832+0000","last_fullsized":"2024-05-11T13:30:41.939832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:32.476049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.019045+0000","last_change":"2024-05-11T13:28:30.428782+0000","last_active":"2024-05-11T13:30:41.019045+0000","last_peered":"2024-05-11T13:30:41.019045+0000","last_clean":"2024-05-11T13:30:41.019045+0000","last_became_active":"2024-05-11T13:28:30.427819+0000","last_became_peered":"2024-05-11T13:28:30.427819+0000","last_unstale":"2024-05-11T13:30:41.019045+0000","last_undegraded":"2024-05-11T13:30:41.019045+0000","last_fullsized":"2024-05-11T13:30:41.019045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:23:24.533765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.545234+0000","last_change":"2024-05-11T13:28:35.210095+0000","last_active":"2024-05-11T13:31:39.545234+0000","last_peered":"2024-05-11T13:31:39.545234+0000","last_clean":"2024-05-11T13:31:39.545234+0000","last_became_active":"2024-05-11T13:28:35.208831+0000","last_became_peered":"2024-05-11T13:28:35.208831+0000","last_unstale":"2024-05-11T13:31:39.545234+0000","last_undegraded":"2024-05-11T13:31:39.545234+0000","last_fullsized":"2024-05-11T13:31:39.545234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:06.957821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.939164+0000","last_change":"2024-05-11T13:28:49.724791+0000","last_active":"2024-05-11T13:30:41.939164+0000","last_peered":"2024-05-11T13:30:41.939164+0000","last_clean":"2024-05-11T13:30:41.939164+0000","last_became_active":"2024-05-11T13:28:49.723855+0000","last_became_peered":"2024-05-11T13:28:49.723855+0000","last_unstale":"2024-05-11T13:30:41.939164+0000","last_undegraded":"2024-05-11T13:30:41.939164+0000","last_fullsized":"2024-05-11T13:30:41.939164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:34.705331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.921294+0000","last_change":"2024-05-11T13:28:30.431206+0000","last_active":"2024-05-11T13:30:41.921294+0000","last_peered":"2024-05-11T13:30:41.921294+0000","last_clean":"2024-05-11T13:30:41.921294+0000","last_became_active":"2024-05-11T13:28:30.430714+0000","last_became_peered":"2024-05-11T13:28:30.430714+0000","last_unstale":"2024-05-11T13:30:41.921294+0000","last_undegraded":"2024-05-11T13:30:41.921294+0000","last_fullsized":"2024-05-11T13:30:41.921294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:29:21.070046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.543830+0000","last_change":"2024-05-11T13:28:35.222254+0000","last_active":"2024-05-11T13:31:39.543830+0000","last_peered":"2024-05-11T13:31:39.543830+0000","last_clean":"2024-05-11T13:31:39.543830+0000","last_became_active":"2024-05-11T13:28:35.219539+0000","last_became_peered":"2024-05-11T13:28:35.219539+0000","last_unstale":"2024-05-11T13:31:39.543830+0000","last_undegraded":"2024-05-11T13:31:39.543830+0000","last_fullsized":"2024-05-11T13:31:39.543830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:55.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:42.067427+0000","last_change":"2024-05-11T13:28:49.692168+0000","last_active":"2024-05-11T13:30:42.067427+0000","last_peered":"2024-05-11T13:30:42.067427+0000","last_clean":"2024-05-11T13:30:42.067427+0000","last_became_active":"2024-05-11T13:28:49.691730+0000","last_became_peered":"2024-05-11T13:28:49.691730+0000","last_unstale":"2024-05-11T13:30:42.067427+0000","last_undegraded":"2024-05-11T13:30:42.067427+0000","last_fullsized":"2024-05-11T13:30:42.067427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:02.935554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":171,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:57.143998+0000","last_change":"2024-05-11T13:28:30.712115+0000","last_active":"2024-05-11T13:31:57.143998+0000","last_peered":"2024-05-11T13:31:57.143998+0000","last_clean":"2024-05-11T13:31:57.143998+0000","last_became_active":"2024-05-11T13:28:30.710613+0000","last_became_peered":"2024-05-11T13:28:30.710613+0000","last_unstale":"2024-05-11T13:31:57.143998+0000","last_undegraded":"2024-05-11T13:31:57.143998+0000","last_fullsized":"2024-05-11T13:31:57.143998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:00.521915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.546340+0000","last_change":"2024-05-11T13:28:35.221340+0000","last_active":"2024-05-11T13:31:39.546340+0000","last_peered":"2024-05-11T13:31:39.546340+0000","last_clean":"2024-05-11T13:31:39.546340+0000","last_became_active":"2024-05-11T13:28:35.220702+0000","last_became_peered":"2024-05-11T13:28:35.220702+0000","last_unstale":"2024-05-11T13:31:39.546340+0000","last_undegraded":"2024-05-11T13:31:39.546340+0000","last_fullsized":"2024-05-11T13:31:39.546340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:59:37.313209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.827356+0000","last_change":"2024-05-11T13:28:49.682917+0000","last_active":"2024-05-11T13:30:41.827356+0000","last_peered":"2024-05-11T13:30:41.827356+0000","last_clean":"2024-05-11T13:30:41.827356+0000","last_became_active":"2024-05-11T13:28:49.681905+0000","last_became_peered":"2024-05-11T13:28:49.681905+0000","last_unstale":"2024-05-11T13:30:41.827356+0000","last_undegraded":"2024-05-11T13:30:41.827356+0000","last_fullsized":"2024-05-11T13:30:41.827356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:09:26.191565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.016879+0000","last_change":"2024-05-11T13:28:30.437356+0000","last_active":"2024-05-11T13:30:41.016879+0000","last_peered":"2024-05-11T13:30:41.016879+0000","last_clean":"2024-05-11T13:30:41.016879+0000","last_became_active":"2024-05-11T13:28:30.437013+0000","last_became_peered":"2024-05-11T13:28:30.437013+0000","last_unstale":"2024-05-11T13:30:41.016879+0000","last_undegraded":"2024-05-11T13:30:41.016879+0000","last_fullsized":"2024-05-11T13:30:41.016879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:58.756156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.539209+0000","last_change":"2024-05-11T13:28:35.206105+0000","last_active":"2024-05-11T13:31:39.539209+0000","last_peered":"2024-05-11T13:31:39.539209+0000","last_clean":"2024-05-11T13:31:39.539209+0000","last_became_active":"2024-05-11T13:28:35.205684+0000","last_became_peered":"2024-05-11T13:28:35.205684+0000","last_unstale":"2024-05-11T13:31:39.539209+0000","last_undegraded":"2024-05-11T13:31:39.539209+0000","last_fullsized":"2024-05-11T13:31:39.539209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:12.326574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.015157+0000","last_change":"2024-05-11T13:28:49.689545+0000","last_active":"2024-05-11T13:30:41.015157+0000","last_peered":"2024-05-11T13:30:41.015157+0000","last_clean":"2024-05-11T13:30:41.015157+0000","last_became_active":"2024-05-11T13:28:49.688528+0000","last_became_peered":"2024-05-11T13:28:49.688528+0000","last_unstale":"2024-05-11T13:30:41.015157+0000","last_undegraded":"2024-05-11T13:30:41.015157+0000","last_fullsized":"2024-05-11T13:30:41.015157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:35.418873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:30:41.017223+0000","last_change":"2024-05-11T13:29:40.560835+0000","last_active":"2024-05-11T13:30:41.017223+0000","last_peered":"2024-05-11T13:30:41.017223+0000","last_clean":"2024-05-11T13:30:41.017223+0000","last_became_active":"2024-05-11T13:29:40.554310+0000","last_became_peered":"2024-05-11T13:29:40.554310+0000","last_unstale":"2024-05-11T13:30:41.017223+0000","last_undegraded":"2024-05-11T13:30:41.017223+0000","last_fullsized":"2024-05-11T13:30:41.017223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:19:26.805261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.539695+0000","last_change":"2024-05-11T13:28:35.220001+0000","last_active":"2024-05-11T13:31:39.539695+0000","last_peered":"2024-05-11T13:31:39.539695+0000","last_clean":"2024-05-11T13:31:39.539695+0000","last_became_active":"2024-05-11T13:28:35.216768+0000","last_became_peered":"2024-05-11T13:28:35.216768+0000","last_unstale":"2024-05-11T13:31:39.539695+0000","last_undegraded":"2024-05-11T13:31:39.539695+0000","last_fullsized":"2024-05-11T13:31:39.539695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:54:23.547689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.531701+0000","last_change":"2024-05-11T13:28:35.213481+0000","last_active":"2024-05-11T13:31:39.531701+0000","last_peered":"2024-05-11T13:31:39.531701+0000","last_clean":"2024-05-11T13:31:39.531701+0000","last_became_active":"2024-05-11T13:28:35.212956+0000","last_became_peered":"2024-05-11T13:28:35.212956+0000","last_unstale":"2024-05-11T13:31:39.531701+0000","last_undegraded":"2024-05-11T13:31:39.531701+0000","last_fullsized":"2024-05-11T13:31:39.531701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:14.814155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530898+0000","last_change":"2024-05-11T13:29:40.554654+0000","last_active":"2024-05-11T13:31:39.530898+0000","last_peered":"2024-05-11T13:31:39.530898+0000","last_clean":"2024-05-11T13:31:39.530898+0000","last_became_active":"2024-05-11T13:29:40.551836+0000","last_became_peered":"2024-05-11T13:29:40.551836+0000","last_unstale":"2024-05-11T13:31:39.530898+0000","last_undegraded":"2024-05-11T13:31:39.530898+0000","last_fullsized":"2024-05-11T13:31:39.530898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:49:44.566051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.548974+0000","last_change":"2024-05-11T13:28:35.208925+0000","last_active":"2024-05-11T13:31:39.548974+0000","last_peered":"2024-05-11T13:31:39.548974+0000","last_clean":"2024-05-11T13:31:39.548974+0000","last_became_active":"2024-05-11T13:28:35.208055+0000","last_became_peered":"2024-05-11T13:28:35.208055+0000","last_unstale":"2024-05-11T13:31:39.548974+0000","last_undegraded":"2024-05-11T13:31:39.548974+0000","last_fullsized":"2024-05-11T13:31:39.548974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:20.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549052+0000","last_change":"2024-05-11T13:28:35.213226+0000","last_active":"2024-05-11T13:31:39.549052+0000","last_peered":"2024-05-11T13:31:39.549052+0000","last_clean":"2024-05-11T13:31:39.549052+0000","last_became_active":"2024-05-11T13:28:35.212839+0000","last_became_peered":"2024-05-11T13:28:35.212839+0000","last_unstale":"2024-05-11T13:31:39.549052+0000","last_undegraded":"2024-05-11T13:31:39.549052+0000","last_fullsized":"2024-05-11T13:31:39.549052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:37:05.312505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549428+0000","last_change":"2024-05-11T13:29:40.848092+0000","last_active":"2024-05-11T13:31:39.549428+0000","last_peered":"2024-05-11T13:31:39.549428+0000","last_clean":"2024-05-11T13:31:39.549428+0000","last_became_active":"2024-05-11T13:29:40.846950+0000","last_became_peered":"2024-05-11T13:29:40.846950+0000","last_unstale":"2024-05-11T13:31:39.549428+0000","last_undegraded":"2024-05-11T13:31:39.549428+0000","last_fullsized":"2024-05-11T13:31:39.549428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:18:06.675351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.530970+0000","last_change":"2024-05-11T13:28:35.217529+0000","last_active":"2024-05-11T13:31:39.530970+0000","last_peered":"2024-05-11T13:31:39.530970+0000","last_clean":"2024-05-11T13:31:39.530970+0000","last_became_active":"2024-05-11T13:28:35.217062+0000","last_became_peered":"2024-05-11T13:28:35.217062+0000","last_unstale":"2024-05-11T13:31:39.530970+0000","last_undegraded":"2024-05-11T13:31:39.530970+0000","last_fullsized":"2024-05-11T13:31:39.530970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:24:33.955794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.550463+0000","last_change":"2024-05-11T13:28:35.217777+0000","last_active":"2024-05-11T13:31:39.550463+0000","last_peered":"2024-05-11T13:31:39.550463+0000","last_clean":"2024-05-11T13:31:39.550463+0000","last_became_active":"2024-05-11T13:28:35.217381+0000","last_became_peered":"2024-05-11T13:28:35.217381+0000","last_unstale":"2024-05-11T13:31:39.550463+0000","last_undegraded":"2024-05-11T13:31:39.550463+0000","last_fullsized":"2024-05-11T13:31:39.550463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:43.973192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.524878+0000","last_change":"2024-05-11T13:28:35.209826+0000","last_active":"2024-05-11T13:31:39.524878+0000","last_peered":"2024-05-11T13:31:39.524878+0000","last_clean":"2024-05-11T13:31:39.524878+0000","last_became_active":"2024-05-11T13:28:35.209461+0000","last_became_peered":"2024-05-11T13:28:35.209461+0000","last_unstale":"2024-05-11T13:31:39.524878+0000","last_undegraded":"2024-05-11T13:31:39.524878+0000","last_fullsized":"2024-05-11T13:31:39.524878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:03.323045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535561+0000","last_change":"2024-05-11T13:28:35.205252+0000","last_active":"2024-05-11T13:31:39.535561+0000","last_peered":"2024-05-11T13:31:39.535561+0000","last_clean":"2024-05-11T13:31:39.535561+0000","last_became_active":"2024-05-11T13:28:35.204835+0000","last_became_peered":"2024-05-11T13:28:35.204835+0000","last_unstale":"2024-05-11T13:31:39.535561+0000","last_undegraded":"2024-05-11T13:31:39.535561+0000","last_fullsized":"2024-05-11T13:31:39.535561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:10.032439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.535006+0000","last_change":"2024-05-11T13:28:35.219718+0000","last_active":"2024-05-11T13:31:39.535006+0000","last_peered":"2024-05-11T13:31:39.535006+0000","last_clean":"2024-05-11T13:31:39.535006+0000","last_became_active":"2024-05-11T13:28:35.219032+0000","last_became_peered":"2024-05-11T13:28:35.219032+0000","last_unstale":"2024-05-11T13:31:39.535006+0000","last_undegraded":"2024-05-11T13:31:39.535006+0000","last_fullsized":"2024-05-11T13:31:39.535006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:03:58.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.547789+0000","last_change":"2024-05-11T13:28:35.217820+0000","last_active":"2024-05-11T13:31:39.547789+0000","last_peered":"2024-05-11T13:31:39.547789+0000","last_clean":"2024-05-11T13:31:39.547789+0000","last_became_active":"2024-05-11T13:28:35.216260+0000","last_became_peered":"2024-05-11T13:28:35.216260+0000","last_unstale":"2024-05-11T13:31:39.547789+0000","last_undegraded":"2024-05-11T13:31:39.547789+0000","last_fullsized":"2024-05-11T13:31:39.547789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:20:47.188238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.549407+0000","last_change":"2024-05-11T13:28:35.209762+0000","last_active":"2024-05-11T13:31:39.549407+0000","last_peered":"2024-05-11T13:31:39.549407+0000","last_clean":"2024-05-11T13:31:39.549407+0000","last_became_active":"2024-05-11T13:28:35.207404+0000","last_became_peered":"2024-05-11T13:28:35.207404+0000","last_unstale":"2024-05-11T13:31:39.549407+0000","last_undegraded":"2024-05-11T13:31:39.549407+0000","last_fullsized":"2024-05-11T13:31:39.549407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:41:20.808009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-11T13:31:39.531809+0000","last_change":"2024-05-11T13:28:35.215137+0000","last_active":"2024-05-11T13:31:39.531809+0000","last_peered":"2024-05-11T13:31:39.531809+0000","last_clean":"2024-05-11T13:31:39.531809+0000","last_became_active":"2024-05-11T13:28:35.214745+0000","last_became_peered":"2024-05-11T13:28:35.214745+0000","last_unstale":"2024-05-11T13:31:39.531809+0000","last_undegraded":"2024-05-11T13:31:39.531809+0000","last_fullsized":"2024-05-11T13:31:39.531809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:44.381501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70392,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1253,"num_read_kb":708,"num_write":257,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678076,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":519366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874307,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32872,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712280,"statfs":{"total":95995035648,"available":95961374720,"internally_reserved":0,"allocated":1736704,"data_stored":1169016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":522788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234066,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":518466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430297,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1116240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593824,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":514728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":578664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738505,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":515652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":577730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149814,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":513904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":519424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790055,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1110758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T13:32:03.055 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-11T13:32:03.055 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-11T13:32:03.055 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-11T13:32:03.055 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-11T13:32:03.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:02 smithi165 ceph-mon[33235]: pgmap v481: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:03.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:02 smithi063 ceph-mon[29809]: pgmap v481: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:03.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:02 smithi071 ceph-mon[33293]: pgmap v481: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:03.570 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:03.983 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:03 smithi063 ceph-mon[29809]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:32:04.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:03 smithi165 ceph-mon[33235]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:32:04.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:03 smithi071 ceph-mon[33293]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:32:05.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:04 smithi063 ceph-mon[29809]: pgmap v482: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:04 smithi165 ceph-mon[33235]: pgmap v482: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:05.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:04 smithi071 ceph-mon[33293]: pgmap v482: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 -- 172.21.15.63:0/1484089987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 msgr2=0x7f8ff8105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1484089987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff8105bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009990 tx=0x7f8fe4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 -- 172.21.15.63:0/1484089987 shutdown_connections 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1484089987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff8106130 0x7f8ff8112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1484089987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ff810b770 0x7f8ff810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1484089987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff8105bf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 -- 172.21.15.63:0/1484089987 >> 172.21.15.63:0/1484089987 conn(0x7f8ff8101260 msgr2=0x7f8ff8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.789+0000 7f8fffe00700 1 -- 172.21.15.63:0/1484089987 shutdown_connections 2024-05-11T13:32:05.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.790+0000 7f8fffe00700 1 -- 172.21.15.63:0/1484089987 wait complete. 2024-05-11T13:32:05.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 Processor -- start 2024-05-11T13:32:05.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 -- start start 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ff8106130 0x7f8ff819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff810b770 0x7f8ff81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81158b0 con 0x7f8ff8105770 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8115730 con 0x7f8ff810b770 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.791+0000 7f8fffe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8115a30 con 0x7f8ff8106130 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53498/0 (socket says 172.21.15.63:53498) 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 -- 172.21.15.63:0/1083019861 learned_addr learned my addr 172.21.15.63:0/1083019861 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffca00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ff8106130 0x7f8ff819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:05.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffde00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff810b770 0x7f8ff81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 -- 172.21.15.63:0/1083019861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ff8106130 msgr2=0x7f8ff819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ff8106130 0x7f8ff819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 -- 172.21.15.63:0/1083019861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff810b770 msgr2=0x7f8ff81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff810b770 0x7f8ff81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffd400700 1 -- 172.21.15.63:0/1083019861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff81a0d90 con 0x7f8ff8105770 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.792+0000 7f8ffca00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ff8106130 0x7f8ff819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.793+0000 7f8ffd400700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff819bd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00d950 tx=0x7f8fec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.793+0000 7f8ff6000700 1 -- 172.21.15.63:0/1083019861 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec016070 con 0x7f8ff8105770 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.793+0000 7f8ff6000700 1 -- 172.21.15.63:0/1083019861 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fec010460 con 0x7f8ff8105770 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.793+0000 7f8ff6000700 1 -- 172.21.15.63:0/1083019861 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00f620 con 0x7f8ff8105770 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.793+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff81a1080 con 0x7f8ff8105770 2024-05-11T13:32:05.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.793+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff81a8900 con 0x7f8ff8105770 2024-05-11T13:32:05.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.794+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff806a240 con 0x7f8ff8105770 2024-05-11T13:32:05.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.797+0000 7f8ff6000700 1 -- 172.21.15.63:0/1083019861 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8fec019070 con 0x7f8ff8105770 2024-05-11T13:32:05.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.798+0000 7f8ff6000700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8fe00b2880 0x7f8fe00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.799+0000 7f8ff6000700 1 -- 172.21.15.63:0/1083019861 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8fec0e21e0 con 0x7f8ff8105770 2024-05-11T13:32:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.799+0000 7f8ffca00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8fe00b2880 0x7f8fe00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:05.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.800+0000 7f8ffca00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8fe00b2880 0x7f8fe00b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4006010 tx=0x7f8fe401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:05.803+0000 7f8ff6000700 1 -- 172.21.15.63:0/1083019861 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fec0a9af0 con 0x7f8ff8105770 2024-05-11T13:32:06.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.091+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8ff802e570 con 0x7f8ff8105770 2024-05-11T13:32:06.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.092+0000 7f8ff6000700 1 -- 172.21.15.63:0/1083019861 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8fec0a93f0 con 0x7f8ff8105770 2024-05-11T13:32:06.096 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:32:06.096 INFO:teuthology.orchestra.run.smithi063.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-11T13:32:06.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.095+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8fe00b2880 msgr2=0x7f8fe00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:06.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.095+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8fe00b2880 0x7f8fe00b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4006010 tx=0x7f8fe401a040 comp rx=0 tx=0).stop 2024-05-11T13:32:06.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 msgr2=0x7f8ff819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff819bd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00d950 tx=0x7f8fec00dd10 comp rx=0 tx=0).stop 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 shutdown_connections 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f8fe00b2880 0x7f8fe00b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff810b770 0x7f8ff81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ff8105770 0x7f8ff819bd80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 --2- 172.21.15.63:0/1083019861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ff8106130 0x7f8ff819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 >> 172.21.15.63:0/1083019861 conn(0x7f8ff8101260 msgr2=0x7f8ff8101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 shutdown_connections 2024-05-11T13:32:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:06.096+0000 7f8fffe00700 1 -- 172.21.15.63:0/1083019861 wait complete. 2024-05-11T13:32:06.867 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-11T13:32:06.868 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:32:07.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:06 smithi063 ceph-mon[29809]: pgmap v483: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:07.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:06 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1083019861' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-11T13:32:07.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:06 smithi165 ceph-mon[33235]: pgmap v483: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:07.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:06 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1083019861' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-11T13:32:07.336 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:07.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:06 smithi071 ceph-mon[33293]: pgmap v483: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:07.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:06 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1083019861' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-11T13:32:09.268 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:08 smithi063 ceph-mon[29809]: pgmap v484: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:08 smithi165 ceph-mon[33235]: pgmap v484: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:09.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:08 smithi071 ceph-mon[33293]: pgmap v484: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:09.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.597+0000 7f88c1400700 1 -- 172.21.15.63:0/767530524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc06bb80 msgr2=0x7f88bc10fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:09.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.597+0000 7f88c1400700 1 --2- 172.21.15.63:0/767530524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc06bb80 0x7f88bc10fac0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009990 tx=0x7f88a4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.598+0000 7f88c1400700 1 -- 172.21.15.63:0/767530524 shutdown_connections 2024-05-11T13:32:09.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.598+0000 7f88c1400700 1 --2- 172.21.15.63:0/767530524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc110000 0x7f88bc1123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.598+0000 7f88c1400700 1 --2- 172.21.15.63:0/767530524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88bc06b260 0x7f88bc06b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.598+0000 7f88c1400700 1 --2- 172.21.15.63:0/767530524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc06bb80 0x7f88bc10fac0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.598+0000 7f88c1400700 1 -- 172.21.15.63:0/767530524 >> 172.21.15.63:0/767530524 conn(0x7f88bc079400 msgr2=0x7f88bc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:09.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.598+0000 7f88c1400700 1 -- 172.21.15.63:0/767530524 shutdown_connections 2024-05-11T13:32:09.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.598+0000 7f88c1400700 1 -- 172.21.15.63:0/767530524 wait complete. 2024-05-11T13:32:09.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.599+0000 7f88c1400700 1 Processor -- start 2024-05-11T13:32:09.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.599+0000 7f88c1400700 1 -- start start 2024-05-11T13:32:09.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88c1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88bc06b260 0x7f88bc19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88c1400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc06bb80 0x7f88bc19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88c1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc110000 0x7f88bc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88c1400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc115840 con 0x7f88bc110000 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88c1400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1156c0 con 0x7f88bc06b260 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88c1400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1159c0 con 0x7f88bc06bb80 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88bb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc110000 0x7f88bc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88bb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc110000 0x7f88bc1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53512/0 (socket says 172.21.15.63:53512) 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88bb200700 1 -- 172.21.15.63:0/2369782725 learned_addr learned my addr 172.21.15.63:0/2369782725 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88ba800700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88bc06b260 0x7f88bc19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88b3e00700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc06bb80 0x7f88bc19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.600+0000 7f88bb200700 1 -- 172.21.15.63:0/2369782725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc06bb80 msgr2=0x7f88bc19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88bb200700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc06bb80 0x7f88bc19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88bb200700 1 -- 172.21.15.63:0/2369782725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88bc06b260 msgr2=0x7f88bc19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:09.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88bb200700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88bc06b260 0x7f88bc19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88bb200700 1 -- 172.21.15.63:0/2369782725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc1a0da0 con 0x7f88bc110000 2024-05-11T13:32:09.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88ba800700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88bc06b260 0x7f88bc19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:09.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88bb200700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc110000 0x7f88bc1a0660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f88b40094f0 tx=0x7f88b4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:09.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88b3400700 1 -- 172.21.15.63:0/2369782725 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004db0 con 0x7f88bc110000 2024-05-11T13:32:09.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88b3400700 1 -- 172.21.15.63:0/2369782725 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88b4005020 con 0x7f88bc110000 2024-05-11T13:32:09.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc1a1090 con 0x7f88bc110000 2024-05-11T13:32:09.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.601+0000 7f88b3400700 1 -- 172.21.15.63:0/2369782725 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4015660 con 0x7f88bc110000 2024-05-11T13:32:09.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.602+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc1a8910 con 0x7f88bc110000 2024-05-11T13:32:09.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.603+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc06a240 con 0x7f88bc110000 2024-05-11T13:32:09.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.607+0000 7f88b3400700 1 -- 172.21.15.63:0/2369782725 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f88b40051d0 con 0x7f88bc110000 2024-05-11T13:32:09.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.608+0000 7f88b3400700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f88a00b2880 0x7f88a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:09.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.609+0000 7f88b3400700 1 -- 172.21.15.63:0/2369782725 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f88b40e2870 con 0x7f88bc110000 2024-05-11T13:32:09.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.609+0000 7f88ba800700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f88a00b2880 0x7f88a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:09.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.609+0000 7f88ba800700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f88a00b2880 0x7f88a00b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88ac005950 tx=0x7f88ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:09.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.611+0000 7f88b3400700 1 -- 172.21.15.63:0/2369782725 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88b4010070 con 0x7f88bc110000 2024-05-11T13:32:09.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.908+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88bc02e570 con 0x7f88bc110000 2024-05-11T13:32:09.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.910+0000 7f88b3400700 1 -- 172.21.15.63:0/2369782725 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f88b40b2030 con 0x7f88bc110000 2024-05-11T13:32:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:32:09.911 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:30:02.730662+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:32:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f88a00b2880 msgr2=0x7f88a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f88a00b2880 0x7f88a00b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88ac005950 tx=0x7f88ac0058e0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc110000 msgr2=0x7f88bc1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc110000 0x7f88bc1a0660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f88b40094f0 tx=0x7f88b4007f90 comp rx=0 tx=0).stop 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 shutdown_connections 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f88a00b2880 0x7f88a00b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88bc110000 0x7f88bc1a0660 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88bc06b260 0x7f88bc19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 --2- 172.21.15.63:0/2369782725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc06bb80 0x7f88bc19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 >> 172.21.15.63:0/2369782725 conn(0x7f88bc079400 msgr2=0x7f88bc1115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 shutdown_connections 2024-05-11T13:32:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:09.913+0000 7f88c1400700 1 -- 172.21.15.63:0/2369782725 wait complete. 2024-05-11T13:32:09.916 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-11T13:32:10.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:09 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2369782725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:10.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:09 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2369782725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:10.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:09 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2369782725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:10.693 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:32:11.158 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:11.312 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:10 smithi063 ceph-mon[29809]: pgmap v485: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:11 smithi165 ceph-mon[33235]: pgmap v485: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:11.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:11 smithi071 ceph-mon[33293]: pgmap v485: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:13.229 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:13 smithi063 ceph-mon[29809]: pgmap v486: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:13.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:13 smithi165 ceph-mon[33235]: pgmap v486: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:13.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:13 smithi071 ceph-mon[33293]: pgmap v486: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:13.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 -- 172.21.15.63:0/1669488531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e410b740 msgr2=0x7f51e410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 --2- 172.21.15.63:0/1669488531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e410b740 0x7f51e410bb20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009990 tx=0x7f51d4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 -- 172.21.15.63:0/1669488531 shutdown_connections 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 --2- 172.21.15.63:0/1669488531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51e4106100 0x7f51e4112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 --2- 172.21.15.63:0/1669488531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e410b740 0x7f51e410bb20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 --2- 172.21.15.63:0/1669488531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51e4105740 0x7f51e4105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 -- 172.21.15.63:0/1669488531 >> 172.21.15.63:0/1669488531 conn(0x7f51e4101230 msgr2=0x7f51e4103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 -- 172.21.15.63:0/1669488531 shutdown_connections 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.550+0000 7f51e9400700 1 -- 172.21.15.63:0/1669488531 wait complete. 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.551+0000 7f51e9400700 1 Processor -- start 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.551+0000 7f51e9400700 1 -- start start 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e9400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51e4105740 0x7f51e419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e4106100 0x7f51e419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:13.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51e410b740 0x7f51e41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e9400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4115840 con 0x7f51e4106100 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e9400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41156c0 con 0x7f51e410b740 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e9400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41159c0 con 0x7f51e4105740 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51d9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e4106100 0x7f51e419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51d9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e4106100 0x7f51e419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41898/0 (socket says 172.21.15.63:41898) 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51d9e00700 1 -- 172.21.15.63:0/3841189424 learned_addr learned my addr 172.21.15.63:0/3841189424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e3200700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51e410b740 0x7f51e41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.552+0000 7f51e2800700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51e4105740 0x7f51e419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51d9e00700 1 -- 172.21.15.63:0/3841189424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51e4105740 msgr2=0x7f51e419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51d9e00700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51e4105740 0x7f51e419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51d9e00700 1 -- 172.21.15.63:0/3841189424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51e410b740 msgr2=0x7f51e41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51d9e00700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51e410b740 0x7f51e41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51d9e00700 1 -- 172.21.15.63:0/3841189424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e41a0db0 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51d9e00700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e4106100 0x7f51e419c2e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c970 tx=0x7f51cc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51dbe00700 1 -- 172.21.15.63:0/3841189424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc007aa0 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51dbe00700 1 -- 172.21.15.63:0/3841189424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51cc004cc0 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e41a10a0 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51dbe00700 1 -- 172.21.15.63:0/3841189424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc005500 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.553+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e41a8920 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.555+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e406a240 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.559+0000 7f51dbe00700 1 -- 172.21.15.63:0/3841189424 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f51cc007c40 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.560+0000 7f51dbe00700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f51c80b2880 0x7f51c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.560+0000 7f51dbe00700 1 -- 172.21.15.63:0/3841189424 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f51cc0e2730 con 0x7f51e4106100 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.561+0000 7f51e2800700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f51c80b2880 0x7f51c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.561+0000 7f51e2800700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f51c80b2880 0x7f51c80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b5c0 tx=0x7f51d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:13.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.563+0000 7f51dbe00700 1 -- 172.21.15.63:0/3841189424 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51cc0aa160 con 0x7f51e4106100 2024-05-11T13:32:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.857+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51e402e570 con 0x7f51e4106100 2024-05-11T13:32:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.859+0000 7f51dbe00700 1 -- 172.21.15.63:0/3841189424 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f51cc0bee20 con 0x7f51e4106100 2024-05-11T13:32:13.860 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:32:13.860 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:30:02.730662+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:32:13.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.863+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f51c80b2880 msgr2=0x7f51c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:13.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.863+0000 7f51e9400700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f51c80b2880 0x7f51c80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b5c0 tx=0x7f51d401a040 comp rx=0 tx=0).stop 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.863+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e4106100 msgr2=0x7f51e419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.863+0000 7f51e9400700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e4106100 0x7f51e419c2e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c970 tx=0x7f51cc00cd30 comp rx=0 tx=0).stop 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.863+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 shutdown_connections 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.863+0000 7f51e9400700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f51c80b2880 0x7f51c80b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.863+0000 7f51e9400700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51e410b740 0x7f51e41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.864+0000 7f51e9400700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51e4105740 0x7f51e419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.864+0000 7f51e9400700 1 --2- 172.21.15.63:0/3841189424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51e4106100 0x7f51e419c2e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:13.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.864+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 >> 172.21.15.63:0/3841189424 conn(0x7f51e4101230 msgr2=0x7f51e4101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:13.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.864+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 shutdown_connections 2024-05-11T13:32:13.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:13.864+0000 7f51e9400700 1 -- 172.21.15.63:0/3841189424 wait complete. 2024-05-11T13:32:13.865 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-11T13:32:14.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:14 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3841189424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:14.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:14 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3841189424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:14.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:14 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3841189424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:14.771 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:32:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:15.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:15 smithi165 ceph-mon[33235]: pgmap v487: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:15.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:15 smithi063 ceph-mon[29809]: pgmap v487: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:15.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:15 smithi071 ceph-mon[33293]: pgmap v487: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:17.287 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:17 smithi063 ceph-mon[29809]: pgmap v488: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:17.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:17 smithi165 ceph-mon[33235]: pgmap v488: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:17.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:17 smithi071 ceph-mon[33293]: pgmap v488: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 -- 172.21.15.63:0/3859049579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28109a90 msgr2=0x7fdb28109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 --2- 172.21.15.63:0/3859049579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28109a90 0x7fdb28109f10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009990 tx=0x7fdb14009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 -- 172.21.15.63:0/3859049579 shutdown_connections 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 --2- 172.21.15.63:0/3859049579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb2810a450 0x7fdb281168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 --2- 172.21.15.63:0/3859049579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2810e960 0x7fdb2810ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 --2- 172.21.15.63:0/3859049579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28109a90 0x7fdb28109f10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 -- 172.21.15.63:0/3859049579 >> 172.21.15.63:0/3859049579 conn(0x7fdb280792e0 msgr2=0x7fdb28108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 -- 172.21.15.63:0/3859049579 shutdown_connections 2024-05-11T13:32:17.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.600+0000 7fdb27e00700 1 -- 172.21.15.63:0/3859049579 wait complete. 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.601+0000 7fdb27e00700 1 Processor -- start 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.601+0000 7fdb27e00700 1 -- start start 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb27e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb28109a90 0x7fdb281a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb27e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2810a450 0x7fdb281a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb27e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2810e960 0x7fdb281a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb27e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28119b40 con 0x7fdb2810a450 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb27e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281199c0 con 0x7fdb28109a90 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb27e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28119cc0 con 0x7fdb2810e960 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb26000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2810a450 0x7fdb281a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb26000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2810a450 0x7fdb281a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41922/0 (socket says 172.21.15.63:41922) 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb26000700 1 -- 172.21.15.63:0/467432192 learned_addr learned my addr 172.21.15.63:0/467432192 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb26a00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb28109a90 0x7fdb281a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:17.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb27400700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2810e960 0x7fdb281a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:17.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb26000700 1 -- 172.21.15.63:0/467432192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2810e960 msgr2=0x7fdb281a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:17.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb26000700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2810e960 0x7fdb281a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.602+0000 7fdb26000700 1 -- 172.21.15.63:0/467432192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb28109a90 msgr2=0x7fdb281a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:17.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb26000700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb28109a90 0x7fdb281a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb26000700 1 -- 172.21.15.63:0/467432192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb281a50c0 con 0x7fdb2810a450 2024-05-11T13:32:17.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb27400700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2810e960 0x7fdb281a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:17.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb26000700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2810a450 0x7fdb281a05f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdb14000c00 tx=0x7fdb14005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:17.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb2ca00700 1 -- 172.21.15.63:0/467432192 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14027070 con 0x7fdb2810a450 2024-05-11T13:32:17.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb2ca00700 1 -- 172.21.15.63:0/467432192 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb1400fb80 con 0x7fdb2810a450 2024-05-11T13:32:17.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb2ca00700 1 -- 172.21.15.63:0/467432192 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14022c30 con 0x7fdb2810a450 2024-05-11T13:32:17.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.603+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb281a5350 con 0x7fdb2810a450 2024-05-11T13:32:17.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.604+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb281acc30 con 0x7fdb2810a450 2024-05-11T13:32:17.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.605+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2806a240 con 0x7fdb2810a450 2024-05-11T13:32:17.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.607+0000 7fdb2ca00700 1 -- 172.21.15.63:0/467432192 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdb1402a070 con 0x7fdb2810a450 2024-05-11T13:32:17.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.609+0000 7fdb2ca00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdb0c0b2900 0x7fdb0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:17.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.609+0000 7fdb2ca00700 1 -- 172.21.15.63:0/467432192 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdb140e36d0 con 0x7fdb2810a450 2024-05-11T13:32:17.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.609+0000 7fdb26a00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdb0c0b2900 0x7fdb0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:17.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.610+0000 7fdb26a00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdb0c0b2900 0x7fdb0c0b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005950 tx=0x7fdb180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:17.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.611+0000 7fdb2ca00700 1 -- 172.21.15.63:0/467432192 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb14031080 con 0x7fdb2810a450 2024-05-11T13:32:17.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.856+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb2802e570 con 0x7fdb2810a450 2024-05-11T13:32:17.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.859+0000 7fdb2ca00700 1 -- 172.21.15.63:0/467432192 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24760 (secure 0 0 0) 0x7fdb14013070 con 0x7fdb2810a450 2024-05-11T13:32:17.860 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:32:17.860 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":90,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:30:41.007052+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:32:17.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.862+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdb0c0b2900 msgr2=0x7fdb0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:17.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdb0c0b2900 0x7fdb0c0b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005950 tx=0x7fdb180058e0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2810a450 msgr2=0x7fdb281a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:17.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2810a450 0x7fdb281a05f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdb14000c00 tx=0x7fdb14005ef0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 shutdown_connections 2024-05-11T13:32:17.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fdb0c0b2900 0x7fdb0c0b4dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2810e960 0x7fdb281a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb28109a90 0x7fdb281a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 --2- 172.21.15.63:0/467432192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2810a450 0x7fdb281a05f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:17.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 >> 172.21.15.63:0/467432192 conn(0x7fdb280792e0 msgr2=0x7fdb280799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:17.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 shutdown_connections 2024-05-11T13:32:17.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:17.863+0000 7fdb27e00700 1 -- 172.21.15.63:0/467432192 wait complete. 2024-05-11T13:32:18.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:18 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/467432192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:32:18.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:18 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/467432192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:32:18.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:18 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/467432192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:32:18.824 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:32:19.303 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:19.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:19 smithi165 ceph-mon[33235]: pgmap v489: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:19.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:19 smithi063 ceph-mon[29809]: pgmap v489: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:19.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:19 smithi071 ceph-mon[33293]: pgmap v489: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:21.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:21 smithi165 ceph-mon[33235]: pgmap v490: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:21.367 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:21 smithi063 ceph-mon[29809]: pgmap v490: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:21.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:21 smithi071 ceph-mon[33293]: pgmap v490: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 -- 172.21.15.63:0/3271987320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 msgr2=0x7f2c2810ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/3271987320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2810ba60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800b0a0 tx=0x7f2c1800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 -- 172.21.15.63:0/3271987320 shutdown_connections 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/3271987320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c28106040 0x7f2c28112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/3271987320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2810ba60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/3271987320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c28105680 0x7f2c28105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 -- 172.21.15.63:0/3271987320 >> 172.21.15.63:0/3271987320 conn(0x7f2c28101170 msgr2=0x7f2c28103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 -- 172.21.15.63:0/3271987320 shutdown_connections 2024-05-11T13:32:21.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.722+0000 7f2c2ce00700 1 -- 172.21.15.63:0/3271987320 wait complete. 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.723+0000 7f2c2ce00700 1 Processor -- start 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c2ce00700 1 -- start start 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c2ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c28105680 0x7f2c281a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c2ce00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c28106040 0x7f2c281a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c2ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c2ce00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28115780 con 0x7f2c2810b680 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c2ce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28115600 con 0x7f2c28105680 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c2ce00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28115900 con 0x7f2c28106040 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c27e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c27e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42636/0 (socket says 172.21.15.63:42636) 2024-05-11T13:32:21.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.724+0000 7f2c27e00700 1 -- 172.21.15.63:0/266807660 learned_addr learned my addr 172.21.15.63:0/266807660 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:21.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c26a00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c28106040 0x7f2c281a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:21.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c27400700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c28105680 0x7f2c281a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:21.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c27e00700 1 -- 172.21.15.63:0/266807660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c28106040 msgr2=0x7f2c281a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c27e00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c28106040 0x7f2c281a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c27e00700 1 -- 172.21.15.63:0/266807660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c28105680 msgr2=0x7f2c281a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c27e00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c28105680 0x7f2c281a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c27e00700 1 -- 172.21.15.63:0/266807660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2819c1c0 con 0x7f2c2810b680 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.725+0000 7f2c27e00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2819ba60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c900 tx=0x7f2c2000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.726+0000 7f2c1fe00700 1 -- 172.21.15.63:0/266807660 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20007b00 con 0x7f2c2810b680 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.726+0000 7f2c1fe00700 1 -- 172.21.15.63:0/266807660 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c20007ca0 con 0x7f2c2810b680 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.726+0000 7f2c1fe00700 1 -- 172.21.15.63:0/266807660 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20018660 con 0x7f2c2810b680 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.726+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2819c4b0 con 0x7f2c2810b680 2024-05-11T13:32:21.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.726+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c281a8770 con 0x7f2c2810b680 2024-05-11T13:32:21.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.727+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2806a240 con 0x7f2c2810b680 2024-05-11T13:32:21.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.731+0000 7f2c1fe00700 1 -- 172.21.15.63:0/266807660 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2c2000f450 con 0x7f2c2810b680 2024-05-11T13:32:21.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.733+0000 7f2c1fe00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2c0c0b2900 0x7f2c0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:21.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.733+0000 7f2c1fe00700 1 -- 172.21.15.63:0/266807660 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c200e26b0 con 0x7f2c2810b680 2024-05-11T13:32:21.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.733+0000 7f2c27400700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2c0c0b2900 0x7f2c0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:21.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.734+0000 7f2c27400700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2c0c0b2900 0x7f2c0c0b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009e60 tx=0x7f2c1800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:21.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:21.736+0000 7f2c1fe00700 1 -- 172.21.15.63:0/266807660 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c20054df0 con 0x7f2c2810b680 2024-05-11T13:32:22.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.024+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c2819cf90 con 0x7f2c2810b680 2024-05-11T13:32:22.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.025+0000 7f2c1fe00700 1 -- 172.21.15.63:0/266807660 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f2c20014030 con 0x7f2c2810b680 2024-05-11T13:32:22.026 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:32:22.026 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:30:02.730662+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:32:22.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2c0c0b2900 msgr2=0x7f2c0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:22.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2c0c0b2900 0x7f2c0c0b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009e60 tx=0x7f2c1800b7b0 comp rx=0 tx=0).stop 2024-05-11T13:32:22.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 msgr2=0x7f2c2819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:22.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2819ba60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c900 tx=0x7f2c2000ccc0 comp rx=0 tx=0).stop 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 shutdown_connections 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2c0c0b2900 0x7f2c0c0b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c2810b680 0x7f2c2819ba60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c28105680 0x7f2c281a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 --2- 172.21.15.63:0/266807660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c28106040 0x7f2c281a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 >> 172.21.15.63:0/266807660 conn(0x7f2c28101170 msgr2=0x7f2c28101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 shutdown_connections 2024-05-11T13:32:22.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:22.029+0000 7f2c2ce00700 1 -- 172.21.15.63:0/266807660 wait complete. 2024-05-11T13:32:22.031 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-11T13:32:22.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:22 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/266807660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:22.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:22 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/266807660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:22.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:22 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/266807660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:32:22.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-11T13:28:38.143704+0000', 'modified': '2024-05-11T13:30:02.730662+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34472, 'mds_1': 34451, 'mds_2': 34427, 'mds_3': 34403, 'mds_4': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6839/3189418422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6838', 'nonce': 3189418422}, {'type': 'v1', 'addr': '172.21.15.63:6839', 'nonce': 3189418422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14577': {'gid': 14577, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6841/3763187067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6840', 'nonce': 3763187067}, {'type': 'v1', 'addr': '172.21.15.63:6841', 'nonce': 3763187067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/2703259087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 2703259087}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 2703259087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/4024086251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 4024086251}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 4024086251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2308228126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2308228126}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2308228126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.165:6839/1121013005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1121013005}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1121013005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34403': {'gid': 34403, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 37, 'addr': '172.21.15.71:6833/1870291573', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 1870291573}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 1870291573}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.71:6835/1316489545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 1316489545}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 1316489545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34451': {'gid': 34451, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.71:6837/136473158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 136473158}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 136473158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.71:6839/1525017608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1525017608}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1525017608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34472, 'qdb_cluster': [24472, 34403, 34427, 34451, 34472]} 2024-05-11T13:32:22.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-11T13:32:22.967 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-11T13:32:23.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:23 smithi063 ceph-mon[29809]: pgmap v491: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:23.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:23 smithi165 ceph-mon[33235]: pgmap v491: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:23.443 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:23.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:23 smithi071 ceph-mon[33293]: pgmap v491: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:25.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:25 smithi063 ceph-mon[29809]: pgmap v492: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:25.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:25 smithi071 ceph-mon[33293]: pgmap v492: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:25.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:25 smithi165 ceph-mon[33235]: pgmap v492: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.806+0000 7f184fa00700 1 -- 172.21.15.63:0/904443068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848105770 msgr2=0x7f1848105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.806+0000 7f184fa00700 1 --2- 172.21.15.63:0/904443068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848105770 0x7f1848105bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1834009990 tx=0x7f1834009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.807+0000 7f184fa00700 1 -- 172.21.15.63:0/904443068 shutdown_connections 2024-05-11T13:32:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.807+0000 7f184fa00700 1 --2- 172.21.15.63:0/904443068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1848106130 0x7f1848112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.807+0000 7f184fa00700 1 --2- 172.21.15.63:0/904443068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f184810b770 0x7f184810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.807+0000 7f184fa00700 1 --2- 172.21.15.63:0/904443068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848105770 0x7f1848105bf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.807+0000 7f184fa00700 1 -- 172.21.15.63:0/904443068 >> 172.21.15.63:0/904443068 conn(0x7f1848101280 msgr2=0x7f18481036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.808+0000 7f184fa00700 1 -- 172.21.15.63:0/904443068 shutdown_connections 2024-05-11T13:32:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.808+0000 7f184fa00700 1 -- 172.21.15.63:0/904443068 wait complete. 2024-05-11T13:32:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.809+0000 7f184fa00700 1 Processor -- start 2024-05-11T13:32:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.809+0000 7f184fa00700 1 -- start start 2024-05-11T13:32:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.809+0000 7f184fa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1848105770 0x7f184819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.809+0000 7f184fa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848106130 0x7f184819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f184fa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f184810b770 0x7f18481a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f184fa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18481158b0 con 0x7f1848106130 2024-05-11T13:32:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f184fa00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1848115730 con 0x7f1848105770 2024-05-11T13:32:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f184fa00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1848115a30 con 0x7f184810b770 2024-05-11T13:32:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848106130 0x7f184819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848106130 0x7f184819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42644/0 (socket says 172.21.15.63:42644) 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 -- 172.21.15.63:0/2153403086 learned_addr learned my addr 172.21.15.63:0/2153403086 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f184d000700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1848105770 0x7f184819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f184da00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f184810b770 0x7f18481a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 -- 172.21.15.63:0/2153403086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f184810b770 msgr2=0x7f18481a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f184810b770 0x7f18481a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 -- 172.21.15.63:0/2153403086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1848105770 msgr2=0x7f184819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1848105770 0x7f184819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.810+0000 7f1847e00700 1 -- 172.21.15.63:0/2153403086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481a0d50 con 0x7f1848106130 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.811+0000 7f184d000700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1848105770 0x7f184819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.811+0000 7f1847e00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848106130 0x7f184819c280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1834000c00 tx=0x7f1834005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.811+0000 7f1845600700 1 -- 172.21.15.63:0/2153403086 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834027070 con 0x7f1848106130 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.811+0000 7f1845600700 1 -- 172.21.15.63:0/2153403086 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f183400fb80 con 0x7f1848106130 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.811+0000 7f1845600700 1 -- 172.21.15.63:0/2153403086 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834022c30 con 0x7f1848106130 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.811+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481a0fe0 con 0x7f1848106130 2024-05-11T13:32:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.811+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18481a88c0 con 0x7f1848106130 2024-05-11T13:32:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.815+0000 7f1845600700 1 -- 172.21.15.63:0/2153403086 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1834022470 con 0x7f1848106130 2024-05-11T13:32:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.816+0000 7f185a2f0b80 1 Processor -- start 2024-05-11T13:32:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.816+0000 7f185a2f0b80 1 -- start start 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.816+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb80500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceb84550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 0x55d2ceb84ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2cead29d0 con 0x55d2ceb80120 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d2cea70710 con 0x55d2ceb84a90 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceace990 con 0x55d2ceb80a40 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f1845600700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f18300b2880 0x7f18300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f184da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb80500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f184da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb80500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42656/0 (socket says 172.21.15.63:42656) 2024-05-11T13:32:25.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f184da00700 1 -- 172.21.15.63:0/486030792 learned_addr learned my addr 172.21.15.63:0/486030792 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f1845600700 1 -- 172.21.15.63:0/2153403086 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f18340e3300 con 0x7f1848106130 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f184d000700 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceb84550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.817+0000 7f1847e00700 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 0x55d2ceb84ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.818+0000 7f184da00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f18300b2880 0x7f18300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.818+0000 7f184da00700 1 -- 172.21.15.63:0/486030792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 msgr2=0x55d2ceb84550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.818+0000 7f184da00700 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceb84550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.818+0000 7f184da00700 1 -- 172.21.15.63:0/486030792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 msgr2=0x55d2ceb84ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.818+0000 7f184da00700 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 0x55d2ceb84ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.818+0000 7f184da00700 1 -- 172.21.15.63:0/486030792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cea1f880 con 0x55d2ceb80120 2024-05-11T13:32:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.818+0000 7f184da00700 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb80500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1840012ae0 tx=0x7f1840012df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7685c18434b27f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.819+0000 7f183a000700 1 -- 172.21.15.63:0/486030792 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184001c040 con 0x55d2ceb80120 2024-05-11T13:32:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.819+0000 7f184da00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f18300b2880 0x7f18300b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1840019000 tx=0x7f184001b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.819+0000 7f183a000700 1 -- 172.21.15.63:0/486030792 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840011040 con 0x55d2ceb80120 2024-05-11T13:32:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.819+0000 7f183a000700 1 -- 172.21.15.63:0/486030792 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840010050 con 0x55d2ceb80120 2024-05-11T13:32:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.819+0000 7f185a2f0b80 1 -- 172.21.15.63:0/486030792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 msgr2=0x55d2ceb80500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.819+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb80500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1840012ae0 tx=0x7f1840012df0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 -- 172.21.15.63:0/486030792 shutdown_connections 2024-05-11T13:32:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 0x55d2ceb84ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb80500 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/486030792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceb84550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 -- 172.21.15.63:0/486030792 >> 172.21.15.63:0/486030792 conn(0x55d2ceaaeb10 msgr2=0x55d2ceb7a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 -- 172.21.15.63:0/486030792 shutdown_connections 2024-05-11T13:32:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 -- 172.21.15.63:0/486030792 wait complete. 2024-05-11T13:32:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.820+0000 7f185a2f0b80 1 -- start start 2024-05-11T13:32:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 1 Processor -- start 2024-05-11T13:32:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb8bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb89360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb84a90 0x55d2ceb898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb8cc40 con 0x55d2ceb80120 2024-05-11T13:32:25.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d2cea38ad0 con 0x55d2ceb80a40 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb8cdc0 con 0x55d2ceb84a90 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.821+0000 7f185a2f0b80 10 client.0 _ll_register_callbacks cb 0x55d2cea79c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb8bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb8bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42672/0 (socket says 172.21.15.63:42672) 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 -- 172.21.15.63:0/1304365826 learned_addr learned my addr 172.21.15.63:0/1304365826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184da00700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb84a90 0x55d2ceb898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f1847e00700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb89360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 -- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb84a90 msgr2=0x55d2ceb898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb84a90 0x55d2ceb898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 -- 172.21.15.63:0/1304365826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 msgr2=0x55d2ceb89360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb89360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184d000700 1 -- 172.21.15.63:0/1304365826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cea983d0 con 0x55d2ceb80120 2024-05-11T13:32:25.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f1847e00700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb89360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.822+0000 7f184da00700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb84a90 0x55d2ceb898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f184d000700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb8bfc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f183c00da50 tx=0x7f183c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6864f585d70b608 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f182ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f182ea00700 1 -- 172.21.15.63:0/1304365826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c01b440 con 0x55d2ceb80120 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f182ea00700 1 -- 172.21.15.63:0/1304365826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f183c011440 con 0x55d2ceb80120 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1840011040 con 0x55d2ceb80120 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f182ea00700 1 -- 172.21.15.63:0/1304365826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f183c011640 con 0x55d2ceb80120 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f185a2f0b80 10 client.15030 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.823+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2ceb975b0 con 0x55d2ceb80120 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f182ea00700 1 -- 172.21.15.63:0/1304365826 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840011040 con 0x55d2ceb80120 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f182ea00700 1 -- 172.21.15.63:0/1304365826 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f183c019070 con 0x55d2ceb80120 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f185a2f0b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:25.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f185a2f0b80 10 client.15030 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-11T13:32:25.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f185a2f0b80 20 client.15030 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:25.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2ceb99bf0 0x55d2ceb9b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f185a2f0b80 4 client.15030 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:32:25.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.824+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 --> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2ceb4d160 con 0x55d2ceb99bf0 2024-05-11T13:32:25.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.825+0000 7f1847e00700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2ceb99bf0 0x55d2ceb9b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.826+0000 7f1847e00700 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2ceb99bf0 0x55d2ceb9b060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0ca0f8f406254a3 server_cookie=a5be263e368b7f1f in_seq=0 out_seq=0 2024-05-11T13:32:25.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.826+0000 7f182ea00700 10 client.15030 ms_handle_connect on v2:172.21.15.165:6838/1121013005 2024-05-11T13:32:25.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.828+0000 7f182ea00700 1 -- 172.21.15.63:0/1304365826 <== mds.4 v2:172.21.15.165:6838/1121013005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d2ceb4d160 con 0x55d2ceb99bf0 2024-05-11T13:32:25.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.828+0000 7f182ea00700 10 client.15030 handle_command_reply: tid=0 2024-05-11T13:32:25.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.828+0000 7f185a2f0b80 1 client.15030 shutdown 2024-05-11T13:32:25.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 msgr2=0x55d2ceb8bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb8bfc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f183c00da50 tx=0x7f183c00de10 comp rx=0 tx=0).stop 2024-05-11T13:32:25.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 shutdown_connections 2024-05-11T13:32:25.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2ceb99bf0 0x55d2ceb9b060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb84a90 0x55d2ceb898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceb8bfc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/1304365826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb89360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 >> 172.21.15.63:0/1304365826 conn(0x55d2ceaaeb10 msgr2=0x55d2ceb75370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:25.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f182ea00700 0 client.15030 ms_handle_reset on v2:172.21.15.165:6838/1121013005 2024-05-11T13:32:25.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 shutdown_connections 2024-05-11T13:32:25.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 1 -- 172.21.15.63:0/1304365826 wait complete. 2024-05-11T13:32:25.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.829+0000 7f185a2f0b80 20 client.15030 trim_cache size 0 max 16384 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.852+0000 7f185a2f0b80 1 Processor -- start 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f185a2f0b80 1 -- start start 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80120 0x55d2ceb85e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb86df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb84a90 0x55d2ceb74ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb993a0 con 0x55d2ceb84a90 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb99a70 con 0x55d2ceb80a40 2024-05-11T13:32:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb9a140 con 0x55d2ceb80120 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.853+0000 7f184da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb84a90 0x55d2ceb74ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb84a90 0x55d2ceb74ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42680/0 (socket says 172.21.15.63:42680) 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 -- 172.21.15.63:0/3053991103 learned_addr learned my addr 172.21.15.63:0/3053991103 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184d000700 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80120 0x55d2ceb85e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f1847e00700 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb86df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 -- 172.21.15.63:0/3053991103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80120 msgr2=0x55d2ceb85e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80120 0x55d2ceb85e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 -- 172.21.15.63:0/3053991103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 msgr2=0x55d2ceb86df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb86df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 -- 172.21.15.63:0/3053991103 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ceb774b0 con 0x55d2ceb84a90 2024-05-11T13:32:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f1847e00700 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb86df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.854+0000 7f184da00700 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb84a90 0x55d2ceb74ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f184001e000 tx=0x7f18400077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bcb57f8f70200f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f1839600700 1 -- 172.21.15.63:0/3053991103 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184002d020 con 0x55d2ceb84a90 2024-05-11T13:32:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f1839600700 1 -- 172.21.15.63:0/3053991103 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840021c80 con 0x55d2ceb84a90 2024-05-11T13:32:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f1839600700 1 -- 172.21.15.63:0/3053991103 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840025040 con 0x55d2ceb84a90 2024-05-11T13:32:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f185a2f0b80 1 -- 172.21.15.63:0/3053991103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb84a90 msgr2=0x55d2ceb74ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb84a90 0x55d2ceb74ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f184001e000 tx=0x7f18400077f0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f185a2f0b80 1 -- 172.21.15.63:0/3053991103 shutdown_connections 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb84a90 0x55d2ceb74ae0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.855+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80120 0x55d2ceb85e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.856+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/3053991103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb80a40 0x55d2ceb86df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.856+0000 7f185a2f0b80 1 -- 172.21.15.63:0/3053991103 >> 172.21.15.63:0/3053991103 conn(0x55d2ceaaeb10 msgr2=0x55d2ceb866e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.856+0000 7f185a2f0b80 1 -- 172.21.15.63:0/3053991103 shutdown_connections 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.856+0000 7f185a2f0b80 1 -- 172.21.15.63:0/3053991103 wait complete. 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.856+0000 7f185a2f0b80 1 -- start start 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.856+0000 7f185a2f0b80 1 Processor -- start 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceba44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceba4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f185a2f0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 0x55d2ceb9f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb9aee0 con 0x55d2ceb80120 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb9a810 con 0x55d2ceb84a90 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f185a2f0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb9b5b0 con 0x55d2ceb80a40 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f185a2f0b80 10 client.0 _ll_register_callbacks cb 0x55d2cea79c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f184d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceba44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f184d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceba44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42688/0 (socket says 172.21.15.63:42688) 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f184d000700 1 -- 172.21.15.63:0/158575899 learned_addr learned my addr 172.21.15.63:0/158575899 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.857+0000 7f1847e00700 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceba4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f184d000700 1 -- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 msgr2=0x55d2ceba4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f184d000700 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceba4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f184d000700 1 -- 172.21.15.63:0/158575899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 msgr2=0x55d2ceb9f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f184d000700 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 0x55d2ceb9f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f184d000700 1 -- 172.21.15.63:0/158575899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ceb9fa80 con 0x55d2ceb80120 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f184d000700 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceba44e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f183c005a80 tx=0x7f183c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412e59b70f4fdd5b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f182ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f182ea00700 1 -- 172.21.15.63:0/158575899 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c02d020 con 0x55d2ceb80120 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f182ea00700 1 -- 172.21.15.63:0/158575899 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f183c0119f0 con 0x55d2ceb80120 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.858+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1840021c80 con 0x55d2ceb80120 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f182ea00700 1 -- 172.21.15.63:0/158575899 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f183c038050 con 0x55d2ceb80120 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 10 client.15036 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2cec0e130 con 0x55d2ceb80120 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f182ea00700 1 -- 172.21.15.63:0/158575899 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840021c80 con 0x55d2ceb80120 2024-05-11T13:32:25.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f182ea00700 1 -- 172.21.15.63:0/158575899 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f183c019040 con 0x55d2ceb80120 2024-05-11T13:32:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 10 client.15036 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-11T13:32:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 20 client.15036 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2cec12e10 0x55d2cec11b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 4 client.15036 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:32:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.859+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 --> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] -- command(tid 0: {"prefix": "status"}) -- 0x55d2ceae03b0 con 0x55d2cec12e10 2024-05-11T13:32:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.860+0000 7f1847e00700 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2cec12e10 0x55d2cec11b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.861+0000 7f1847e00700 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2cec12e10 0x55d2cec11b20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=146801df1f05fe0b server_cookie=564d384cf6230c61 in_seq=0 out_seq=0 2024-05-11T13:32:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.861+0000 7f182ea00700 10 client.15036 ms_handle_connect on v2:172.21.15.165:6838/1121013005 2024-05-11T13:32:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f182ea00700 1 -- 172.21.15.63:0/158575899 <== mds.4 v2:172.21.15.165:6838/1121013005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d2ceae03b0 con 0x55d2cec12e10 2024-05-11T13:32:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f182ea00700 10 client.15036 handle_command_reply: tid=0 2024-05-11T13:32:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 client.15036 shutdown 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 msgr2=0x55d2ceba44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceba44e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f183c005a80 tx=0x7f183c004430 comp rx=0 tx=0).stop 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 shutdown_connections 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55d2cec12e10 0x55d2cec11b20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2ceb84a90 0x55d2ceb9f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2ceb80120 0x55d2ceba44e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 --2- 172.21.15.63:0/158575899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2ceb80a40 0x55d2ceba4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 >> 172.21.15.63:0/158575899 conn(0x55d2ceaaeb10 msgr2=0x55d2ceb74ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f182ea00700 0 client.15036 ms_handle_reset on v2:172.21.15.165:6838/1121013005 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 shutdown_connections 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 1 -- 172.21.15.63:0/158575899 wait complete. 2024-05-11T13:32:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.862+0000 7f185a2f0b80 20 client.15036 trim_cache size 0 max 16384 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 4, 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 24472, 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 148.10623987400001, 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 260.17378357500002 2024-05-11T13:32:25.865 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f18300b2880 msgr2=0x7f18300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f18300b2880 0x7f18300b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1840019000 tx=0x7f184001b000 comp rx=0 tx=0).stop 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848106130 msgr2=0x7f184819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848106130 0x7f184819c280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1834000c00 tx=0x7f1834005ef0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 shutdown_connections 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f18300b2880 0x7f18300b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f184810b770 0x7f18481a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1848105770 0x7f184819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 --2- 172.21.15.63:0/2153403086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1848106130 0x7f184819c280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.863+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 >> 172.21.15.63:0/2153403086 conn(0x7f1848101280 msgr2=0x7f1848101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:25.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.864+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 shutdown_connections 2024-05-11T13:32:25.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:25.864+0000 7f184fa00700 1 -- 172.21.15.63:0/2153403086 wait complete. 2024-05-11T13:32:26.690 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.k status 2024-05-11T13:32:27.150 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:27.353 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:27 smithi063 ceph-mon[29809]: pgmap v493: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:27.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:27 smithi071 ceph-mon[33293]: pgmap v493: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:27.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:27 smithi165 ceph-mon[33235]: pgmap v493: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:28.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:28 smithi063 ceph-mon[29809]: pgmap v494: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:28.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:28 smithi071 ceph-mon[33293]: pgmap v494: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:28.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:28 smithi165 ceph-mon[33235]: pgmap v494: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.405+0000 7feb4ca00700 1 -- 172.21.15.63:0/704088638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48114490 msgr2=0x7feb48116880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.405+0000 7feb4ca00700 1 --2- 172.21.15.63:0/704088638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48114490 0x7feb48116880 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb40009990 tx=0x7feb40009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.405+0000 7feb4ca00700 1 -- 172.21.15.63:0/704088638 shutdown_connections 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.405+0000 7feb4ca00700 1 --2- 172.21.15.63:0/704088638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48114490 0x7feb48116880 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.405+0000 7feb4ca00700 1 --2- 172.21.15.63:0/704088638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb480758f0 0x7feb48075cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.405+0000 7feb4ca00700 1 --2- 172.21.15.63:0/704088638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb48076210 0x7feb48113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.405+0000 7feb4ca00700 1 -- 172.21.15.63:0/704088638 >> 172.21.15.63:0/704088638 conn(0x7feb480fefd0 msgr2=0x7feb481013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.406+0000 7feb4ca00700 1 -- 172.21.15.63:0/704088638 shutdown_connections 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.406+0000 7feb4ca00700 1 -- 172.21.15.63:0/704088638 wait complete. 2024-05-11T13:32:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 Processor -- start 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 -- start start 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb480758f0 0x7feb481a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48076210 0x7feb481a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb48114490 0x7feb481a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb48119b50 con 0x7feb48076210 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb481199d0 con 0x7feb48114490 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.407+0000 7feb4ca00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feb48119cd0 con 0x7feb480758f0 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48076210 0x7feb481a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48076210 0x7feb481a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42716/0 (socket says 172.21.15.63:42716) 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 -- 172.21.15.63:0/1784958795 learned_addr learned my addr 172.21.15.63:0/1784958795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb47400700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb480758f0 0x7feb481a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb47e00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb48114490 0x7feb481a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 -- 172.21.15.63:0/1784958795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb480758f0 msgr2=0x7feb481a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb480758f0 0x7feb481a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 -- 172.21.15.63:0/1784958795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb48114490 msgr2=0x7feb481a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb48114490 0x7feb481a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb3ea00700 1 -- 172.21.15.63:0/1784958795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb481a5010 con 0x7feb48076210 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.408+0000 7feb47400700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb480758f0 0x7feb481a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:29.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.409+0000 7feb3ea00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48076210 0x7feb481a0540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c9b0 tx=0x7feb3000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:29.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.409+0000 7feb44c00700 1 -- 172.21.15.63:0/1784958795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30007b00 con 0x7feb48076210 2024-05-11T13:32:29.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.409+0000 7feb44c00700 1 -- 172.21.15.63:0/1784958795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb30007ca0 con 0x7feb48076210 2024-05-11T13:32:29.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.409+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb481a5300 con 0x7feb48076210 2024-05-11T13:32:29.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.409+0000 7feb44c00700 1 -- 172.21.15.63:0/1784958795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30018660 con 0x7feb48076210 2024-05-11T13:32:29.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.409+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb481acb80 con 0x7feb48076210 2024-05-11T13:32:29.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.413+0000 7feb44c00700 1 -- 172.21.15.63:0/1784958795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feb3000f450 con 0x7feb48076210 2024-05-11T13:32:29.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.414+0000 7feb58a47b80 1 Processor -- start 2024-05-11T13:32:29.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.414+0000 7feb58a47b80 1 -- start start 2024-05-11T13:32:29.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.414+0000 7feb44c00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7feb2c0b2880 0x7feb2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef11300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef11d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb58a47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b70ee639d0 con 0x55b70ef17760 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb58a47b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b70ee01710 con 0x55b70ef118d0 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb58a47b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b70ee5f990 con 0x55b70ef16e40 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb44c00700 1 -- 172.21.15.63:0/1784958795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb300e2610 con 0x7feb48076210 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb47e00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7feb2c0b2880 0x7feb2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef11300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef11300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42732/0 (socket says 172.21.15.63:42732) 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb3ea00700 1 -- 172.21.15.63:0/78336403 learned_addr learned my addr 172.21.15.63:0/78336403 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:29.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.415+0000 7feb47400700 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3ea00700 1 -- 172.21.15.63:0/78336403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 msgr2=0x55b70ef17220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3ea00700 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef17220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3ea00700 1 -- 172.21.15.63:0/78336403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 msgr2=0x55b70ef11d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3ea00700 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef11d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3ea00700 1 -- 172.21.15.63:0/78336403 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70edb0880 con 0x55b70ef17760 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb47e00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7feb2c0b2880 0x7feb2c0b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feb4000b5c0 tx=0x7feb400057e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3ea00700 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef11300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb300a58a0 tx=0x7feb300a9090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0857a51d46298c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3cc00700 1 -- 172.21.15.63:0/78336403 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300a9820 con 0x55b70ef17760 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3cc00700 1 -- 172.21.15.63:0/78336403 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb300adbd0 con 0x55b70ef17760 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.416+0000 7feb3cc00700 1 -- 172.21.15.63:0/78336403 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300a9a80 con 0x55b70ef17760 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 -- 172.21.15.63:0/78336403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 msgr2=0x55b70ef11300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef11300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb300a58a0 tx=0x7feb300a9090 comp rx=0 tx=0).stop 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 -- 172.21.15.63:0/78336403 shutdown_connections 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef11d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef17220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 --2- 172.21.15.63:0/78336403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef11300 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 -- 172.21.15.63:0/78336403 >> 172.21.15.63:0/78336403 conn(0x55b70ee3fb10 msgr2=0x55b70ef0bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:29.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 -- 172.21.15.63:0/78336403 shutdown_connections 2024-05-11T13:32:29.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.417+0000 7feb58a47b80 1 -- 172.21.15.63:0/78336403 wait complete. 2024-05-11T13:32:29.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.418+0000 7feb58a47b80 1 -- start start 2024-05-11T13:32:29.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.418+0000 7feb58a47b80 1 Processor -- start 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef118d0 0x55b70ef1d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef16e40 0x55b70ef1a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef1ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb58a47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef1e1b0 con 0x55b70ef118d0 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb58a47b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b70edc9ad0 con 0x55b70ef16e40 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb58a47b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef1e330 con 0x55b70ef17760 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb58a47b80 10 client.0 _ll_register_callbacks cb 0x55b70edf7330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:29.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef118d0 0x55b70ef1d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef118d0 0x55b70ef1d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42742/0 (socket says 172.21.15.63:42742) 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.419+0000 7feb47400700 1 -- 172.21.15.63:0/585854263 learned_addr learned my addr 172.21.15.63:0/585854263 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb3ea00700 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef16e40 0x55b70ef1a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb47e00700 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef1ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb47400700 1 -- 172.21.15.63:0/585854263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 msgr2=0x55b70ef1ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb47400700 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef1ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb47400700 1 -- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef16e40 msgr2=0x55b70ef1a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb47400700 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef16e40 0x55b70ef1a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb47400700 1 -- 172.21.15.63:0/585854263 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70ee293d0 con 0x55b70ef118d0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb47400700 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef118d0 0x55b70ef1d440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feb38007b80 tx=0x7feb38007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42f3e43a2884635 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.420+0000 7feb35600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.421+0000 7feb35600700 1 -- 172.21.15.63:0/585854263 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3801a070 con 0x55b70ef118d0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.421+0000 7feb35600700 1 -- 172.21.15.63:0/585854263 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb38012c00 con 0x55b70ef118d0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.421+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb300adbd0 con 0x55b70ef118d0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.421+0000 7feb35600700 1 -- 172.21.15.63:0/585854263 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb38026050 con 0x55b70ef118d0 2024-05-11T13:32:29.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.421+0000 7feb58a47b80 10 client.15048 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.421+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b70ef28a30 con 0x55b70ef118d0 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.421+0000 7feb35600700 1 -- 172.21.15.63:0/585854263 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb300adbd0 con 0x55b70ef118d0 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.422+0000 7feb35600700 1 -- 172.21.15.63:0/585854263 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb380153c0 con 0x55b70ef118d0 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.422+0000 7feb58a47b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.422+0000 7feb58a47b80 10 client.15048 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.422+0000 7feb58a47b80 20 client.15048 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.422+0000 7feb58a47b80 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef2b060 0x55b70ef2c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.422+0000 7feb58a47b80 4 client.15048 mds_command: new command op to 34403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.422+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 --> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b70eede010 con 0x55b70ef2b060 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.423+0000 7feb3ea00700 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef2b060 0x55b70ef2c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.424+0000 7feb3ea00700 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef2b060 0x55b70ef2c4d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fa04a908a81eb6 server_cookie=e2a2f154649b78e3 in_seq=0 out_seq=0 2024-05-11T13:32:29.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.424+0000 7feb35600700 10 client.15048 ms_handle_connect on v2:172.21.15.71:6832/1870291573 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.426+0000 7feb35600700 1 -- 172.21.15.63:0/585854263 <== mds.3 v2:172.21.15.71:6832/1870291573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b70eede010 con 0x55b70ef2b060 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.426+0000 7feb35600700 10 client.15048 handle_command_reply: tid=0 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.426+0000 7feb58a47b80 1 client.15048 shutdown 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.426+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef118d0 msgr2=0x55b70ef1d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.426+0000 7feb58a47b80 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef118d0 0x55b70ef1d440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feb38007b80 tx=0x7feb38007e90 comp rx=0 tx=0).stop 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 shutdown_connections 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef2b060 0x55b70ef2c4d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef1ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef118d0 0x55b70ef1d440 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 --2- 172.21.15.63:0/585854263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef16e40 0x55b70ef1a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb35600700 0 client.15048 ms_handle_reset on v2:172.21.15.71:6832/1870291573 2024-05-11T13:32:29.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 >> 172.21.15.63:0/585854263 conn(0x55b70ee3fb10 msgr2=0x55b70ef102f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:29.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 shutdown_connections 2024-05-11T13:32:29.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 1 -- 172.21.15.63:0/585854263 wait complete. 2024-05-11T13:32:29.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.427+0000 7feb58a47b80 20 client.15048 trim_cache size 0 max 16384 2024-05-11T13:32:29.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.450+0000 7feb58a47b80 1 Processor -- start 2024-05-11T13:32:29.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.450+0000 7feb58a47b80 1 -- start start 2024-05-11T13:32:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef11300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef16e40 0x55b70ef35930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb58a47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef2a810 con 0x55b70ef16e40 2024-05-11T13:32:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb58a47b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef2aee0 con 0x55b70ef118d0 2024-05-11T13:32:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb58a47b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef2b5b0 con 0x55b70ef17760 2024-05-11T13:32:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef16e40 0x55b70ef35930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef16e40 0x55b70ef35930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42750/0 (socket says 172.21.15.63:42750) 2024-05-11T13:32:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb3ea00700 1 -- 172.21.15.63:0/3955214320 learned_addr learned my addr 172.21.15.63:0/3955214320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb47400700 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef11300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.451+0000 7feb47e00700 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb3ea00700 1 -- 172.21.15.63:0/3955214320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 msgr2=0x55b70ef35e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb3ea00700 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef35e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb3ea00700 1 -- 172.21.15.63:0/3955214320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 msgr2=0x55b70ef11300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb3ea00700 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef11300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb3ea00700 1 -- 172.21.15.63:0/3955214320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70ef08770 con 0x55b70ef16e40 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb3ea00700 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef16e40 0x55b70ef35930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb300a6a90 tx=0x7feb300b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f220cbf4c73cb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb37e00700 1 -- 172.21.15.63:0/3955214320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300b2680 con 0x55b70ef16e40 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb37e00700 1 -- 172.21.15.63:0/3955214320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb300a9820 con 0x55b70ef16e40 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.452+0000 7feb37e00700 1 -- 172.21.15.63:0/3955214320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3000fa50 con 0x55b70ef16e40 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 -- 172.21.15.63:0/3955214320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef16e40 msgr2=0x55b70ef35930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef16e40 0x55b70ef35930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb300a6a90 tx=0x7feb300b2040 comp rx=0 tx=0).stop 2024-05-11T13:32:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 -- 172.21.15.63:0/3955214320 shutdown_connections 2024-05-11T13:32:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef17760 0x55b70ef35e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef11300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 --2- 172.21.15.63:0/3955214320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef16e40 0x55b70ef35930 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 -- 172.21.15.63:0/3955214320 >> 172.21.15.63:0/3955214320 conn(0x55b70ee3fb10 msgr2=0x55b70ef0ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 -- 172.21.15.63:0/3955214320 shutdown_connections 2024-05-11T13:32:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.453+0000 7feb58a47b80 1 -- 172.21.15.63:0/3955214320 wait complete. 2024-05-11T13:32:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.454+0000 7feb58a47b80 1 -- start start 2024-05-11T13:32:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.454+0000 7feb58a47b80 1 Processor -- start 2024-05-11T13:32:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.454+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.454+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef26d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.454+0000 7feb58a47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef27260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb58a47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef2c350 con 0x55b70ef17760 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb58a47b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef2bc80 con 0x55b70ef118d0 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb58a47b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b70ef2ca20 con 0x55b70ef16e40 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb58a47b80 10 client.0 _ll_register_callbacks cb 0x55b70edf7330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef27260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef27260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42754/0 (socket says 172.21.15.63:42754) 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 -- 172.21.15.63:0/2844926207 learned_addr learned my addr 172.21.15.63:0/2844926207 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47400700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb3ea00700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef26d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 -- 172.21.15.63:0/2844926207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 msgr2=0x55b70ef26d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef26d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 -- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 msgr2=0x55b70ef267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef267e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb47e00700 1 -- 172.21.15.63:0/2844926207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70ef33ca0 con 0x55b70ef17760 2024-05-11T13:32:29.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.455+0000 7feb3ea00700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef26d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.456+0000 7feb47e00700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef27260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb40005cd0 tx=0x7feb40009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b4ee60f048e502 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.456+0000 7feb35600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.456+0000 7feb35600700 1 -- 172.21.15.63:0/2844926207 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40035060 con 0x55b70ef17760 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.456+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb300a9820 con 0x55b70ef17760 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.456+0000 7feb35600700 1 -- 172.21.15.63:0/2844926207 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb4000fde0 con 0x55b70ef17760 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb35600700 1 -- 172.21.15.63:0/2844926207 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb4003c050 con 0x55b70ef17760 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 10 client.15054 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b70ef967e0 con 0x55b70ef17760 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb35600700 1 -- 172.21.15.63:0/2844926207 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb300a9820 con 0x55b70ef17760 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb35600700 1 -- 172.21.15.63:0/2844926207 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb40024070 con 0x55b70ef17760 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 10 client.15054 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 20 client.15054 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef9b3e0 0x55b70ef9a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 4 client.15054 mds_command: new command op to 34403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:32:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.457+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 --> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] -- command(tid 0: {"prefix": "status"}) -- 0x55b70ee713b0 con 0x55b70ef9b3e0 2024-05-11T13:32:29.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.458+0000 7feb47400700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef9b3e0 0x55b70ef9a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.459+0000 7feb47400700 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef9b3e0 0x55b70ef9a0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e39f60dd5aa6d280 server_cookie=8225a172e08093a3 in_seq=0 out_seq=0 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.459+0000 7feb35600700 10 client.15054 ms_handle_connect on v2:172.21.15.71:6832/1870291573 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.459+0000 7feb35600700 1 -- 172.21.15.63:0/2844926207 <== mds.3 v2:172.21.15.71:6832/1870291573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b70ee713b0 con 0x55b70ef9b3e0 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.459+0000 7feb35600700 10 client.15054 handle_command_reply: tid=0 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.459+0000 7feb58a47b80 1 client.15054 shutdown 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 msgr2=0x55b70ef27260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef27260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb40005cd0 tx=0x7feb40009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 shutdown_connections 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b70ef17760 0x55b70ef27260 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b70ef118d0 0x55b70ef267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55b70ef9b3e0 0x55b70ef9a0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 --2- 172.21.15.63:0/2844926207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70ef16e40 0x55b70ef26d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 >> 172.21.15.63:0/2844926207 conn(0x55b70ee3fb10 msgr2=0x55b70ef18d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb35600700 0 client.15054 ms_handle_reset on v2:172.21.15.71:6832/1870291573 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 shutdown_connections 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 1 -- 172.21.15.63:0/2844926207 wait complete. 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.460+0000 7feb58a47b80 20 client.15054 trim_cache size 0 max 16384 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 3, 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34403, 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:32:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 152.72036566599999, 2024-05-11T13:32:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:32:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:32:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:32:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 295.64504608200002 2024-05-11T13:32:29.464 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:32:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7feb2c0b2880 msgr2=0x7feb2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7feb2c0b2880 0x7feb2c0b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feb4000b5c0 tx=0x7feb400057e0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48076210 msgr2=0x7feb481a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48076210 0x7feb481a0540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c9b0 tx=0x7feb3000cd70 comp rx=0 tx=0).stop 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 shutdown_connections 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7feb2c0b2880 0x7feb2c0b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb48114490 0x7feb481a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb480758f0 0x7feb481a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 --2- 172.21.15.63:0/1784958795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb48076210 0x7feb481a0540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 >> 172.21.15.63:0/1784958795 conn(0x7feb480fefd0 msgr2=0x7feb480ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 shutdown_connections 2024-05-11T13:32:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:29.461+0000 7feb4ca00700 1 -- 172.21.15.63:0/1784958795 wait complete. 2024-05-11T13:32:30.338 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.e status 2024-05-11T13:32:30.801 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:30.990 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:30 smithi063 ceph-mon[29809]: pgmap v495: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:31.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:30 smithi165 ceph-mon[33235]: pgmap v495: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:31.157 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:30 smithi071 ceph-mon[33293]: pgmap v495: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:32.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:32 smithi063 ceph-mon[29809]: pgmap v496: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:33.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:32 smithi165 ceph-mon[33235]: pgmap v496: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 -- 172.21.15.63:0/2376659041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b1410b790 msgr2=0x7f3b1410bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/2376659041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b1410b790 0x7f3b1410bb70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009990 tx=0x7f3b04009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 -- 172.21.15.63:0/2376659041 shutdown_connections 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/2376659041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b14106150 0x7f3b14112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/2376659041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b1410b790 0x7f3b1410bb70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/2376659041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b14105790 0x7f3b14105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 -- 172.21.15.63:0/2376659041 >> 172.21.15.63:0/2376659041 conn(0x7f3b14101280 msgr2=0x7f3b141036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 -- 172.21.15.63:0/2376659041 shutdown_connections 2024-05-11T13:32:33.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.179+0000 7f3b1ae00700 1 -- 172.21.15.63:0/2376659041 wait complete. 2024-05-11T13:32:33.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.181+0000 7f3b1ae00700 1 Processor -- start 2024-05-11T13:32:33.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.181+0000 7f3b1ae00700 1 -- start start 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.181+0000 7f3b1ae00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b14105790 0x7f3b1419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b1ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b14106150 0x7f3b1419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b1ae00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1410b790 0x7f3b141a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b1ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b14115890 con 0x7f3b14106150 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b1ae00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3b14115710 con 0x7f3b14105790 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b1ae00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b14115a10 con 0x7f3b1410b790 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b14106150 0x7f3b1419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b14106150 0x7f3b1419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52306/0 (socket says 172.21.15.63:52306) 2024-05-11T13:32:33.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b13400700 1 -- 172.21.15.63:0/1908923453 learned_addr learned my addr 172.21.15.63:0/1908923453 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:33.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b13e00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b14105790 0x7f3b1419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.182+0000 7f3b18e00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1410b790 0x7f3b141a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b13400700 1 -- 172.21.15.63:0/1908923453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1410b790 msgr2=0x7f3b141a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b13400700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1410b790 0x7f3b141a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b13400700 1 -- 172.21.15.63:0/1908923453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b14105790 msgr2=0x7f3b1419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b13400700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b14105790 0x7f3b1419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b13400700 1 -- 172.21.15.63:0/1908923453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b141a0d90 con 0x7f3b14106150 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b13400700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b14106150 0x7f3b1419c2c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d950 tx=0x7f3b0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b10c00700 1 -- 172.21.15.63:0/1908923453 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00016070 con 0x7f3b14106150 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b10c00700 1 -- 172.21.15.63:0/1908923453 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b00009dd0 con 0x7f3b14106150 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b10c00700 1 -- 172.21.15.63:0/1908923453 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000f3c0 con 0x7f3b14106150 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.183+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b141a1080 con 0x7f3b14106150 2024-05-11T13:32:33.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.184+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b141a8900 con 0x7f3b14106150 2024-05-11T13:32:33.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.187+0000 7f3b10c00700 1 -- 172.21.15.63:0/1908923453 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3b00019070 con 0x7f3b14106150 2024-05-11T13:32:33.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.188+0000 7f3b25857b80 1 Processor -- start 2024-05-11T13:32:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.188+0000 7f3b25857b80 1 -- start start 2024-05-11T13:32:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b10c00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3af80b2880 0x7f3af80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b10c00700 1 -- 172.21.15.63:0/1908923453 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3b000e2290 con 0x7f3b14106150 2024-05-11T13:32:33.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048889550 0x55a048883300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048883920 0x55a048883d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b25857b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a0487d59d0 con 0x55a048888c30 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b25857b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a048773710 con 0x55a048883920 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b25857b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a0487d1990 con 0x55a048889550 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b18e00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3af80b2880 0x7f3af80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b13e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b13e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52310/0 (socket says 172.21.15.63:52310) 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.189+0000 7f3b13e00700 1 -- 172.21.15.63:0/1732035189 learned_addr learned my addr 172.21.15.63:0/1732035189 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b18e00700 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048883920 0x55a048883d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13e00700 1 -- 172.21.15.63:0/1732035189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048889550 msgr2=0x55a048883300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13400700 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048889550 0x55a048883300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13e00700 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048889550 0x55a048883300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13e00700 1 -- 172.21.15.63:0/1732035189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048883920 msgr2=0x55a048883d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13e00700 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048883920 0x55a048883d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13e00700 1 -- 172.21.15.63:0/1732035189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a048722880 con 0x55a048888c30 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13400700 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048889550 0x55a048883300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:33.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b18e00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3af80b2880 0x7f3af80b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0060b0 tx=0x7f3b0c007890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:33.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.190+0000 7f3b13e00700 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009750 tx=0x7f3b040036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994e3e175a666217 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.191+0000 7f3b09600700 1 -- 172.21.15.63:0/1732035189 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04027070 con 0x55a048888c30 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.191+0000 7f3b09600700 1 -- 172.21.15.63:0/1732035189 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b040037c0 con 0x55a048888c30 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.191+0000 7f3b09600700 1 -- 172.21.15.63:0/1732035189 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400ba30 con 0x55a048888c30 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 -- 172.21.15.63:0/1732035189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 msgr2=0x55a048889010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009750 tx=0x7f3b040036a0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 -- 172.21.15.63:0/1732035189 shutdown_connections 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048883920 0x55a048883d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889010 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 --2- 172.21.15.63:0/1732035189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048889550 0x55a048883300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 -- 172.21.15.63:0/1732035189 >> 172.21.15.63:0/1732035189 conn(0x55a0487b1b10 msgr2=0x55a04887dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 -- 172.21.15.63:0/1732035189 shutdown_connections 2024-05-11T13:32:33.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 -- 172.21.15.63:0/1732035189 wait complete. 2024-05-11T13:32:33.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.192+0000 7f3b25857b80 1 -- start start 2024-05-11T13:32:33.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 1 Processor -- start 2024-05-11T13:32:33.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048888c30 0x55a04888c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048889550 0x55a04888cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a04888ff60 con 0x55a048889550 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a04873bad0 con 0x55a048888c30 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a0488900e0 con 0x55a048883920 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.193+0000 7f3b25857b80 10 client.0 _ll_register_callbacks cb 0x55a048769330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048889550 0x55a04888cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048889550 0x55a04888cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52320/0 (socket says 172.21.15.63:52320) 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 -- 172.21.15.63:0/3229489724 learned_addr learned my addr 172.21.15.63:0/3229489724 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b13e00700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b13400700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048888c30 0x55a04888c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 -- 172.21.15.63:0/3229489724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 msgr2=0x55a04888f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 -- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048888c30 msgr2=0x55a04888c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048888c30 0x55a04888c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 -- 172.21.15.63:0/3229489724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04879b3d0 con 0x55a048889550 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b13e00700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.194+0000 7f3b18e00700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048889550 0x55a04888cbf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009390 tx=0x7f3b0c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b7b6ab3d959e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.195+0000 7f3afe000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.195+0000 7f3afe000700 1 -- 172.21.15.63:0/3229489724 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c012040 con 0x55a048889550 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.195+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b040037c0 con 0x55a048889550 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.195+0000 7f3afe000700 1 -- 172.21.15.63:0/3229489724 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3b0c0189c0 con 0x55a048889550 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.195+0000 7f3afe000700 1 -- 172.21.15.63:0/3229489724 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b0c030050 con 0x55a048889550 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.195+0000 7f3b25857b80 10 client.15066 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:33.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.195+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a04889a7b0 con 0x55a048889550 2024-05-11T13:32:33.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3afe000700 1 -- 172.21.15.63:0/3229489724 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b040037c0 con 0x55a048889550 2024-05-11T13:32:33.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3afe000700 1 -- 172.21.15.63:0/3229489724 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3b0c018040 con 0x55a048889550 2024-05-11T13:32:33.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3b25857b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3b25857b80 10 client.15066 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3b25857b80 20 client.15066 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04889ce30 0x55a04889e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3b25857b80 4 client.15066 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.196+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 --> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a048850010 con 0x55a04889ce30 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.197+0000 7f3b13e00700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04889ce30 0x55a04889e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.198+0000 7f3b13e00700 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04889ce30 0x55a04889e2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8f8f36d3c89df05 server_cookie=fab1ac4ee28c0f1f in_seq=0 out_seq=0 2024-05-11T13:32:33.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.198+0000 7f3afe000700 10 client.15066 ms_handle_connect on v2:172.21.15.71:6834/1316489545 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3afe000700 1 -- 172.21.15.63:0/3229489724 <== mds.2 v2:172.21.15.71:6834/1316489545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a048850010 con 0x55a04889ce30 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3afe000700 10 client.15066 handle_command_reply: tid=0 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 client.15066 shutdown 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048889550 msgr2=0x55a04888cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048889550 0x55a04888cbf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009390 tx=0x7f3b0c019000 comp rx=0 tx=0).stop 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 shutdown_connections 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04889ce30 0x55a04889e2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048889550 0x55a04888cbf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3229489724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048888c30 0x55a04888c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3afe000700 0 client.15066 ms_handle_reset on v2:172.21.15.71:6834/1316489545 2024-05-11T13:32:33.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.200+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 >> 172.21.15.63:0/3229489724 conn(0x55a0487b1b10 msgr2=0x55a0488786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:33.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.201+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 shutdown_connections 2024-05-11T13:32:33.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.201+0000 7f3b25857b80 1 -- 172.21.15.63:0/3229489724 wait complete. 2024-05-11T13:32:33.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.201+0000 7f3b25857b80 20 client.15066 trim_cache size 0 max 16384 2024-05-11T13:32:33.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:32 smithi071 ceph-mon[33293]: pgmap v496: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:33.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 Processor -- start 2024-05-11T13:32:33.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 -- start start 2024-05-11T13:32:33.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a048889e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a04889c5e0 con 0x55a048888c30 2024-05-11T13:32:33.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a04889ccb0 con 0x55a048889550 2024-05-11T13:32:33.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b25857b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a04889d380 con 0x55a048883920 2024-05-11T13:32:33.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.224+0000 7f3b13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52324/0 (socket says 172.21.15.63:52324) 2024-05-11T13:32:33.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b13400700 1 -- 172.21.15.63:0/2923380368 learned_addr learned my addr 172.21.15.63:0/2923380368 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:33.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b13e00700 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b18e00700 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a048889e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b13400700 1 -- 172.21.15.63:0/2923380368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 msgr2=0x55a04888cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b13400700 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b13400700 1 -- 172.21.15.63:0/2923380368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 msgr2=0x55a048889e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b13400700 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a048889e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b13400700 1 -- 172.21.15.63:0/2923380368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04887fb50 con 0x55a048888c30 2024-05-11T13:32:33.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.225+0000 7f3b18e00700 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a048889e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:33.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.226+0000 7f3b13400700 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3b000a91a0 tx=0x7f3b000a9c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a63bcdf2aa6aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:33.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.226+0000 7f3b08c00700 1 -- 172.21.15.63:0/2923380368 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00020040 con 0x55a048888c30 2024-05-11T13:32:33.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.226+0000 7f3b08c00700 1 -- 172.21.15.63:0/2923380368 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b000e0e50 con 0x55a048888c30 2024-05-11T13:32:33.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.226+0000 7f3b08c00700 1 -- 172.21.15.63:0/2923380368 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000106a0 con 0x55a048888c30 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 -- 172.21.15.63:0/2923380368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 msgr2=0x55a048889950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3b000a91a0 tx=0x7f3b000a9c20 comp rx=0 tx=0).stop 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 -- 172.21.15.63:0/2923380368 shutdown_connections 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a048889e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048883920 0x55a04888cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 --2- 172.21.15.63:0/2923380368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048888c30 0x55a048889950 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 -- 172.21.15.63:0/2923380368 >> 172.21.15.63:0/2923380368 conn(0x55a0487b1b10 msgr2=0x55a04888afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 -- 172.21.15.63:0/2923380368 shutdown_connections 2024-05-11T13:32:33.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.227+0000 7f3b25857b80 1 -- 172.21.15.63:0/2923380368 wait complete. 2024-05-11T13:32:33.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.228+0000 7f3b25857b80 1 -- start start 2024-05-11T13:32:33.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.228+0000 7f3b25857b80 1 Processor -- start 2024-05-11T13:32:33.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.228+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048883920 0x55a04888c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048888c30 0x55a04888c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b25857b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a0488a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b25857b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a04889e120 con 0x55a048883920 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b25857b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a04889da50 con 0x55a048889550 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b25857b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a04889e7f0 con 0x55a048888c30 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b25857b80 10 client.0 _ll_register_callbacks cb 0x55a048769330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048883920 0x55a04888c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048883920 0x55a04888c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52336/0 (socket says 172.21.15.63:52336) 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 -- 172.21.15.63:0/3424074289 learned_addr learned my addr 172.21.15.63:0/3424074289 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b18e00700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a0488a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 -- 172.21.15.63:0/3424074289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048888c30 msgr2=0x55a04888c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13400700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048888c30 0x55a04888c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048888c30 0x55a04888c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 -- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 msgr2=0x55a0488a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a0488a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13e00700 1 -- 172.21.15.63:0/3424074289 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0488a6350 con 0x55a048883920 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b18e00700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a0488a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.229+0000 7f3b13400700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048888c30 0x55a04888c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3b13e00700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048883920 0x55a04888c1d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3b04025040 tx=0x7f3b040093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dec48068f44bb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3afe000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:33.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3afe000700 1 -- 172.21.15.63:0/3424074289 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04035070 con 0x55a048883920 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b000e0e50 con 0x55a048883920 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3afe000700 1 -- 172.21.15.63:0/3424074289 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3b040058e0 con 0x55a048883920 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3afe000700 1 -- 172.21.15.63:0/3424074289 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b04041050 con 0x55a048883920 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3b25857b80 10 client.15072 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.230+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a048908980 con 0x55a048883920 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3afe000700 1 -- 172.21.15.63:0/3424074289 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b000e0e50 con 0x55a048883920 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3afe000700 1 -- 172.21.15.63:0/3424074289 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3b04021030 con 0x55a048883920 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3b25857b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3b25857b80 10 client.15072 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3b25857b80 20 client.15072 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04890cfc0 0x55a04890bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3b25857b80 4 client.15072 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:32:33.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.231+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 --> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] -- command(tid 0: {"prefix": "status"}) -- 0x55a0487e33b0 con 0x55a04890cfc0 2024-05-11T13:32:33.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.232+0000 7f3b13400700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04890cfc0 0x55a04890bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:33.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b13400700 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04890cfc0 0x55a04890bcd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a622d375d85dbda server_cookie=84262d5f3c430d8e in_seq=0 out_seq=0 2024-05-11T13:32:33.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3afe000700 10 client.15072 ms_handle_connect on v2:172.21.15.71:6834/1316489545 2024-05-11T13:32:33.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3afe000700 1 -- 172.21.15.63:0/3424074289 <== mds.2 v2:172.21.15.71:6834/1316489545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a0487e33b0 con 0x55a04890cfc0 2024-05-11T13:32:33.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3afe000700 10 client.15072 handle_command_reply: tid=0 2024-05-11T13:32:33.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 client.15072 shutdown 2024-05-11T13:32:33.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048883920 msgr2=0x55a04888c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048883920 0x55a04888c1d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3b04025040 tx=0x7f3b040093b0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 shutdown_connections 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a048889550 0x55a0488a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x55a04890cfc0 0x55a04890bcd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a048883920 0x55a04888c1d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 --2- 172.21.15.63:0/3424074289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a048888c30 0x55a04888c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.233+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 >> 172.21.15.63:0/3424074289 conn(0x55a0487b1b10 msgr2=0x55a04888a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.234+0000 7f3afe000700 0 client.15072 ms_handle_reset on v2:172.21.15.71:6834/1316489545 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.234+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 shutdown_connections 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.234+0000 7f3b25857b80 1 -- 172.21.15.63:0/3424074289 wait complete. 2024-05-11T13:32:33.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.234+0000 7f3b25857b80 20 client.15072 trim_cache size 0 max 16384 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 2, 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34427, 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:32:33.239 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 157.507209682, 2024-05-11T13:32:33.240 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:32:33.240 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:32:33.240 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:32:33.240 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 284.03026719000002 2024-05-11T13:32:33.240 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:32:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3af80b2880 msgr2=0x7f3af80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3af80b2880 0x7f3af80b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0060b0 tx=0x7f3b0c007890 comp rx=0 tx=0).stop 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b14106150 msgr2=0x7f3b1419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b14106150 0x7f3b1419c2c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d950 tx=0x7f3b0000dc60 comp rx=0 tx=0).stop 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 shutdown_connections 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f3af80b2880 0x7f3af80b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1410b790 0x7f3b141a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b14105790 0x7f3b1419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 --2- 172.21.15.63:0/1908923453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b14106150 0x7f3b1419c2c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 >> 172.21.15.63:0/1908923453 conn(0x7f3b14101280 msgr2=0x7f3b14101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 shutdown_connections 2024-05-11T13:32:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:33.235+0000 7f3b1ae00700 1 -- 172.21.15.63:0/1908923453 wait complete. 2024-05-11T13:32:34.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-11T13:32:34.584 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:34.976 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:34 smithi063 ceph-mon[29809]: pgmap v497: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:35.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:34 smithi165 ceph-mon[33235]: pgmap v497: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:35.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:34 smithi071 ceph-mon[33293]: pgmap v497: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:36.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 -- 172.21.15.63:0/1920714346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b010fd10 msgr2=0x7f70b0112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 --2- 172.21.15.63:0/1920714346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b010fd10 0x7f70b0112120 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b0a0 tx=0x7f70a800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 -- 172.21.15.63:0/1920714346 shutdown_connections 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 --2- 172.21.15.63:0/1920714346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b010fd10 0x7f70b0112120 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 --2- 172.21.15.63:0/1920714346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70b0075920 0x7f70b0075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 --2- 172.21.15.63:0/1920714346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70b0076240 0x7f70b010f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 -- 172.21.15.63:0/1920714346 >> 172.21.15.63:0/1920714346 conn(0x7f70b00fef60 msgr2=0x7f70b0101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 -- 172.21.15.63:0/1920714346 shutdown_connections 2024-05-11T13:32:36.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.823+0000 7f70b6400700 1 -- 172.21.15.63:0/1920714346 wait complete. 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.824+0000 7f70b6400700 1 Processor -- start 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.824+0000 7f70b6400700 1 -- start start 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b6400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b0075920 0x7f70b01a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b6400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70b0076240 0x7f70b01a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b6400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70b010fd10 0x7f70b019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b6400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0115710 con 0x7f70b0075920 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b6400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0115590 con 0x7f70b0076240 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b6400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0115890 con 0x7f70b010fd10 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b0075920 0x7f70b01a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b0075920 0x7f70b01a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52356/0 (socket says 172.21.15.63:52356) 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 -- 172.21.15.63:0/3598061650 learned_addr learned my addr 172.21.15.63:0/3598061650 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70afe00700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70b0076240 0x7f70b01a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 -- 172.21.15.63:0/3598061650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70b010fd10 msgr2=0x7f70b019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70b010fd10 0x7f70b019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 -- 172.21.15.63:0/3598061650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70b0076240 msgr2=0x7f70b01a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70b0076240 0x7f70b01a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.825+0000 7f70b5000700 1 -- 172.21.15.63:0/3598061650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b019c190 con 0x7f70b0075920 2024-05-11T13:32:36.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.826+0000 7f70b5000700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b0075920 0x7f70b01a1820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c9b0 tx=0x7f70a400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:36.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.826+0000 7f70ad600700 1 -- 172.21.15.63:0/3598061650 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4007b00 con 0x7f70b0075920 2024-05-11T13:32:36.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.826+0000 7f70ad600700 1 -- 172.21.15.63:0/3598061650 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70a4007ca0 con 0x7f70b0075920 2024-05-11T13:32:36.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.826+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b019c480 con 0x7f70b0075920 2024-05-11T13:32:36.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.826+0000 7f70ad600700 1 -- 172.21.15.63:0/3598061650 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4018660 con 0x7f70b0075920 2024-05-11T13:32:36.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.826+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b01a86f0 con 0x7f70b0075920 2024-05-11T13:32:36.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.830+0000 7f70ad600700 1 -- 172.21.15.63:0/3598061650 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f70a400f450 con 0x7f70b0075920 2024-05-11T13:32:36.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.831+0000 7f70c23fdb80 1 Processor -- start 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.831+0000 7f70c23fdb80 1 -- start start 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c8de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70ad600700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f70980b2880 0x7f70980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c88b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555fd2bda9d0 con 0x555fd2c886c0 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fd2b78710 con 0x555fd2c8e3c0 2024-05-11T13:32:36.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555fd2bd6990 con 0x555fd2c8daa0 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70ad600700 1 -- 172.21.15.63:0/3598061650 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f70a40e26d0 con 0x7f70b0075920 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70afe00700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f70980b2880 0x7f70980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70b5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c88b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70b5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c88b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52366/0 (socket says 172.21.15.63:52366) 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70b5a00700 1 -- 172.21.15.63:0/669368592 learned_addr learned my addr 172.21.15.63:0/669368592 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70b5000700 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.832+0000 7f70b5a00700 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c8de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70b5a00700 1 -- 172.21.15.63:0/669368592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 msgr2=0x555fd2c8de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70b5a00700 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c8de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70b5a00700 1 -- 172.21.15.63:0/669368592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 msgr2=0x555fd2c880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70b5a00700 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70b5a00700 1 -- 172.21.15.63:0/669368592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd2b27880 con 0x555fd2c886c0 2024-05-11T13:32:36.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70b5000700 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:36.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70afe00700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f70980b2880 0x7f70980b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70b0110fc0 tx=0x7f709c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:36.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70b5a00700 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c88b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70a8015000 tx=0x7f70a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2300590b4bad12a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:36.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70a2000700 1 -- 172.21.15.63:0/669368592 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8025070 con 0x555fd2c886c0 2024-05-11T13:32:36.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70a2000700 1 -- 172.21.15.63:0/669368592 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70a80074e0 con 0x555fd2c886c0 2024-05-11T13:32:36.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.833+0000 7f70a2000700 1 -- 172.21.15.63:0/669368592 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80052a0 con 0x555fd2c886c0 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 -- 172.21.15.63:0/669368592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 msgr2=0x555fd2c88b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c88b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70a8015000 tx=0x7f70a8009780 comp rx=0 tx=0).stop 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 -- 172.21.15.63:0/669368592 shutdown_connections 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c88b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c8de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/669368592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 -- 172.21.15.63:0/669368592 >> 172.21.15.63:0/669368592 conn(0x555fd2bb6b10 msgr2=0x555fd2c82a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 -- 172.21.15.63:0/669368592 shutdown_connections 2024-05-11T13:32:36.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.834+0000 7f70c23fdb80 1 -- 172.21.15.63:0/669368592 wait complete. 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.835+0000 7f70c23fdb80 1 -- start start 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.835+0000 7f70c23fdb80 1 Processor -- start 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c886c0 0x555fd2c940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c8daa0 0x555fd2c91450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c91990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555fd2c94e10 con 0x555fd2c8daa0 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fd2b40ad0 con 0x555fd2c886c0 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555fd2c94f90 con 0x555fd2c8e3c0 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70c23fdb80 10 client.0 _ll_register_callbacks cb 0x555fd2b81c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70afe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c8daa0 0x555fd2c91450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70afe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c8daa0 0x555fd2c91450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52376/0 (socket says 172.21.15.63:52376) 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70afe00700 1 -- 172.21.15.63:0/2145723081 learned_addr learned my addr 172.21.15.63:0/2145723081 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:36.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.836+0000 7f70b5000700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c886c0 0x555fd2c940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70b5a00700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c91990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70afe00700 1 -- 172.21.15.63:0/2145723081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 msgr2=0x555fd2c91990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70afe00700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c91990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70afe00700 1 -- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c886c0 msgr2=0x555fd2c940b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70afe00700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c886c0 0x555fd2c940b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70afe00700 1 -- 172.21.15.63:0/2145723081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd2ba03d0 con 0x555fd2c8daa0 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70b5000700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c886c0 0x555fd2c940b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70afe00700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c8daa0 0x555fd2c91450 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b990 tx=0x7f709c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3c97b41f285a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:36.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f7096a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f7096a00700 1 -- 172.21.15.63:0/2145723081 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c022070 con 0x555fd2c8daa0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f7096a00700 1 -- 172.21.15.63:0/2145723081 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f709c011de0 con 0x555fd2c8daa0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.837+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a80074e0 con 0x555fd2c8daa0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.838+0000 7f7096a00700 1 -- 172.21.15.63:0/2145723081 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f709c02e050 con 0x555fd2c8daa0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.838+0000 7f70c23fdb80 10 client.15084 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.838+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fd2c9f670 con 0x555fd2c8daa0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.838+0000 7f7096a00700 1 -- 172.21.15.63:0/2145723081 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70a80074e0 con 0x555fd2c8daa0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.838+0000 7f7096a00700 1 -- 172.21.15.63:0/2145723081 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f709c026520 con 0x555fd2c8daa0 2024-05-11T13:32:36.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.838+0000 7f70c23fdb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.839+0000 7f70c23fdb80 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.839+0000 7f70c23fdb80 20 client.15084 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.839+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2ca1ce0 0x555fd2ca3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.839+0000 7f70c23fdb80 4 client.15084 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.839+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 --> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fd2c55160 con 0x555fd2ca1ce0 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.839+0000 7f70b5000700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2ca1ce0 0x555fd2ca3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.841+0000 7f70b5000700 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2ca1ce0 0x555fd2ca3150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9f7a150b9258b2 server_cookie=d35f6f9d6fb74555 in_seq=0 out_seq=0 2024-05-11T13:32:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.841+0000 7f7096a00700 10 client.15084 ms_handle_connect on v2:172.21.15.71:6836/136473158 2024-05-11T13:32:36.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.843+0000 7f7096a00700 1 -- 172.21.15.63:0/2145723081 <== mds.1 v2:172.21.15.71:6836/136473158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555fd2c55160 con 0x555fd2ca1ce0 2024-05-11T13:32:36.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.843+0000 7f7096a00700 10 client.15084 handle_command_reply: tid=0 2024-05-11T13:32:36.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.843+0000 7f70c23fdb80 1 client.15084 shutdown 2024-05-11T13:32:36.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.843+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c8daa0 msgr2=0x555fd2c91450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.843+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c8daa0 0x555fd2c91450 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b990 tx=0x7f709c004750 comp rx=0 tx=0).stop 2024-05-11T13:32:36.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 shutdown_connections 2024-05-11T13:32:36.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c91990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2ca1ce0 0x555fd2ca3150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c886c0 0x555fd2c940b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/2145723081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c8daa0 0x555fd2c91450 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f7096a00700 0 client.15084 ms_handle_reset on v2:172.21.15.71:6836/136473158 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 >> 172.21.15.63:0/2145723081 conn(0x555fd2bb6b10 msgr2=0x555fd2c7d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 shutdown_connections 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 1 -- 172.21.15.63:0/2145723081 wait complete. 2024-05-11T13:32:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.844+0000 7f70c23fdb80 20 client.15084 trim_cache size 0 max 16384 2024-05-11T13:32:36.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 Processor -- start 2024-05-11T13:32:36.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 -- start start 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8daa0 0x555fd2c8fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555fd2ca1490 con 0x555fd2c886c0 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fd2ca1b60 con 0x555fd2c8daa0 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.868+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555fd2ca2230 con 0x555fd2c8e3c0 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c880a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52386/0 (socket says 172.21.15.63:52386) 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 -- 172.21.15.63:0/916315052 learned_addr learned my addr 172.21.15.63:0/916315052 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70afe00700 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8daa0 0x555fd2c8fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5a00700 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 -- 172.21.15.63:0/916315052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 msgr2=0x555fd2c902c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c902c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 -- 172.21.15.63:0/916315052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8daa0 msgr2=0x555fd2c8fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8daa0 0x555fd2c8fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.869+0000 7f70b5000700 1 -- 172.21.15.63:0/916315052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd2c7afd0 con 0x555fd2c886c0 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.870+0000 7f70b5000700 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c880a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f70a40a5960 tx=0x7f70a40e2950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0a4d420083cb8d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.870+0000 7f70a1600700 1 -- 172.21.15.63:0/916315052 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a40adc90 con 0x555fd2c886c0 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.870+0000 7f70a1600700 1 -- 172.21.15.63:0/916315052 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70a408fe50 con 0x555fd2c886c0 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.870+0000 7f70a1600700 1 -- 172.21.15.63:0/916315052 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a40a5b70 con 0x555fd2c886c0 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 -- 172.21.15.63:0/916315052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 msgr2=0x555fd2c880a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c880a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f70a40a5960 tx=0x7f70a40e2950 comp rx=0 tx=0).stop 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 -- 172.21.15.63:0/916315052 shutdown_connections 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8e3c0 0x555fd2c902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c880a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/916315052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8daa0 0x555fd2c8fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 -- 172.21.15.63:0/916315052 >> 172.21.15.63:0/916315052 conn(0x555fd2bb6b10 msgr2=0x555fd2c8b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:36.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 -- 172.21.15.63:0/916315052 shutdown_connections 2024-05-11T13:32:36.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.871+0000 7f70c23fdb80 1 -- 172.21.15.63:0/916315052 wait complete. 2024-05-11T13:32:36.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 -- start start 2024-05-11T13:32:36.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 Processor -- start 2024-05-11T13:32:36.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c87110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c87650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c87b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555fd2ca2fd0 con 0x555fd2c886c0 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fd2ca2900 con 0x555fd2c8e3c0 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555fd2ca36a0 con 0x555fd2c8daa0 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.872+0000 7f70c23fdb80 10 client.0 _ll_register_callbacks cb 0x555fd2b81c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c87110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c87110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52390/0 (socket says 172.21.15.63:52390) 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 -- 172.21.15.63:0/1261449048 learned_addr learned my addr 172.21.15.63:0/1261449048 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70afe00700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c87650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 -- 172.21.15.63:0/1261449048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 msgr2=0x555fd2c87650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c87650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5a00700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c87b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 -- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 msgr2=0x555fd2c87b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c87b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5000700 1 -- 172.21.15.63:0/1261449048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd2caa880 con 0x555fd2c886c0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.873+0000 7f70b5a00700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c87b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f70b5000700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c87110 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f70a40a6b50 tx=0x7f70a40addc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0e06f9ffa36373 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f7096a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f7096a00700 1 -- 172.21.15.63:0/1261449048 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a40c2020 con 0x555fd2c886c0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f7096a00700 1 -- 172.21.15.63:0/1261449048 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f70a40cc0b0 con 0x555fd2c886c0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a408fe50 con 0x555fd2c886c0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f7096a00700 1 -- 172.21.15.63:0/1261449048 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70a40a5b70 con 0x555fd2c886c0 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f70c23fdb80 10 client.15090 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:36.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fd2d0d3b0 con 0x555fd2c886c0 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.874+0000 7f7096a00700 1 -- 172.21.15.63:0/1261449048 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70a408fe50 con 0x555fd2c886c0 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f7096a00700 1 -- 172.21.15.63:0/1261449048 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70a40e2040 con 0x555fd2c886c0 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f70c23fdb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f70c23fdb80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f70c23fdb80 20 client.15090 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2d11a60 0x555fd2d10770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f70c23fdb80 4 client.15090 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 --> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] -- command(tid 0: {"prefix": "status"}) -- 0x555fd2be83b0 con 0x555fd2d11a60 2024-05-11T13:32:36.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.875+0000 7f70afe00700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2d11a60 0x555fd2d10770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:36.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.876+0000 7f70afe00700 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2d11a60 0x555fd2d10770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8614ab7955e49dc server_cookie=e7c732526e58833b in_seq=0 out_seq=0 2024-05-11T13:32:36.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.876+0000 7f7096a00700 10 client.15090 ms_handle_connect on v2:172.21.15.71:6836/136473158 2024-05-11T13:32:36.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f7096a00700 1 -- 172.21.15.63:0/1261449048 <== mds.1 v2:172.21.15.71:6836/136473158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555fd2be83b0 con 0x555fd2d11a60 2024-05-11T13:32:36.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f7096a00700 10 client.15090 handle_command_reply: tid=0 2024-05-11T13:32:36.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 client.15090 shutdown 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 msgr2=0x555fd2c87110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c87110 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f70a40a6b50 tx=0x7f70a40addc0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 shutdown_connections 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x555fd2d11a60 0x555fd2d10770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fd2c8e3c0 0x555fd2c87b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fd2c886c0 0x555fd2c87110 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 --2- 172.21.15.63:0/1261449048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fd2c8daa0 0x555fd2c87650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 >> 172.21.15.63:0/1261449048 conn(0x555fd2bb6b10 msgr2=0x555fd2c8fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f7096a00700 0 client.15090 ms_handle_reset on v2:172.21.15.71:6836/136473158 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.877+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 shutdown_connections 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.878+0000 7f70c23fdb80 1 -- 172.21.15.63:0/1261449048 wait complete. 2024-05-11T13:32:36.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.878+0000 7f70c23fdb80 20 client.15090 trim_cache size 0 max 16384 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 1, 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34451, 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 162.17564056200001, 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 275.43840585700002 2024-05-11T13:32:36.881 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:32:36.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.878+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f70980b2880 msgr2=0x7f70980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.878+0000 7f70b6400700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f70980b2880 0x7f70980b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70b0110fc0 tx=0x7f709c00b590 comp rx=0 tx=0).stop 2024-05-11T13:32:36.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.878+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b0075920 msgr2=0x7f70b01a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:36.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.878+0000 7f70b6400700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b0075920 0x7f70b01a1820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c9b0 tx=0x7f70a400cd70 comp rx=0 tx=0).stop 2024-05-11T13:32:36.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 shutdown_connections 2024-05-11T13:32:36.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f70980b2880 0x7f70980b4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70b010fd10 0x7f70b019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70b0075920 0x7f70b01a1820 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 --2- 172.21.15.63:0/3598061650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70b0076240 0x7f70b01a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:36.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 >> 172.21.15.63:0/3598061650 conn(0x7f70b00fef60 msgr2=0x7f70b00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:36.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 shutdown_connections 2024-05-11T13:32:36.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:36.879+0000 7f70b6400700 1 -- 172.21.15.63:0/3598061650 wait complete. 2024-05-11T13:32:36.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:36 smithi063 ceph-mon[29809]: pgmap v498: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:37.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:36 smithi165 ceph-mon[33235]: pgmap v498: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:37.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:36 smithi071 ceph-mon[33293]: pgmap v498: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:37.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-11T13:32:38.369 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:38.884 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:38 smithi063 ceph-mon[29809]: pgmap v499: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:39.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:38 smithi165 ceph-mon[33235]: pgmap v499: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:39.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:38 smithi071 ceph-mon[33293]: pgmap v499: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:40.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 -- 172.21.15.63:0/2754674598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 msgr2=0x7f73ac10fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 --2- 172.21.15.63:0/2754674598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac10fa10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7398009990 tx=0x7f7398009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 -- 172.21.15.63:0/2754674598 shutdown_connections 2024-05-11T13:32:40.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 --2- 172.21.15.63:0/2754674598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac10ff50 0x7f73ac112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 --2- 172.21.15.63:0/2754674598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73ac075780 0x7f73ac075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 --2- 172.21.15.63:0/2754674598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac10fa10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 -- 172.21.15.63:0/2754674598 >> 172.21.15.63:0/2754674598 conn(0x7f73ac07a6b0 msgr2=0x7f73ac07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:40.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.605+0000 7f73b2400700 1 -- 172.21.15.63:0/2754674598 shutdown_connections 2024-05-11T13:32:40.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.606+0000 7f73b2400700 1 -- 172.21.15.63:0/2754674598 wait complete. 2024-05-11T13:32:40.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 Processor -- start 2024-05-11T13:32:40.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 -- start start 2024-05-11T13:32:40.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac075780 0x7f73ac19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73ac10ff50 0x7f73ac1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac115810 con 0x7f73ac0760a0 2024-05-11T13:32:40.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac115690 con 0x7f73ac075780 2024-05-11T13:32:40.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.607+0000 7f73b2400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac115990 con 0x7f73ac10ff50 2024-05-11T13:32:40.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60310/0 (socket says 172.21.15.63:60310) 2024-05-11T13:32:40.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 -- 172.21.15.63:0/1830955499 learned_addr learned my addr 172.21.15.63:0/1830955499 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:40.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73ab400700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac075780 0x7f73ac19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73abe00700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73ac10ff50 0x7f73ac1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 -- 172.21.15.63:0/1830955499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73ac10ff50 msgr2=0x7f73ac1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73ac10ff50 0x7f73ac1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 -- 172.21.15.63:0/1830955499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac075780 msgr2=0x7f73ac19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac075780 0x7f73ac19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.608+0000 7f73aaa00700 1 -- 172.21.15.63:0/1830955499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac1a0d80 con 0x7f73ac0760a0 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.609+0000 7f73aaa00700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac19c300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7398000c00 tx=0x7f73980057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.609+0000 7f73a3e00700 1 -- 172.21.15.63:0/1830955499 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398027070 con 0x7f73ac0760a0 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.609+0000 7f73a3e00700 1 -- 172.21.15.63:0/1830955499 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f739800fb00 con 0x7f73ac0760a0 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.609+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac1a1010 con 0x7f73ac0760a0 2024-05-11T13:32:40.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.609+0000 7f73a3e00700 1 -- 172.21.15.63:0/1830955499 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398022b50 con 0x7f73ac0760a0 2024-05-11T13:32:40.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.610+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1a88f0 con 0x7f73ac0760a0 2024-05-11T13:32:40.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.613+0000 7f73a3e00700 1 -- 172.21.15.63:0/1830955499 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f739802a070 con 0x7f73ac0760a0 2024-05-11T13:32:40.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.615+0000 7f73bcd64b80 1 Processor -- start 2024-05-11T13:32:40.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.615+0000 7f73bcd64b80 1 -- start start 2024-05-11T13:32:40.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.615+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560abc30 0x55b6560ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560a6920 0x55b6560a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b655ff89d0 con 0x55b6560abc30 2024-05-11T13:32:40.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b655f96710 con 0x55b6560a6920 2024-05-11T13:32:40.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b655ff4990 con 0x55b6560ac550 2024-05-11T13:32:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73a3e00700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f73900b2880 0x7f73900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73a3e00700 1 -- 172.21.15.63:0/1830955499 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f73980e3590 con 0x7f73ac0760a0 2024-05-11T13:32:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73aaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560a6920 0x55b6560a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73aaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560a6920 0x55b6560a6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:34170/0 (socket says 172.21.15.63:34170) 2024-05-11T13:32:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73aaa00700 1 -- 172.21.15.63:0/1797913342 learned_addr learned my addr 172.21.15.63:0/1797913342 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.616+0000 7f73abe00700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f73900b2880 0x7f73900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.617+0000 7f73abe00700 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560abc30 0x55b6560ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.617+0000 7f73ab400700 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.617+0000 7f73abe00700 1 -- 172.21.15.63:0/1797913342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 msgr2=0x55b6560a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.617+0000 7f73abe00700 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.617+0000 7f73abe00700 1 -- 172.21.15.63:0/1797913342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560a6920 msgr2=0x55b6560a6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.617+0000 7f73abe00700 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560a6920 0x55b6560a6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.617+0000 7f73abe00700 1 -- 172.21.15.63:0/1797913342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b655f45880 con 0x55b6560abc30 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.618+0000 7f73ab400700 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.618+0000 7f73abe00700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f73900b2880 0x7f73900b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73a40060b0 tx=0x7f73a4009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.618+0000 7f73abe00700 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560abc30 0x55b6560ac010 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007fd0 tx=0x7f73a401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7ac2a9eb19a515 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.618+0000 7f73a0c00700 1 -- 172.21.15.63:0/1797913342 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4004bb0 con 0x55b6560abc30 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.618+0000 7f73a0c00700 1 -- 172.21.15.63:0/1797913342 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73a401e070 con 0x55b6560abc30 2024-05-11T13:32:40.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.618+0000 7f73a0c00700 1 -- 172.21.15.63:0/1797913342 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a401ad90 con 0x55b6560abc30 2024-05-11T13:32:40.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 -- 172.21.15.63:0/1797913342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560abc30 msgr2=0x55b6560ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560abc30 0x55b6560ac010 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007fd0 tx=0x7f73a401b000 comp rx=0 tx=0).stop 2024-05-11T13:32:40.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 -- 172.21.15.63:0/1797913342 shutdown_connections 2024-05-11T13:32:40.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560a6920 0x55b6560a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560abc30 0x55b6560ac010 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/1797913342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 -- 172.21.15.63:0/1797913342 >> 172.21.15.63:0/1797913342 conn(0x55b655fd4b10 msgr2=0x55b6560a0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:40.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.619+0000 7f73bcd64b80 1 -- 172.21.15.63:0/1797913342 shutdown_connections 2024-05-11T13:32:40.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.620+0000 7f73bcd64b80 1 -- 172.21.15.63:0/1797913342 wait complete. 2024-05-11T13:32:40.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.620+0000 7f73bcd64b80 1 -- start start 2024-05-11T13:32:40.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.621+0000 7f73bcd64b80 1 Processor -- start 2024-05-11T13:32:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.621+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560abc30 0x55b6560af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560ac550 0x55b6560afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b6560b2f60 con 0x55b6560a6920 2024-05-11T13:32:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b655f5ead0 con 0x55b6560ac550 2024-05-11T13:32:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6560b30e0 con 0x55b6560abc30 2024-05-11T13:32:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73bcd64b80 10 client.0 _ll_register_callbacks cb 0x55b655f8c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60342/0 (socket says 172.21.15.63:60342) 2024-05-11T13:32:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73ab400700 1 -- 172.21.15.63:0/3279265596 learned_addr learned my addr 172.21.15.63:0/3279265596 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73aaa00700 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560abc30 0x55b6560af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.622+0000 7f73abe00700 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560ac550 0x55b6560afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f73ab400700 1 -- 172.21.15.63:0/3279265596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560abc30 msgr2=0x55b6560af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f73ab400700 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560abc30 0x55b6560af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f73ab400700 1 -- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560ac550 msgr2=0x55b6560afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f73ab400700 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560ac550 0x55b6560afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f73ab400700 1 -- 172.21.15.63:0/3279265596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b655fbe3d0 con 0x55b6560a6920 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f73ab400700 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560b2390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f739c00b9c0 tx=0x7f739c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99dec9c57bcf444 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f7395600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f7395600700 1 -- 172.21.15.63:0/3279265596 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c01b470 con 0x55b6560a6920 2024-05-11T13:32:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f7395600700 1 -- 172.21.15.63:0/3279265596 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f739c012450 con 0x55b6560a6920 2024-05-11T13:32:40.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.623+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a401e070 con 0x55b6560a6920 2024-05-11T13:32:40.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.624+0000 7f7395600700 1 -- 172.21.15.63:0/3279265596 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f739c012650 con 0x55b6560a6920 2024-05-11T13:32:40.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.624+0000 7f73bcd64b80 10 client.15102 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:40.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.624+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6560bd8b0 con 0x55b6560a6920 2024-05-11T13:32:40.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.624+0000 7f7395600700 1 -- 172.21.15.63:0/3279265596 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73a401e070 con 0x55b6560a6920 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.624+0000 7f7395600700 1 -- 172.21.15.63:0/3279265596 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f739c01ba30 con 0x55b6560a6920 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.624+0000 7f73bcd64b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.625+0000 7f73bcd64b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.625+0000 7f73bcd64b80 20 client.15102 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.625+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b6560bfea0 0x55b6560c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.625+0000 7f73bcd64b80 4 client.15102 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.625+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b656073010 con 0x55b6560bfea0 2024-05-11T13:32:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.625+0000 7f73aaa00700 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b6560bfea0 0x55b6560c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.626+0000 7f73aaa00700 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b6560bfea0 0x55b6560c1310 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29c12627e8982f7 server_cookie=5c34d41a2fd04457 in_seq=0 out_seq=0 2024-05-11T13:32:40.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.626+0000 7f7395600700 10 client.15102 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:32:40.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.628+0000 7f7395600700 1 -- 172.21.15.63:0/3279265596 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b656073010 con 0x55b6560bfea0 2024-05-11T13:32:40.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.628+0000 7f7395600700 10 client.15102 handle_command_reply: tid=0 2024-05-11T13:32:40.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.628+0000 7f73bcd64b80 1 client.15102 shutdown 2024-05-11T13:32:40.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.628+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 msgr2=0x55b6560b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.628+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560b2390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f739c00b9c0 tx=0x7f739c00bd80 comp rx=0 tx=0).stop 2024-05-11T13:32:40.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 shutdown_connections 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b6560bfea0 0x55b6560c1310 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560ac550 0x55b6560afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560b2390 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/3279265596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560abc30 0x55b6560af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 >> 172.21.15.63:0/3279265596 conn(0x55b655fd4b10 msgr2=0x55b65609b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f7395600700 0 client.15102 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 shutdown_connections 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 1 -- 172.21.15.63:0/3279265596 wait complete. 2024-05-11T13:32:40.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.629+0000 7f73bcd64b80 20 client.15102 trim_cache size 0 max 16384 2024-05-11T13:32:40.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 Processor -- start 2024-05-11T13:32:40.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 -- start start 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b6560bf650 con 0x55b6560a6920 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b6560bfd20 con 0x55b6560abc30 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.653+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6560c03f0 con 0x55b6560ac550 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60344/0 (socket says 172.21.15.63:60344) 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 -- 172.21.15.63:0/2445946322 learned_addr learned my addr 172.21.15.63:0/2445946322 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:40.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73abe00700 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73aaa00700 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 -- 172.21.15.63:0/2445946322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 msgr2=0x55b6560a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 -- 172.21.15.63:0/2445946322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 msgr2=0x55b6560ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.654+0000 7f73ab400700 1 -- 172.21.15.63:0/2445946322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6560bc870 con 0x55b6560a6920 2024-05-11T13:32:40.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.655+0000 7f73ab400700 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560a6300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f739c01be20 tx=0x7f739c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc4be0cfc235ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:40.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.655+0000 7f7397e00700 1 -- 172.21.15.63:0/2445946322 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c003f20 con 0x55b6560a6920 2024-05-11T13:32:40.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.655+0000 7f7397e00700 1 -- 172.21.15.63:0/2445946322 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f739c016070 con 0x55b6560a6920 2024-05-11T13:32:40.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.655+0000 7f7397e00700 1 -- 172.21.15.63:0/2445946322 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c005490 con 0x55b6560a6920 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 -- 172.21.15.63:0/2445946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 msgr2=0x55b6560a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560a6300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f739c01be20 tx=0x7f739c005ce0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 -- 172.21.15.63:0/2445946322 shutdown_connections 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560ac550 0x55b6560a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560a6920 0x55b6560a6300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/2445946322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 -- 172.21.15.63:0/2445946322 >> 172.21.15.63:0/2445946322 conn(0x55b655fd4b10 msgr2=0x55b6560ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 -- 172.21.15.63:0/2445946322 shutdown_connections 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.656+0000 7f73bcd64b80 1 -- 172.21.15.63:0/2445946322 wait complete. 2024-05-11T13:32:40.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.657+0000 7f73bcd64b80 1 -- start start 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.657+0000 7f73bcd64b80 1 Processor -- start 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560a6920 0x55b6560ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73bcd64b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560ac550 0x55b6560c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b6560c1190 con 0x55b6560ac550 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b6560c0ac0 con 0x55b6560abc30 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73bcd64b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6560c1860 con 0x55b6560a6920 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73bcd64b80 10 client.0 _ll_register_callbacks cb 0x55b655f8c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73abe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560ac550 0x55b6560c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73abe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560ac550 0x55b6560c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60346/0 (socket says 172.21.15.63:60346) 2024-05-11T13:32:40.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73abe00700 1 -- 172.21.15.63:0/323224785 learned_addr learned my addr 172.21.15.63:0/323224785 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:40.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73aaa00700 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.658+0000 7f73ab400700 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560a6920 0x55b6560ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f73abe00700 1 -- 172.21.15.63:0/323224785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560a6920 msgr2=0x55b6560ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f73abe00700 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560a6920 0x55b6560ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f73abe00700 1 -- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 msgr2=0x55b6560cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:32:40.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f73abe00700 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f73abe00700 1 -- 172.21.15.63:0/323224785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6560c5cb0 con 0x55b6560ac550 2024-05-11T13:32:40.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f73abe00700 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560ac550 0x55b6560c5740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009ec0 tx=0x7f73a4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b18099641db36dc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:40.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f7395600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.659+0000 7f7395600700 1 -- 172.21.15.63:0/323224785 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4031070 con 0x55b6560ac550 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739c016070 con 0x55b6560ac550 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f7395600700 1 -- 172.21.15.63:0/323224785 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f73a401a8e0 con 0x55b6560ac550 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f7395600700 1 -- 172.21.15.63:0/323224785 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73a403c050 con 0x55b6560ac550 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f73bcd64b80 10 client.15108 fetch_fsmap learned FSMap version 28 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b65612bb70 con 0x55b6560ac550 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f7395600700 1 -- 172.21.15.63:0/323224785 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f739c016070 con 0x55b6560ac550 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f7395600700 1 -- 172.21.15.63:0/323224785 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73a4022c30 con 0x55b6560ac550 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.660+0000 7f73bcd64b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.661+0000 7f73bcd64b80 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.661+0000 7f73bcd64b80 20 client.15108 populate_metadata read hostname 'smithi063' 2024-05-11T13:32:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.661+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b656130420 0x55b65612f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:40.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.661+0000 7f73bcd64b80 4 client.15108 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:32:40.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.661+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "status"}) -- 0x55b6560063b0 con 0x55b656130420 2024-05-11T13:32:40.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.661+0000 7f73ab400700 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b656130420 0x55b65612f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.662+0000 7f73ab400700 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b656130420 0x55b65612f130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710fe4ebd0beac70 server_cookie=fd5661416ec0776e in_seq=0 out_seq=0 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.662+0000 7f7395600700 10 client.15108 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f7395600700 1 -- 172.21.15.63:0/323224785 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b6560063b0 con 0x55b656130420 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f7395600700 10 client.15108 handle_command_reply: tid=0 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 client.15108 shutdown 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560ac550 msgr2=0x55b6560c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560ac550 0x55b6560c5740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009ec0 tx=0x7f73a4004db0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 shutdown_connections 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b6560ac550 0x55b6560c5740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6560a6920 0x55b6560ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b656130420 0x55b65612f130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 --2- 172.21.15.63:0/323224785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6560abc30 0x55b6560cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 >> 172.21.15.63:0/323224785 conn(0x55b655fd4b10 msgr2=0x55b65609b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f7395600700 0 client.15108 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 shutdown_connections 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.663+0000 7f73bcd64b80 1 -- 172.21.15.63:0/323224785 wait complete. 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.664+0000 7f73bcd64b80 20 client.15108 trim_cache size 0 max 16384 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:32:40.665 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 0, 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34472, 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 242.44162384500001, 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 261.61960036300002 2024-05-11T13:32:40.666 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.664+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f73900b2880 msgr2=0x7f73900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.664+0000 7f73b2400700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f73900b2880 0x7f73900b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73a40060b0 tx=0x7f73a4009c10 comp rx=0 tx=0).stop 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.664+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 msgr2=0x7f73ac19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.664+0000 7f73b2400700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac19c300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7398000c00 tx=0x7f73980057a0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.664+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 shutdown_connections 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.664+0000 7f73b2400700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f73900b2880 0x7f73900b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.665+0000 7f73b2400700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73ac10ff50 0x7f73ac1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.665+0000 7f73b2400700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac075780 0x7f73ac19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.665+0000 7f73b2400700 1 --2- 172.21.15.63:0/1830955499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac0760a0 0x7f73ac19c300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.665+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 >> 172.21.15.63:0/1830955499 conn(0x7f73ac07a6b0 msgr2=0x7f73ac100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.665+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 shutdown_connections 2024-05-11T13:32:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:40.665+0000 7f73b2400700 1 -- 172.21.15.63:0/1830955499 wait complete. 2024-05-11T13:32:40.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:40 smithi063 ceph-mon[29809]: pgmap v500: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:41.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:40 smithi165 ceph-mon[33235]: pgmap v500: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:41.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:40 smithi071 ceph-mon[33293]: pgmap v500: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:41.570 INFO:tasks.cephadm:Setup complete, yielding 2024-05-11T13:32:41.570 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-11T13:32:41.581 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:32:41.582 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-11T13:32:42.014 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:42.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:42 smithi063 ceph-mon[29809]: pgmap v501: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:43.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:42 smithi165 ceph-mon[33235]: pgmap v501: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:43.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:42 smithi071 ceph-mon[33293]: pgmap v501: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 -- 172.21.15.63:0/2604483209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60077580 msgr2=0x7fcf600759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 --2- 172.21.15.63:0/2604483209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60077580 0x7fcf600759e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009990 tx=0x7fcf54009ca0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 -- 172.21.15.63:0/2604483209 shutdown_connections 2024-05-11T13:32:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 --2- 172.21.15.63:0/2604483209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf6010b490 0x7fcf6010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 --2- 172.21.15.63:0/2604483209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60077580 0x7fcf600759e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 --2- 172.21.15.63:0/2604483209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf60075fb0 0x7fcf60076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 -- 172.21.15.63:0/2604483209 >> 172.21.15.63:0/2604483209 conn(0x7fcf600feb20 msgr2=0x7fcf60100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 -- 172.21.15.63:0/2604483209 shutdown_connections 2024-05-11T13:32:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.465+0000 7fcf68200700 1 -- 172.21.15.63:0/2604483209 wait complete. 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.466+0000 7fcf68200700 1 Processor -- start 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.466+0000 7fcf68200700 1 -- start start 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf68200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60075fb0 0x7fcf6019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf68200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60077580 0x7fcf6019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf68200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf6010b490 0x7fcf601a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf68200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60111150 con 0x7fcf60075fb0 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf68200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60110fd0 con 0x7fcf60077580 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf68200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601112d0 con 0x7fcf6010b490 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf65800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60075fb0 0x7fcf6019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf65800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60075fb0 0x7fcf6019b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60362/0 (socket says 172.21.15.63:60362) 2024-05-11T13:32:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.467+0000 7fcf65800700 1 -- 172.21.15.63:0/3719325387 learned_addr learned my addr 172.21.15.63:0/3719325387 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf66200700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf6010b490 0x7fcf601a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf64e00700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60077580 0x7fcf6019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf65800700 1 -- 172.21.15.63:0/3719325387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf6010b490 msgr2=0x7fcf601a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf65800700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf6010b490 0x7fcf601a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf65800700 1 -- 172.21.15.63:0/3719325387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60077580 msgr2=0x7fcf6019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf65800700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60077580 0x7fcf6019bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf65800700 1 -- 172.21.15.63:0/3719325387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf601a3500 con 0x7fcf60075fb0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf64e00700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60077580 0x7fcf6019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf65800700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60075fb0 0x7fcf6019b8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcf540036f0 tx=0x7fcf540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf5e000700 1 -- 172.21.15.63:0/3719325387 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54027070 con 0x7fcf60075fb0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf5e000700 1 -- 172.21.15.63:0/3719325387 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf54004120 con 0x7fcf60075fb0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf5e000700 1 -- 172.21.15.63:0/3719325387 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54022d50 con 0x7fcf60075fb0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.468+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601a3790 con 0x7fcf60075fb0 2024-05-11T13:32:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.469+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a3c40 con 0x7fcf60075fb0 2024-05-11T13:32:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.470+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6006a240 con 0x7fcf60075fb0 2024-05-11T13:32:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.472+0000 7fcf5e000700 1 -- 172.21.15.63:0/3719325387 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcf5402a070 con 0x7fcf60075fb0 2024-05-11T13:32:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.475+0000 7fcf5e000700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcf480b2900 0x7fcf480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.476+0000 7fcf5e000700 1 -- 172.21.15.63:0/3719325387 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcf540e35e0 con 0x7fcf60075fb0 2024-05-11T13:32:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.476+0000 7fcf64e00700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcf480b2900 0x7fcf480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.476+0000 7fcf64e00700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcf480b2900 0x7fcf480b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:44.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.482+0000 7fcf5e000700 1 -- 172.21.15.63:0/3719325387 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf54013070 con 0x7fcf60075fb0 2024-05-11T13:32:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.743+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcf60063990 con 0x7fcf480b2900 2024-05-11T13:32:44.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.746+0000 7fcf5e000700 1 -- 172.21.15.63:0/3719325387 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcf60063990 con 0x7fcf480b2900 2024-05-11T13:32:44.746 INFO:teuthology.orchestra.run.smithi063.stdout:Backend: cephadm 2024-05-11T13:32:44.747 INFO:teuthology.orchestra.run.smithi063.stdout:Available: Yes 2024-05-11T13:32:44.747 INFO:teuthology.orchestra.run.smithi063.stdout:Paused: No 2024-05-11T13:32:44.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.749+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcf480b2900 msgr2=0x7fcf480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:44.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.749+0000 7fcf68200700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcf480b2900 0x7fcf480b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c005f00 comp rx=0 tx=0).stop 2024-05-11T13:32:44.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.749+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60075fb0 msgr2=0x7fcf6019b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:44.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.749+0000 7fcf68200700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60075fb0 0x7fcf6019b8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcf540036f0 tx=0x7fcf540047b0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 shutdown_connections 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fcf480b2900 0x7fcf480b4dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf6010b490 0x7fcf601a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf60075fb0 0x7fcf6019b8a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 --2- 172.21.15.63:0/3719325387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60077580 0x7fcf6019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 >> 172.21.15.63:0/3719325387 conn(0x7fcf600feb20 msgr2=0x7fcf60107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 shutdown_connections 2024-05-11T13:32:44.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:44.750+0000 7fcf68200700 1 -- 172.21.15.63:0/3719325387 wait complete. 2024-05-11T13:32:45.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:44 smithi165 ceph-mon[33235]: pgmap v502: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:45.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:44 smithi063 ceph-mon[29809]: pgmap v502: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:45.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:44 smithi071 ceph-mon[33293]: pgmap v502: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:45.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-11T13:32:45.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:45 smithi063 ceph-mon[29809]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:46.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:45 smithi165 ceph-mon[33235]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:46.078 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:46.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:45 smithi071 ceph-mon[33293]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:46.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:46 smithi063 ceph-mon[29809]: pgmap v503: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:47.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:46 smithi165 ceph-mon[33235]: pgmap v503: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:47.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:46 smithi071 ceph-mon[33293]: pgmap v503: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:48.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 -- 172.21.15.63:0/3838148212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 msgr2=0x7fb4c810ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:48.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 --2- 172.21.15.63:0/3838148212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c810ba10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000b0d0 tx=0x7fb4c000b3e0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 -- 172.21.15.63:0/3838148212 shutdown_connections 2024-05-11T13:32:48.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 --2- 172.21.15.63:0/3838148212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c810ba10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 --2- 172.21.15.63:0/3838148212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c81035c0 0x7fb4c81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 --2- 172.21.15.63:0/3838148212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4c81047c0 0x7fb4c8104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 -- 172.21.15.63:0/3838148212 >> 172.21.15.63:0/3838148212 conn(0x7fb4c80feb20 msgr2=0x7fb4c8100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 -- 172.21.15.63:0/3838148212 shutdown_connections 2024-05-11T13:32:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.489+0000 7fb4cca00700 1 -- 172.21.15.63:0/3838148212 wait complete. 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.490+0000 7fb4cca00700 1 Processor -- start 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4cca00700 1 -- start start 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4cca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4c81035c0 0x7fb4c8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4cca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c81047c0 0x7fb4c8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4cca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4cca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810ef50 con 0x7fb4c8105160 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4cca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810edd0 con 0x7fb4c81035c0 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4cca00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810f0d0 con 0x7fb4c81047c0 2024-05-11T13:32:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.491+0000 7fb4c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60386/0 (socket says 172.21.15.63:60386) 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 -- 172.21.15.63:0/2348777460 learned_addr learned my addr 172.21.15.63:0/2348777460 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c5e00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4c81035c0 0x7fb4c8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c5400700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c81047c0 0x7fb4c8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 -- 172.21.15.63:0/2348777460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c81047c0 msgr2=0x7fb4c8199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c81047c0 0x7fb4c8199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 -- 172.21.15.63:0/2348777460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4c81035c0 msgr2=0x7fb4c8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4c81035c0 0x7fb4c8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 -- 172.21.15.63:0/2348777460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c81a1280 con 0x7fb4c8105160 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c5400700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c81047c0 0x7fb4c8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.492+0000 7fb4c6800700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c81a0be0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00079b0 tx=0x7fb4c00079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.493+0000 7fb4bea00700 1 -- 172.21.15.63:0/2348777460 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c001f070 con 0x7fb4c8105160 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.493+0000 7fb4bea00700 1 -- 172.21.15.63:0/2348777460 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4c0007cb0 con 0x7fb4c8105160 2024-05-11T13:32:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.493+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c81a1510 con 0x7fb4c8105160 2024-05-11T13:32:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.493+0000 7fb4bea00700 1 -- 172.21.15.63:0/2348777460 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c001b690 con 0x7fb4c8105160 2024-05-11T13:32:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.493+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c81a19c0 con 0x7fb4c8105160 2024-05-11T13:32:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.494+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c806a240 con 0x7fb4c8105160 2024-05-11T13:32:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.496+0000 7fb4bea00700 1 -- 172.21.15.63:0/2348777460 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb4c0022070 con 0x7fb4c8105160 2024-05-11T13:32:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.498+0000 7fb4bea00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4ac0b2900 0x7fb4ac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.498+0000 7fb4bea00700 1 -- 172.21.15.63:0/2348777460 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb4c000f020 con 0x7fb4c8105160 2024-05-11T13:32:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.498+0000 7fb4c5e00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4ac0b2900 0x7fb4ac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:48.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.499+0000 7fb4c5e00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4ac0b2900 0x7fb4ac0b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8104620 tx=0x7fb4b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.503+0000 7fb4bea00700 1 -- 172.21.15.63:0/2348777460 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4c0029020 con 0x7fb4c8105160 2024-05-11T13:32:48.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.753+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb4c8063990 con 0x7fb4ac0b2900 2024-05-11T13:32:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.782+0000 7fb4bea00700 1 -- 172.21.15.63:0/2348777460 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fb4c8063990 con 0x7fb4ac0b2900 2024-05-11T13:32:48.784 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-11T13:32:48.784 INFO:teuthology.orchestra.run.smithi063.stdout:mds.a smithi063 running (4m) 2m ago 4m 15.3M - 19.0.0-3601-g30850680 3f6a7beffd02 8e87e1d62b62 2024-05-11T13:32:48.784 INFO:teuthology.orchestra.run.smithi063.stdout:mds.b smithi071 running (4m) 2m ago 4m 25.0M - 19.0.0-3601-g30850680 3f6a7beffd02 29a345b07e8d 2024-05-11T13:32:48.784 INFO:teuthology.orchestra.run.smithi063.stdout:mds.c smithi165 running (4m) 2m ago 4m 16.5M - 19.0.0-3601-g30850680 3f6a7beffd02 84d4b42d326e 2024-05-11T13:32:48.784 INFO:teuthology.orchestra.run.smithi063.stdout:mds.d smithi063 running (5m) 2m ago 5m 13.1M - 19.0.0-3601-g30850680 3f6a7beffd02 4d1e9d1e3aab 2024-05-11T13:32:48.784 INFO:teuthology.orchestra.run.smithi063.stdout:mds.e smithi071 running (5m) 2m ago 4m 17.0M - 19.0.0-3601-g30850680 3f6a7beffd02 b7299ece7f76 2024-05-11T13:32:48.784 INFO:teuthology.orchestra.run.smithi063.stdout:mds.f smithi165 running (4m) 2m ago 4m 15.5M - 19.0.0-3601-g30850680 3f6a7beffd02 dfa8e6f55615 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mds.g smithi063 running (4m) 2m ago 4m 15.4M - 19.0.0-3601-g30850680 3f6a7beffd02 dda168a2ca79 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mds.h smithi071 running (4m) 2m ago 4m 16.8M - 19.0.0-3601-g30850680 3f6a7beffd02 45d2b6c5d1c8 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mds.i smithi165 running (4m) 2m ago 4m 16.7M - 19.0.0-3601-g30850680 3f6a7beffd02 7dfe9bca856c 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mds.j smithi063 running (5m) 2m ago 5m 13.2M - 19.0.0-3601-g30850680 3f6a7beffd02 bba3825e142c 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mds.k smithi071 running (5m) 2m ago 5m 16.8M - 19.0.0-3601-g30850680 3f6a7beffd02 e4eb76c487ca 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mds.l smithi165 running (5m) 2m ago 5m 15.6M - 19.0.0-3601-g30850680 3f6a7beffd02 3a85cb4d9fd1 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.x smithi063 *:9283,8765 running (15m) 2m ago 15m 533M - 19.0.0-3601-g30850680 3f6a7beffd02 a5aa873b8ad4 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.y smithi071 *:8443,8765 running (12m) 2m ago 12m 467M - 19.0.0-3601-g30850680 3f6a7beffd02 bdd917e054d8 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.z smithi165 *:8443,8765 running (12m) 2m ago 12m 467M - 19.0.0-3601-g30850680 3f6a7beffd02 2186da7408dd 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mon.a smithi063 running (15m) 2m ago 15m 50.5M 2048M 19.0.0-3601-g30850680 3f6a7beffd02 d455117ba008 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mon.b smithi071 running (12m) 2m ago 12m 40.5M 2048M 19.0.0-3601-g30850680 3f6a7beffd02 48814d2bd8db 2024-05-11T13:32:48.785 INFO:teuthology.orchestra.run.smithi063.stdout:mon.c smithi165 running (13m) 2m ago 13m 40.8M 2048M 19.0.0-3601-g30850680 3f6a7beffd02 a0e327cdd6b0 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (12m) 2m ago 12m 52.8M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 84e49e906380 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi071 running (11m) 2m ago 11m 53.3M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 e5f5311e5f5b 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi165 running (11m) 2m ago 11m 54.2M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 fab6e274f9e7 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi063 running (10m) 2m ago 10m 43.9M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 b3d1adf47e1d 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi071 running (9m) 2m ago 9m 50.2M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 325ebabc81df 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi165 running (9m) 2m ago 9m 44.3M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 9dc946963ef7 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6 smithi063 running (8m) 2m ago 8m 42.0M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 41e473a021a4 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7 smithi071 running (8m) 2m ago 8m 43.6M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 9e0be783d577 2024-05-11T13:32:48.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.8 smithi165 running (7m) 2m ago 7m 51.7M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 1800be810d98 2024-05-11T13:32:48.787 INFO:teuthology.orchestra.run.smithi063.stdout:osd.9 smithi063 running (6m) 2m ago 6m 49.7M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 3402f78966ac 2024-05-11T13:32:48.787 INFO:teuthology.orchestra.run.smithi063.stdout:osd.10 smithi071 running (6m) 2m ago 6m 52.3M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 230106416b19 2024-05-11T13:32:48.787 INFO:teuthology.orchestra.run.smithi063.stdout:osd.11 smithi165 running (5m) 2m ago 5m 48.5M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 d35dc1d0e064 2024-05-11T13:32:48.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4ac0b2900 msgr2=0x7fb4ac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:48.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4ac0b2900 0x7fb4ac0b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8104620 tx=0x7fb4b8009260 comp rx=0 tx=0).stop 2024-05-11T13:32:48.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 msgr2=0x7fb4c81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c81a0be0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00079b0 tx=0x7fb4c00079e0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 shutdown_connections 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fb4ac0b2900 0x7fb4ac0b4dc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c8105160 0x7fb4c81a0be0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4c81035c0 0x7fb4c8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 --2- 172.21.15.63:0/2348777460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c81047c0 0x7fb4c8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 >> 172.21.15.63:0/2348777460 conn(0x7fb4c80feb20 msgr2=0x7fb4c8107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 shutdown_connections 2024-05-11T13:32:48.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:48.786+0000 7fb4cca00700 1 -- 172.21.15.63:0/2348777460 wait complete. 2024-05-11T13:32:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:48 smithi165 ceph-mon[33235]: pgmap v504: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:49.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:48 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:32:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:48 smithi063 ceph-mon[29809]: pgmap v504: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:49.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:48 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:32:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:48 smithi071 ceph-mon[33293]: pgmap v504: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:49.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:48 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:32:49.729 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-11T13:32:49.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:49 smithi063 ceph-mon[29809]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:50.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:49 smithi165 ceph-mon[33235]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:50.159 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:50.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:49 smithi071 ceph-mon[33293]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:50 smithi063 ceph-mon[29809]: pgmap v505: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:32:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:32:50.935 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:50 smithi063 ceph-mon[29809]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:32:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:50 smithi165 ceph-mon[33235]: pgmap v505: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:32:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:32:51.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:50 smithi165 ceph-mon[33235]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:32:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:50 smithi071 ceph-mon[33293]: pgmap v505: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:32:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:32:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:32:51.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:50 smithi071 ceph-mon[33293]: from='mgr.14152 172.21.15.63:0/2843664329' entity='mgr.x' 2024-05-11T13:32:52.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 -- 172.21.15.63:0/3433604348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 msgr2=0x7fd0f8105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 --2- 172.21.15.63:0/3433604348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f8105c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80099c0 tx=0x7fd0e8009cd0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 -- 172.21.15.63:0/3433604348 shutdown_connections 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 --2- 172.21.15.63:0/3433604348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f81073a0 0x7fd0f810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 --2- 172.21.15.63:0/3433604348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f8105c20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 --2- 172.21.15.63:0/3433604348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f8106a00 0x7fd0f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 -- 172.21.15.63:0/3433604348 >> 172.21.15.63:0/3433604348 conn(0x7fd0f8100da0 msgr2=0x7fd0f81031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 -- 172.21.15.63:0/3433604348 shutdown_connections 2024-05-11T13:32:52.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.346+0000 7fd0fec00700 1 -- 172.21.15.63:0/3433604348 wait complete. 2024-05-11T13:32:52.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.347+0000 7fd0fec00700 1 Processor -- start 2024-05-11T13:32:52.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.347+0000 7fd0fec00700 1 -- start start 2024-05-11T13:32:52.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.347+0000 7fd0fec00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:52.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.347+0000 7fd0fec00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8106a00 0x7fd0f819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:52.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0fec00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f81073a0 0x7fd0f81a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:52.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0fec00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8111170 con 0x7fd0f8105800 2024-05-11T13:32:52.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0fec00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8110ff0 con 0x7fd0f8106a00 2024-05-11T13:32:52.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0fec00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81112f0 con 0x7fd0f81073a0 2024-05-11T13:32:52.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:52.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f819b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37498/0 (socket says 172.21.15.63:37498) 2024-05-11T13:32:52.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0f7e00700 1 -- 172.21.15.63:0/797939949 learned_addr learned my addr 172.21.15.63:0/797939949 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:52.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0f7400700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8106a00 0x7fd0f819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.348+0000 7fd0fcc00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f81073a0 0x7fd0f81a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f7e00700 1 -- 172.21.15.63:0/797939949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f81073a0 msgr2=0x7fd0f81a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f7e00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f81073a0 0x7fd0f81a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f7e00700 1 -- 172.21.15.63:0/797939949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8106a00 msgr2=0x7fd0f819bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f7e00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8106a00 0x7fd0f819bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f7e00700 1 -- 172.21.15.63:0/797939949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f81a33c0 con 0x7fd0f8105800 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f7e00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f819b760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800b580 tx=0x7fd0e800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f4c00700 1 -- 172.21.15.63:0/797939949 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8027070 con 0x7fd0f8105800 2024-05-11T13:32:52.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f4c00700 1 -- 172.21.15.63:0/797939949 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e8022470 con 0x7fd0f8105800 2024-05-11T13:32:52.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0f4c00700 1 -- 172.21.15.63:0/797939949 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8003a50 con 0x7fd0f8105800 2024-05-11T13:32:52.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f81a3650 con 0x7fd0f8105800 2024-05-11T13:32:52.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.349+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f81a3a50 con 0x7fd0f8105800 2024-05-11T13:32:52.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.351+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f806a240 con 0x7fd0f8105800 2024-05-11T13:32:52.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.355+0000 7fd0f4c00700 1 -- 172.21.15.63:0/797939949 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd0e8022920 con 0x7fd0f8105800 2024-05-11T13:32:52.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.357+0000 7fd0f4c00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd0dc0b2900 0x7fd0dc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:52.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.357+0000 7fd0f4c00700 1 -- 172.21.15.63:0/797939949 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd0e80e3c00 con 0x7fd0f8105800 2024-05-11T13:32:52.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.357+0000 7fd0f7400700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd0dc0b2900 0x7fd0dc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:52.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.358+0000 7fd0f7400700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd0dc0b2900 0x7fd0dc0b4dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400ac50 tx=0x7fd0e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:52.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.360+0000 7fd0f4c00700 1 -- 172.21.15.63:0/797939949 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0e8030020 con 0x7fd0f8105800 2024-05-11T13:32:52.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.639+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd0f8063990 con 0x7fd0dc0b2900 2024-05-11T13:32:52.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.656+0000 7fd0f4c00700 1 -- 172.21.15.63:0/797939949 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd0f8063990 con 0x7fd0dc0b2900 2024-05-11T13:32:52.656 INFO:teuthology.orchestra.run.smithi063.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-11T13:32:52.656 INFO:teuthology.orchestra.run.smithi063.stdout:mds.all 12/12 2m ago 5m smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-11T13:32:52.656 INFO:teuthology.orchestra.run.smithi063.stdout:mgr 3/3 2m ago 12m smithi063=x;smithi071=y;smithi165=z;count:3 2024-05-11T13:32:52.657 INFO:teuthology.orchestra.run.smithi063.stdout:mon 3/3 2m ago 13m smithi063:172.21.15.63=a;smithi071:172.21.15.71=b;smithi165:172.21.15.165=c;count:3 2024-05-11T13:32:52.657 INFO:teuthology.orchestra.run.smithi063.stdout:osd 12 2m ago - 2024-05-11T13:32:52.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd0dc0b2900 msgr2=0x7fd0dc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:52.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd0dc0b2900 0x7fd0dc0b4dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400ac50 tx=0x7fd0e400a380 comp rx=0 tx=0).stop 2024-05-11T13:32:52.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 msgr2=0x7fd0f819b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:52.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f819b760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800b580 tx=0x7fd0e800f670 comp rx=0 tx=0).stop 2024-05-11T13:32:52.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 shutdown_connections 2024-05-11T13:32:52.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd0dc0b2900 0x7fd0dc0b4dc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f81073a0 0x7fd0f81a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0f8105800 0x7fd0f819b760 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 --2- 172.21.15.63:0/797939949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8106a00 0x7fd0f819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:52.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.659+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 >> 172.21.15.63:0/797939949 conn(0x7fd0f8100da0 msgr2=0x7fd0f8109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:52.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.660+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 shutdown_connections 2024-05-11T13:32:52.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:52.660+0000 7fd0fec00700 1 -- 172.21.15.63:0/797939949 wait complete. 2024-05-11T13:32:52.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:52 smithi063 ceph-mon[29809]: pgmap v506: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:52 smithi165 ceph-mon[33235]: pgmap v506: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:53.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:52 smithi071 ceph-mon[33293]: pgmap v506: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:53.603 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-11T13:32:54.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:53 smithi165 ceph-mon[33235]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:54.081 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:54.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:53 smithi063 ceph-mon[29809]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:54.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:53 smithi071 ceph-mon[33293]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:54.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:54 smithi063 ceph-mon[29809]: pgmap v507: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:55.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:54 smithi165 ceph-mon[33235]: pgmap v507: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:55.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:54 smithi071 ceph-mon[33293]: pgmap v507: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:32:56.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.447+0000 7f1e26200700 1 -- 172.21.15.63:0/1259573398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201051a0 msgr2=0x7f1e2010ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:56.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.447+0000 7f1e26200700 1 --2- 172.21.15.63:0/1259573398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201051a0 0x7f1e2010ba50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b0d0 tx=0x7f1e1800b3e0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.447+0000 7f1e26200700 1 -- 172.21.15.63:0/1259573398 shutdown_connections 2024-05-11T13:32:56.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.447+0000 7f1e26200700 1 --2- 172.21.15.63:0/1259573398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201051a0 0x7f1e2010ba50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.447+0000 7f1e26200700 1 --2- 172.21.15.63:0/1259573398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e20103600 0x7f1e20103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.447+0000 7f1e26200700 1 --2- 172.21.15.63:0/1259573398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e20104800 0x7f1e20104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.447+0000 7f1e26200700 1 -- 172.21.15.63:0/1259573398 >> 172.21.15.63:0/1259573398 conn(0x7f1e200feba0 msgr2=0x7f1e20100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:56.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.448+0000 7f1e26200700 1 -- 172.21.15.63:0/1259573398 shutdown_connections 2024-05-11T13:32:56.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.448+0000 7f1e26200700 1 -- 172.21.15.63:0/1259573398 wait complete. 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 Processor -- start 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 -- start start 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20103600 0x7f1e200772f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e20104800 0x7f1e20075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e201051a0 0x7f1e20075e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010ef90 con 0x7f1e20103600 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010ee10 con 0x7f1e20104800 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.449+0000 7f1e26200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010f110 con 0x7f1e201051a0 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20103600 0x7f1e200772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20103600 0x7f1e200772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37526/0 (socket says 172.21.15.63:37526) 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 -- 172.21.15.63:0/3533916938 learned_addr learned my addr 172.21.15.63:0/3533916938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1ea00700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e20104800 0x7f1e20075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1fe00700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e201051a0 0x7f1e20075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 -- 172.21.15.63:0/3533916938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e201051a0 msgr2=0x7f1e20075e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e201051a0 0x7f1e20075e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 -- 172.21.15.63:0/3533916938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e20104800 msgr2=0x7f1e20075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e20104800 0x7f1e20075940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1f400700 1 -- 172.21.15.63:0/3533916938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e20076730 con 0x7f1e20103600 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1fe00700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e201051a0 0x7f1e20075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.450+0000 7f1e1ea00700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e20104800 0x7f1e20075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:32:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.451+0000 7f1e1f400700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20103600 0x7f1e200772f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000d980 tx=0x7f1e1000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:56.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.451+0000 7f1e17e00700 1 -- 172.21.15.63:0/3533916938 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10017070 con 0x7f1e20103600 2024-05-11T13:32:56.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.451+0000 7f1e17e00700 1 -- 172.21.15.63:0/3533916938 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e10010460 con 0x7f1e20103600 2024-05-11T13:32:56.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.451+0000 7f1e17e00700 1 -- 172.21.15.63:0/3533916938 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000de60 con 0x7f1e20103600 2024-05-11T13:32:56.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.451+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e201a36d0 con 0x7f1e20103600 2024-05-11T13:32:56.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.451+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a3b90 con 0x7f1e20103600 2024-05-11T13:32:56.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.452+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2006a240 con 0x7f1e20103600 2024-05-11T13:32:56.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.455+0000 7f1e17e00700 1 -- 172.21.15.63:0/3533916938 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e100108c0 con 0x7f1e20103600 2024-05-11T13:32:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.456+0000 7f1e17e00700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1e040b2900 0x7f1e040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:32:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.456+0000 7f1e17e00700 1 -- 172.21.15.63:0/3533916938 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1e100e2170 con 0x7f1e20103600 2024-05-11T13:32:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.456+0000 7f1e1ea00700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1e040b2900 0x7f1e040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:32:56.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.457+0000 7f1e1ea00700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1e040b2900 0x7f1e040b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c005fd0 tx=0x7f1e0c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:32:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.459+0000 7f1e17e00700 1 -- 172.21.15.63:0/3533916938 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e100a9b00 con 0x7f1e20103600 2024-05-11T13:32:56.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.708+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1e20063990 con 0x7f1e040b2900 2024-05-11T13:32:56.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.711+0000 7f1e17e00700 1 -- 172.21.15.63:0/3533916938 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1e20063990 con 0x7f1e040b2900 2024-05-11T13:32:56.712 INFO:teuthology.orchestra.run.smithi063.stdout:HOST ADDR LABELS STATUS 2024-05-11T13:32:56.712 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 172.21.15.63 2024-05-11T13:32:56.712 INFO:teuthology.orchestra.run.smithi063.stdout:smithi071 172.21.15.71 2024-05-11T13:32:56.712 INFO:teuthology.orchestra.run.smithi063.stdout:smithi165 172.21.15.165 2024-05-11T13:32:56.712 INFO:teuthology.orchestra.run.smithi063.stdout:3 hosts in cluster 2024-05-11T13:32:56.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.715+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1e040b2900 msgr2=0x7f1e040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:56.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.715+0000 7f1e26200700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1e040b2900 0x7f1e040b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c005fd0 tx=0x7f1e0c005e30 comp rx=0 tx=0).stop 2024-05-11T13:32:56.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.715+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20103600 msgr2=0x7f1e200772f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:32:56.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.715+0000 7f1e26200700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20103600 0x7f1e200772f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000d980 tx=0x7f1e1000dd40 comp rx=0 tx=0).stop 2024-05-11T13:32:56.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.715+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 shutdown_connections 2024-05-11T13:32:56.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.716+0000 7f1e26200700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f1e040b2900 0x7f1e040b4dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.716+0000 7f1e26200700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e201051a0 0x7f1e20075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.716+0000 7f1e26200700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20103600 0x7f1e200772f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.716+0000 7f1e26200700 1 --2- 172.21.15.63:0/3533916938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e20104800 0x7f1e20075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:32:56.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.716+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 >> 172.21.15.63:0/3533916938 conn(0x7f1e200feba0 msgr2=0x7f1e20107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:32:56.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.716+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 shutdown_connections 2024-05-11T13:32:56.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:32:56.716+0000 7f1e26200700 1 -- 172.21.15.63:0/3533916938 wait complete. 2024-05-11T13:32:57.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:56 smithi165 ceph-mon[33235]: pgmap v508: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:32:57.134 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:56 smithi063 ceph-mon[29809]: pgmap v508: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:32:57.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:56 smithi071 ceph-mon[33293]: pgmap v508: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:32:57.535 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-11T13:32:58.032 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:32:58.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:57 smithi063 ceph-mon[29809]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:58.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:57 smithi071 ceph-mon[33293]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:58.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:57 smithi165 ceph-mon[33235]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:32:59.101 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:32:58 smithi063 ceph-mon[29809]: pgmap v509: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:59.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:32:58 smithi071 ceph-mon[33293]: pgmap v509: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:32:59.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:32:58 smithi165 ceph-mon[33235]: pgmap v509: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:00.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 -- 172.21.15.63:0/392338083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01069f0 msgr2=0x7fc8c0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 --2- 172.21.15.63:0/392338083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01069f0 0x7fc8c0106e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0099c0 tx=0x7fc8ac009cd0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 -- 172.21.15.63:0/392338083 shutdown_connections 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 --2- 172.21.15.63:0/392338083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8c0107390 0x7fc8c010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 --2- 172.21.15.63:0/392338083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8c01057f0 0x7fc8c0105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 --2- 172.21.15.63:0/392338083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01069f0 0x7fc8c0106e50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 -- 172.21.15.63:0/392338083 >> 172.21.15.63:0/392338083 conn(0x7fc8c0100db0 msgr2=0x7fc8c01031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 -- 172.21.15.63:0/392338083 shutdown_connections 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.536+0000 7fc8c5000700 1 -- 172.21.15.63:0/392338083 wait complete. 2024-05-11T13:33:00.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.537+0000 7fc8c5000700 1 Processor -- start 2024-05-11T13:33:00.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.537+0000 7fc8c5000700 1 -- start start 2024-05-11T13:33:00.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8c5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01057f0 0x7fc8c019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:00.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8c5000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8c01069f0 0x7fc8c019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:00.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8c5000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8c0107390 0x7fc8c01a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:00.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8c5000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0111180 con 0x7fc8c01057f0 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8c5000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0111000 con 0x7fc8c01069f0 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8c5000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0111300 con 0x7fc8c0107390 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8bde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01057f0 0x7fc8c019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8bde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01057f0 0x7fc8c019b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52976/0 (socket says 172.21.15.63:52976) 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.538+0000 7fc8bde00700 1 -- 172.21.15.63:0/2321354137 learned_addr learned my addr 172.21.15.63:0/2321354137 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8bd400700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8c01069f0 0x7fc8c019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8be800700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8c0107390 0x7fc8c01a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8bde00700 1 -- 172.21.15.63:0/2321354137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8c0107390 msgr2=0x7fc8c01a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8bde00700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8c0107390 0x7fc8c01a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8bde00700 1 -- 172.21.15.63:0/2321354137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8c01069f0 msgr2=0x7fc8c019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8bde00700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8c01069f0 0x7fc8c019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8bde00700 1 -- 172.21.15.63:0/2321354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c01a3480 con 0x7fc8c01057f0 2024-05-11T13:33:00.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8bde00700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01057f0 0x7fc8c019b820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000d9b0 tx=0x7fc8b000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:00.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8b6a00700 1 -- 172.21.15.63:0/2321354137 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0017070 con 0x7fc8c01057f0 2024-05-11T13:33:00.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.539+0000 7fc8b6a00700 1 -- 172.21.15.63:0/2321354137 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b0010460 con 0x7fc8c01057f0 2024-05-11T13:33:00.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.540+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c01a3770 con 0x7fc8c01057f0 2024-05-11T13:33:00.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.540+0000 7fc8b6a00700 1 -- 172.21.15.63:0/2321354137 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000f620 con 0x7fc8c01057f0 2024-05-11T13:33:00.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.540+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c01a3c60 con 0x7fc8c01057f0 2024-05-11T13:33:00.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.542+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c006a240 con 0x7fc8c01057f0 2024-05-11T13:33:00.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.544+0000 7fc8b6a00700 1 -- 172.21.15.63:0/2321354137 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc8b0010610 con 0x7fc8c01057f0 2024-05-11T13:33:00.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.546+0000 7fc8b6a00700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc8a40b2880 0x7fc8a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:00.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.546+0000 7fc8b6a00700 1 -- 172.21.15.63:0/2321354137 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc8b00e21d0 con 0x7fc8c01057f0 2024-05-11T13:33:00.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.546+0000 7fc8bd400700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc8a40b2880 0x7fc8a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:00.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.547+0000 7fc8bd400700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc8a40b2880 0x7fc8a40b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac000c00 tx=0x7fc8ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:00.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.551+0000 7fc8b6a00700 1 -- 172.21.15.63:0/2321354137 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8b00a9ae0 con 0x7fc8c01057f0 2024-05-11T13:33:00.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.804+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 --> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc8c0063990 con 0x7fc8a40b2880 2024-05-11T13:33:00.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.809+0000 7fc8b6a00700 1 -- 172.21.15.63:0/2321354137 <== mgr.14152 v2:172.21.15.63:6800/1742389159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc8c0063990 con 0x7fc8a40b2880 2024-05-11T13:33:00.810 INFO:teuthology.orchestra.run.smithi063.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-11T13:33:00.810 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5344005B400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-11T13:33:00.810 INFO:teuthology.orchestra.run.smithi063.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-11T13:33:00.810 INFO:teuthology.orchestra.run.smithi063.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-11T13:33:00.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc8a40b2880 msgr2=0x7fc8a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:00.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc8a40b2880 0x7fc8a40b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac000c00 tx=0x7fc8ac01a040 comp rx=0 tx=0).stop 2024-05-11T13:33:00.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01057f0 msgr2=0x7fc8c019b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:00.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01057f0 0x7fc8c019b820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000d9b0 tx=0x7fc8b000dd70 comp rx=0 tx=0).stop 2024-05-11T13:33:00.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 shutdown_connections 2024-05-11T13:33:00.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fc8a40b2880 0x7fc8a40b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8c0107390 0x7fc8c01a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c01057f0 0x7fc8c019b820 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 --2- 172.21.15.63:0/2321354137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8c01069f0 0x7fc8c019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:00.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 >> 172.21.15.63:0/2321354137 conn(0x7fc8c0100db0 msgr2=0x7fc8c0109d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:00.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 shutdown_connections 2024-05-11T13:33:00.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:00.813+0000 7fc8c5000700 1 -- 172.21.15.63:0/2321354137 wait complete. 2024-05-11T13:33:01.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:00 smithi063 ceph-mon[29809]: pgmap v510: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:01.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:00 smithi071 ceph-mon[33293]: pgmap v510: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:01.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:00 smithi165 ceph-mon[33235]: pgmap v510: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:01.744 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-11T13:33:01.753 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:33:01.753 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-11T13:33:01.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:01 smithi063 ceph-mon[29809]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:02.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:01 smithi071 ceph-mon[33293]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:02.240 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:02.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:01 smithi165 ceph-mon[33235]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:03.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:02 smithi063 ceph-mon[29809]: pgmap v511: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:03.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:02 smithi071 ceph-mon[33293]: pgmap v511: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:03.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:02 smithi165 ceph-mon[33235]: pgmap v511: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:04.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 -- 172.21.15.63:0/3586176103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 msgr2=0x7ff37010b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:04.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 --2- 172.21.15.63:0/3586176103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff37010b710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00b0a0 tx=0x7ff36c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 -- 172.21.15.63:0/3586176103 shutdown_connections 2024-05-11T13:33:04.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 --2- 172.21.15.63:0/3586176103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff37010b710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 --2- 172.21.15.63:0/3586176103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3701040c0 0x7ff3701064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 --2- 172.21.15.63:0/3586176103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3701069f0 0x7ff370108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 -- 172.21.15.63:0/3586176103 >> 172.21.15.63:0/3586176103 conn(0x7ff3700fda50 msgr2=0x7ff3700ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:04.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 -- 172.21.15.63:0/3586176103 shutdown_connections 2024-05-11T13:33:04.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.491+0000 7ff378800700 1 -- 172.21.15.63:0/3586176103 wait complete. 2024-05-11T13:33:04.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 Processor -- start 2024-05-11T13:33:04.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 -- start start 2024-05-11T13:33:04.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3701040c0 0x7ff3701995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3701069f0 0x7ff370199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff3701a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010eec0 con 0x7ff370109320 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010ed40 con 0x7ff3701040c0 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff378800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010f040 con 0x7ff3701069f0 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.493+0000 7ff376800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff3701a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff376800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff3701a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53000/0 (socket says 172.21.15.63:53000) 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff376800700 1 -- 172.21.15.63:0/2695130644 learned_addr learned my addr 172.21.15.63:0/2695130644 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff375e00700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3701040c0 0x7ff3701995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff375400700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3701069f0 0x7ff370199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff376800700 1 -- 172.21.15.63:0/2695130644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3701069f0 msgr2=0x7ff370199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff376800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3701069f0 0x7ff370199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff376800700 1 -- 172.21.15.63:0/2695130644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3701040c0 msgr2=0x7ff3701995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff376800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3701040c0 0x7ff3701995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.494+0000 7ff376800700 1 -- 172.21.15.63:0/2695130644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3701a1250 con 0x7ff370109320 2024-05-11T13:33:04.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.495+0000 7ff376800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff3701a0bb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006070 tx=0x7ff36c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:04.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.495+0000 7ff36aa00700 1 -- 172.21.15.63:0/2695130644 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c021070 con 0x7ff370109320 2024-05-11T13:33:04.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.495+0000 7ff36aa00700 1 -- 172.21.15.63:0/2695130644 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff36c007d00 con 0x7ff370109320 2024-05-11T13:33:04.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.495+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3701a14e0 con 0x7ff370109320 2024-05-11T13:33:04.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.495+0000 7ff36aa00700 1 -- 172.21.15.63:0/2695130644 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c01b7b0 con 0x7ff370109320 2024-05-11T13:33:04.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.495+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3701a1990 con 0x7ff370109320 2024-05-11T13:33:04.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.496+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37006a240 con 0x7ff370109320 2024-05-11T13:33:04.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.499+0000 7ff36aa00700 1 -- 172.21.15.63:0/2695130644 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff36c024070 con 0x7ff370109320 2024-05-11T13:33:04.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.500+0000 7ff36aa00700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff3580b2900 0x7ff3580b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:04.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.500+0000 7ff36aa00700 1 -- 172.21.15.63:0/2695130644 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff36c00f020 con 0x7ff370109320 2024-05-11T13:33:04.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.501+0000 7ff375e00700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff3580b2900 0x7ff3580b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:04.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.501+0000 7ff375e00700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff3580b2900 0x7ff3580b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff364005950 tx=0x7ff3640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:04.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.505+0000 7ff36aa00700 1 -- 172.21.15.63:0/2695130644 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff36c02b020 con 0x7ff370109320 2024-05-11T13:33:04.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.796+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff370064160 con 0x7ff370109320 2024-05-11T13:33:04.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.797+0000 7ff36aa00700 1 -- 172.21.15.63:0/2695130644 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3094 (secure 0 0 0) 0x7ff36c0ab340 con 0x7ff370109320 2024-05-11T13:33:04.798 INFO:teuthology.orchestra.run.smithi063.stdout:e28 2024-05-11T13:33:04.798 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-11T13:33:04.798 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T13:33:04.798 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-11T13:33:04.798 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:04.798 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 28 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-11T13:28:38.143704+0000 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-11T13:30:02.730662+0000 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 300 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-11T13:33:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 5 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1,2,3,4 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34472,1=34451,2=34427,3=34403,4=24472} 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3,4] 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 0 2024-05-11T13:33:04.800 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 34472 members: 24472,34403,34427,34451,34472 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b{0:34472} state up:active seq 6 addr [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.165:6836/2308228126,v1:172.21.15.165:6837/2308228126] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.h{1:34451} state up:active seq 30 addr [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.f{1:24448} state up:standby-replay seq 1 addr [v2:172.21.15.165:6834/4024086251,v1:172.21.15.165:6835/4024086251] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.e{2:34427} state up:active seq 33 addr [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.l{2:24409} state up:standby-replay seq 1 addr [v2:172.21.15.165:6832/2703259087,v1:172.21.15.165:6833/2703259087] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.k{3:34403} state up:active seq 37 addr [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a{3:14577} state up:standby-replay seq 1 addr [v2:172.21.15.63:6840/3763187067,v1:172.21.15.63:6841/3763187067] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.i{4:24472} state up:active seq 30 addr [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.g{4:14571} state up:standby-replay seq 1 addr [v2:172.21.15.63:6838/3189418422,v1:172.21.15.63:6839/3189418422] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-11T13:33:04.801 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:04.802 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.j{-1:14520} state up:standby seq 1 addr [v2:172.21.15.63:6834/1042825923,v1:172.21.15.63:6835/1042825923] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.802 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d{-1:14532} state up:standby seq 1 addr [v2:172.21.15.63:6836/3240025359,v1:172.21.15.63:6837/3240025359] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T13:33:04.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff3580b2900 msgr2=0x7ff3580b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:04.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff3580b2900 0x7ff3580b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff364005950 tx=0x7ff3640058e0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 msgr2=0x7ff3701a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:04.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff3701a0bb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006070 tx=0x7ff36c00bd70 comp rx=0 tx=0).stop 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 shutdown_connections 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff3580b2900 0x7ff3580b4dc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff370109320 0x7ff3701a0bb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3701040c0 0x7ff3701995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 --2- 172.21.15.63:0/2695130644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3701069f0 0x7ff370199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 >> 172.21.15.63:0/2695130644 conn(0x7ff3700fda50 msgr2=0x7ff370104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 shutdown_connections 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:04.801+0000 7ff378800700 1 -- 172.21.15.63:0/2695130644 wait complete. 2024-05-11T13:33:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-11T13:33:05.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:04 smithi063 ceph-mon[29809]: pgmap v512: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:05.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:04 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2695130644' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-11T13:33:05.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:04 smithi071 ceph-mon[33293]: pgmap v512: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:05.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:04 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2695130644' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-11T13:33:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:04 smithi165 ceph-mon[33235]: pgmap v512: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:05.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:04 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2695130644' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-11T13:33:05.763 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-11T13:33:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:07.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:06 smithi063 ceph-mon[29809]: pgmap v513: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:07.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:06 smithi071 ceph-mon[33293]: pgmap v513: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:07.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:06 smithi165 ceph-mon[33235]: pgmap v513: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 -- 172.21.15.63:0/819772914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 msgr2=0x7f1784106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 --2- 172.21.15.63:0/819772914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f1784106430 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1774009990 tx=0x7f1774009ca0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 -- 172.21.15.63:0/819772914 shutdown_connections 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 --2- 172.21.15.63:0/819772914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17841092a0 0x7f178410b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 --2- 172.21.15.63:0/819772914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f1784106430 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 --2- 172.21.15.63:0/819772914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1784106970 0x7f1784108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 -- 172.21.15.63:0/819772914 >> 172.21.15.63:0/819772914 conn(0x7f17840fda50 msgr2=0x7f17840ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 -- 172.21.15.63:0/819772914 shutdown_connections 2024-05-11T13:33:08.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.546+0000 7f178a400700 1 -- 172.21.15.63:0/819772914 wait complete. 2024-05-11T13:33:08.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 Processor -- start 2024-05-11T13:33:08.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 -- start start 2024-05-11T13:33:08.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f17841039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:08.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1784106970 0x7f1784101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:08.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17841092a0 0x7f1784102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:08.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f178410ef70 con 0x7f1784104040 2024-05-11T13:33:08.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f178410edf0 con 0x7f1784106970 2024-05-11T13:33:08.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.548+0000 7f178a400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f178410f0f0 con 0x7f17841092a0 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f17841039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f17841039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53024/0 (socket says 172.21.15.63:53024) 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 -- 172.21.15.63:0/793347910 learned_addr learned my addr 172.21.15.63:0/793347910 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f177aa00700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1784106970 0x7f1784101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783e00700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17841092a0 0x7f1784102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 -- 172.21.15.63:0/793347910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17841092a0 msgr2=0x7f1784102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17841092a0 0x7f1784102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 -- 172.21.15.63:0/793347910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1784106970 msgr2=0x7f1784101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1784106970 0x7f1784101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.549+0000 7f1783400700 1 -- 172.21.15.63:0/793347910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1784102de0 con 0x7f1784104040 2024-05-11T13:33:08.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.550+0000 7f1783400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f17841039a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1774000c00 tx=0x7f17740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:08.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.550+0000 7f1780c00700 1 -- 172.21.15.63:0/793347910 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774003c10 con 0x7f1784104040 2024-05-11T13:33:08.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.550+0000 7f1780c00700 1 -- 172.21.15.63:0/793347910 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1774004830 con 0x7f1784104040 2024-05-11T13:33:08.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.550+0000 7f1780c00700 1 -- 172.21.15.63:0/793347910 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774005350 con 0x7f1784104040 2024-05-11T13:33:08.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.550+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841a3830 con 0x7f1784104040 2024-05-11T13:33:08.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.550+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a3bd0 con 0x7f1784104040 2024-05-11T13:33:08.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.552+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178406a240 con 0x7f1784104040 2024-05-11T13:33:08.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.554+0000 7f1780c00700 1 -- 172.21.15.63:0/793347910 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17740054f0 con 0x7f1784104040 2024-05-11T13:33:08.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.555+0000 7f1780c00700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f17640b2880 0x7f17640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:08.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.556+0000 7f1780c00700 1 -- 172.21.15.63:0/793347910 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f17740e3840 con 0x7f1784104040 2024-05-11T13:33:08.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.556+0000 7f177aa00700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f17640b2880 0x7f17640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:08.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.557+0000 7f177aa00700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f17640b2880 0x7f17640b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f176c005950 tx=0x7f176c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:08.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.559+0000 7f1780c00700 1 -- 172.21.15.63:0/793347910 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1774013070 con 0x7f1784104040 2024-05-11T13:33:08.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.801+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f178402e5a0 con 0x7f1784104040 2024-05-11T13:33:08.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.804+0000 7f1780c00700 1 -- 172.21.15.63:0/793347910 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5668 (secure 0 0 0) 0x7f17740ab1e0 con 0x7f1784104040 2024-05-11T13:33:08.806 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 90 2024-05-11T13:33:08.806 INFO:teuthology.orchestra.run.smithi063.stdout:fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T13:33:08.806 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-11T13:17:22.623485+0000 2024-05-11T13:33:08.806 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-11T13:30:41.007052+0000 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:crush_version 27 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:full_ratio 0.95 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:backfillfull_ratio 0.9 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:nearfull_ratio 0.85 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:require_min_compat_client luminous 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:min_compat_client luminous 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:require_osd_release squid 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:stretch_mode_enabled false 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-11T13:33:08.807 INFO:teuthology.orchestra.run.smithi063.stdout:max_osd 12 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6802/1611555800,v1:172.21.15.63:6803/1611555800] [v2:172.21.15.63:6804/1611555800,v1:172.21.15.63:6805/1611555800] exists,up 22f2805e-feb6-4126-b7f7-d4567818ce77 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/969700086,v1:172.21.15.71:6801/969700086] [v2:172.21.15.71:6802/969700086,v1:172.21.15.71:6803/969700086] exists,up 29a3b52e-3ed5-403c-b9c4-0a69b8d1b989 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/1296073597,v1:172.21.15.165:6801/1296073597] [v2:172.21.15.165:6802/1296073597,v1:172.21.15.165:6803/1296073597] exists,up 5f94b33d-6308-46a0-9f5c-63bd1bcaa477 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6810/206479443,v1:172.21.15.63:6811/206479443] [v2:172.21.15.63:6812/206479443,v1:172.21.15.63:6813/206479443] exists,up d387295c-4396-466f-92e3-400391f0e674 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 up in weight 1 up_from 32 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/2625617400,v1:172.21.15.71:6809/2625617400] [v2:172.21.15.71:6810/2625617400,v1:172.21.15.71:6811/2625617400] exists,up d209500e-da38-45b9-9ae7-eddaafae3127 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/2071542018,v1:172.21.15.165:6809/2071542018] [v2:172.21.15.165:6810/2071542018,v1:172.21.15.165:6811/2071542018] exists,up 7dd5fca3-e67c-4824-af4e-2e6efa55db80 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6 up in weight 1 up_from 43 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6818/1008047147,v1:172.21.15.63:6819/1008047147] [v2:172.21.15.63:6820/1008047147,v1:172.21.15.63:6821/1008047147] exists,up 18fec429-9a88-4389-b373-438e9cad3dde 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7 up in weight 1 up_from 48 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/3194467426,v1:172.21.15.71:6817/3194467426] [v2:172.21.15.71:6818/3194467426,v1:172.21.15.71:6819/3194467426] exists,up ad08b6e9-8303-4d9b-8a92-f837c203f300 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.8 up in weight 1 up_from 54 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/4105128904,v1:172.21.15.165:6817/4105128904] [v2:172.21.15.165:6818/4105128904,v1:172.21.15.165:6819/4105128904] exists,up a5ddd20e-3176-42ba-853e-47ca81ab5ef0 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.9 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6826/1803317888,v1:172.21.15.63:6827/1803317888] [v2:172.21.15.63:6828/1803317888,v1:172.21.15.63:6829/1803317888] exists,up 4d377b63-0bda-4801-8f95-5e2e9b6bf9ee 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.10 up in weight 1 up_from 65 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6824/3023940928,v1:172.21.15.71:6825/3023940928] [v2:172.21.15.71:6826/3023940928,v1:172.21.15.71:6827/3023940928] exists,up cb80b66c-5c56-420b-8b5d-a0128ec05742 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.11 up in weight 1 up_from 71 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/2607132784,v1:172.21.15.165:6825/2607132784] [v2:172.21.15.165:6826/2607132784,v1:172.21.15.165:6827/2607132784] exists,up 222725ae-5dca-47ba-9146-2c4ef02a3272 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 2.e [2,1] 2024-05-11T13:33:08.808 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 2.f [8,11] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 2.1b [2,4] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 2.1c [8,11] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.7 [3,9] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.21 [4,8] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.24 [4,9] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.36 [4,9] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.39 [5,6] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.f [9,10] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.19 [9,8] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.1c [9,8] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.1d [5,8] 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/3897275629 expires 2024-05-12T13:18:39.354713+0000 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6801/405023443 expires 2024-05-12T13:18:39.354713+0000 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6800/405023443 expires 2024-05-12T13:18:39.354713+0000 2024-05-11T13:33:08.809 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/2606154945 expires 2024-05-12T13:18:39.354713+0000 2024-05-11T13:33:08.810 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/2618629485 expires 2024-05-12T13:18:04.171969+0000 2024-05-11T13:33:08.810 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/2351804499 expires 2024-05-12T13:18:04.171969+0000 2024-05-11T13:33:08.810 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/4232658320 expires 2024-05-12T13:18:04.171969+0000 2024-05-11T13:33:08.810 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6800/1250505383 expires 2024-05-12T13:18:04.171969+0000 2024-05-11T13:33:08.810 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6801/1250505383 expires 2024-05-12T13:18:04.171969+0000 2024-05-11T13:33:08.810 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/1483448731 expires 2024-05-12T13:18:39.354713+0000 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f17640b2880 msgr2=0x7f17640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f17640b2880 0x7f17640b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f176c005950 tx=0x7f176c0058e0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 msgr2=0x7f17841039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f17841039a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1774000c00 tx=0x7f17740047b0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 shutdown_connections 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f17640b2880 0x7f17640b4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17841092a0 0x7f1784102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1784104040 0x7f17841039a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 --2- 172.21.15.63:0/793347910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1784106970 0x7f1784101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 >> 172.21.15.63:0/793347910 conn(0x7f17840fda50 msgr2=0x7f1784104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:08.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.807+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 shutdown_connections 2024-05-11T13:33:08.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:08.808+0000 7f178a400700 1 -- 172.21.15.63:0/793347910 wait complete. 2024-05-11T13:33:09.182 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:08 smithi063 ceph-mon[29809]: pgmap v514: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:09.182 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:08 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/793347910' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-11T13:33:09.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:08 smithi071 ceph-mon[33293]: pgmap v514: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:09.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:08 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/793347910' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-11T13:33:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:08 smithi165 ceph-mon[33235]: pgmap v514: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:09.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:08 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/793347910' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-11T13:33:09.592 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-11T13:33:09.603 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:33:10.070 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:11.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:10 smithi063 ceph-mon[29809]: pgmap v515: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:11.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:10 smithi071 ceph-mon[33293]: pgmap v515: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:11.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:10 smithi165 ceph-mon[33235]: pgmap v515: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 -- 172.21.15.63:0/2887529690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c075920 msgr2=0x7fe97c075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 --2- 172.21.15.63:0/2887529690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c075920 0x7fe97c075d00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009990 tx=0x7fe96c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 -- 172.21.15.63:0/2887529690 shutdown_connections 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 --2- 172.21.15.63:0/2887529690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe97c10fd10 0x7fe97c112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 --2- 172.21.15.63:0/2887529690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c075920 0x7fe97c075d00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 --2- 172.21.15.63:0/2887529690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe97c076240 0x7fe97c10f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 -- 172.21.15.63:0/2887529690 >> 172.21.15.63:0/2887529690 conn(0x7fe97c0fef60 msgr2=0x7fe97c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 -- 172.21.15.63:0/2887529690 shutdown_connections 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.434+0000 7fe981c00700 1 -- 172.21.15.63:0/2887529690 wait complete. 2024-05-11T13:33:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.435+0000 7fe981c00700 1 Processor -- start 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.435+0000 7fe981c00700 1 -- start start 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe981c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe97c075920 0x7fe97c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe981c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c076240 0x7fe97c1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe981c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe97c10fd10 0x7fe97c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe981c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c115770 con 0x7fe97c076240 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe981c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1155f0 con 0x7fe97c10fd10 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe981c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1158f0 con 0x7fe97c075920 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe97b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c076240 0x7fe97c1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe97b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c076240 0x7fe97c1a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44030/0 (socket says 172.21.15.63:44030) 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe97b400700 1 -- 172.21.15.63:0/1011329848 learned_addr learned my addr 172.21.15.63:0/1011329848 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe981200700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe97c10fd10 0x7fe97c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.436+0000 7fe97be00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe97c075920 0x7fe97c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe97b400700 1 -- 172.21.15.63:0/1011329848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe97c075920 msgr2=0x7fe97c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe97b400700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe97c075920 0x7fe97c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe97b400700 1 -- 172.21.15.63:0/1011329848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe97c10fd10 msgr2=0x7fe97c19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe97b400700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe97c10fd10 0x7fe97c19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe97b400700 1 -- 172.21.15.63:0/1011329848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c19c410 con 0x7fe97c076240 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe981200700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe97c10fd10 0x7fe97c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:33:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe97b400700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c076240 0x7fe97c1a1fa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe96800d980 tx=0x7fe96800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.437+0000 7fe978c00700 1 -- 172.21.15.63:0/1011329848 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968017070 con 0x7fe97c076240 2024-05-11T13:33:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.438+0000 7fe978c00700 1 -- 172.21.15.63:0/1011329848 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe968004830 con 0x7fe97c076240 2024-05-11T13:33:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.438+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c19c700 con 0x7fe97c076240 2024-05-11T13:33:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.438+0000 7fe978c00700 1 -- 172.21.15.63:0/1011329848 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680053a0 con 0x7fe97c076240 2024-05-11T13:33:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.438+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c1a89e0 con 0x7fe97c076240 2024-05-11T13:33:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.439+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c076fd0 con 0x7fe97c076240 2024-05-11T13:33:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.441+0000 7fe978c00700 1 -- 172.21.15.63:0/1011329848 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe9680049e0 con 0x7fe97c076240 2024-05-11T13:33:12.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.443+0000 7fe978c00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe9600b2900 0x7fe9600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:12.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.443+0000 7fe978c00700 1 -- 172.21.15.63:0/1011329848 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe9680e2540 con 0x7fe97c076240 2024-05-11T13:33:12.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.443+0000 7fe97be00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe9600b2900 0x7fe9600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.444+0000 7fe97be00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe9600b2900 0x7fe9600b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b5c0 tx=0x7fe96c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.448+0000 7fe978c00700 1 -- 172.21.15.63:0/1011329848 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe968020020 con 0x7fe97c076240 2024-05-11T13:33:12.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.742+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe97c06a270 con 0x7fe97c076240 2024-05-11T13:33:12.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.743+0000 7fe978c00700 1 -- 172.21.15.63:0/1011329848 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fe97c06a270 con 0x7fe97c076240 2024-05-11T13:33:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:12.745 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:30:02.730662+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe9600b2900 msgr2=0x7fe9600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe9600b2900 0x7fe9600b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b5c0 tx=0x7fe96c01a040 comp rx=0 tx=0).stop 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c076240 msgr2=0x7fe97c1a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c076240 0x7fe97c1a1fa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe96800d980 tx=0x7fe96800dd40 comp rx=0 tx=0).stop 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 shutdown_connections 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fe9600b2900 0x7fe9600b4dc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe97c10fd10 0x7fe97c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe97c075920 0x7fe97c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 --2- 172.21.15.63:0/1011329848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe97c076240 0x7fe97c1a1fa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 >> 172.21.15.63:0/1011329848 conn(0x7fe97c0fef60 msgr2=0x7fe97c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 shutdown_connections 2024-05-11T13:33:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:12.747+0000 7fe981c00700 1 -- 172.21.15.63:0/1011329848 wait complete. 2024-05-11T13:33:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-11T13:33:13.146 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:12 smithi063 ceph-mon[29809]: pgmap v516: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:13.146 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:12 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1011329848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:33:13.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:12 smithi071 ceph-mon[33293]: pgmap v516: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:13.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:12 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1011329848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:33:13.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:12 smithi165 ceph-mon[33235]: pgmap v516: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:13.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:12 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1011329848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:33:13.717 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:33:14.191 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:15.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:14 smithi063 ceph-mon[29809]: pgmap v517: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:15.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:14 smithi071 ceph-mon[33293]: pgmap v517: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:15.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:14 smithi165 ceph-mon[33235]: pgmap v517: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:16.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.756+0000 7f0470c00700 1 -- 172.21.15.63:0/1901632758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06bbf0 msgr2=0x7f046c1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:16.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.756+0000 7f0470c00700 1 --2- 172.21.15.63:0/1901632758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06bbf0 0x7f046c1103e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0464009990 tx=0x7f0464009ca0 comp rx=0 tx=0).stop 2024-05-11T13:33:16.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.757+0000 7f0470c00700 1 -- 172.21.15.63:0/1901632758 shutdown_connections 2024-05-11T13:33:16.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.757+0000 7f0470c00700 1 --2- 172.21.15.63:0/1901632758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06bbf0 0x7f046c1103e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:16.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.757+0000 7f0470c00700 1 --2- 172.21.15.63:0/1901632758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f046c1095c0 0x7f046c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:16.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.757+0000 7f0470c00700 1 --2- 172.21.15.63:0/1901632758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f046c06b230 0x7f046c06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:16.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.757+0000 7f0470c00700 1 -- 172.21.15.63:0/1901632758 >> 172.21.15.63:0/1901632758 conn(0x7f046c079400 msgr2=0x7f046c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:16.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.757+0000 7f0470c00700 1 -- 172.21.15.63:0/1901632758 shutdown_connections 2024-05-11T13:33:16.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.757+0000 7f0470c00700 1 -- 172.21.15.63:0/1901632758 wait complete. 2024-05-11T13:33:16.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 Processor -- start 2024-05-11T13:33:16.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 -- start start 2024-05-11T13:33:16.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06b230 0x7f046c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:16.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f046c06bbf0 0x7f046c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f046c1095c0 0x7f046c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f046c113600 con 0x7f046c06b230 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f046c113480 con 0x7f046c1095c0 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.758+0000 7f0470c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f046c113780 con 0x7f046c06bbf0 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06b230 0x7f046c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06b230 0x7f046c19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44046/0 (socket says 172.21.15.63:44046) 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 -- 172.21.15.63:0/486931479 learned_addr learned my addr 172.21.15.63:0/486931479 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f046a800700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f046c1095c0 0x7f046c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0461400700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f046c06bbf0 0x7f046c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 -- 172.21.15.63:0/486931479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f046c06bbf0 msgr2=0x7f046c19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f046c06bbf0 0x7f046c19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 -- 172.21.15.63:0/486931479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f046c1095c0 msgr2=0x7f046c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:16.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f046c1095c0 0x7f046c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0469e00700 1 -- 172.21.15.63:0/486931479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c1a0d50 con 0x7f046c06b230 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.759+0000 7f0461400700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f046c06bbf0 0x7f046c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.760+0000 7f0469e00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06b230 0x7f046c19bd90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f045c0060b0 tx=0x7f045c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.760+0000 7f0463400700 1 -- 172.21.15.63:0/486931479 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c01a070 con 0x7f046c06b230 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.760+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c1a1040 con 0x7f046c06b230 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.760+0000 7f0463400700 1 -- 172.21.15.63:0/486931479 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f045c00ddd0 con 0x7f046c06b230 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.760+0000 7f0463400700 1 -- 172.21.15.63:0/486931479 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c014690 con 0x7f046c06b230 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.760+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c1a88c0 con 0x7f046c06b230 2024-05-11T13:33:16.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.762+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046c06a240 con 0x7f046c06b230 2024-05-11T13:33:16.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.764+0000 7f0463400700 1 -- 172.21.15.63:0/486931479 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f045c01d070 con 0x7f046c06b230 2024-05-11T13:33:16.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.765+0000 7f0463400700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f04500b2880 0x7f04500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:16.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.766+0000 7f0463400700 1 -- 172.21.15.63:0/486931479 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f045c0e22f0 con 0x7f046c06b230 2024-05-11T13:33:16.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.766+0000 7f0461400700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f04500b2880 0x7f04500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:16.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.766+0000 7f0461400700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f04500b2880 0x7f04500b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f046c104540 tx=0x7f045400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:16.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:16.768+0000 7f0463400700 1 -- 172.21.15.63:0/486931479 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f045c010070 con 0x7f046c06b230 2024-05-11T13:33:17.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.059+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f046c19d0a0 con 0x7f046c06b230 2024-05-11T13:33:17.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.062+0000 7f0463400700 1 -- 172.21.15.63:0/486931479 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f046c19d0a0 con 0x7f046c06b230 2024-05-11T13:33:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:17.063 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:30:02.730662+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:33:17.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f04500b2880 msgr2=0x7f04500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:17.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f04500b2880 0x7f04500b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f046c104540 tx=0x7f045400b2c0 comp rx=0 tx=0).stop 2024-05-11T13:33:17.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06b230 msgr2=0x7f046c19bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:17.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06b230 0x7f046c19bd90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f045c0060b0 tx=0x7f045c00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:33:17.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 shutdown_connections 2024-05-11T13:33:17.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f04500b2880 0x7f04500b4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:17.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f046c1095c0 0x7f046c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:17.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f046c06b230 0x7f046c19bd90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:17.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 --2- 172.21.15.63:0/486931479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f046c06bbf0 0x7f046c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:17.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 >> 172.21.15.63:0/486931479 conn(0x7f046c079400 msgr2=0x7f046c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:17.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 shutdown_connections 2024-05-11T13:33:17.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:17.065+0000 7f0470c00700 1 -- 172.21.15.63:0/486931479 wait complete. 2024-05-11T13:33:17.068 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-11T13:33:17.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:16 smithi063 ceph-mon[29809]: pgmap v518: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:17.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:16 smithi071 ceph-mon[33293]: pgmap v518: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:17.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:16 smithi165 ceph-mon[33235]: pgmap v518: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:17.972 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:33:18.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:17 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/486931479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:33:18.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:17 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/486931479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:33:18.316 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:17 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/486931479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:33:18.449 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:19.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:18 smithi063 ceph-mon[29809]: pgmap v519: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:19.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:18 smithi071 ceph-mon[33293]: pgmap v519: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:19.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:18 smithi165 ceph-mon[33235]: pgmap v519: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:20.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 -- 172.21.15.63:0/3759874448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 msgr2=0x7f71481099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:20.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 --2- 172.21.15.63:0/3759874448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481099a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7138009990 tx=0x7f7138009ca0 comp rx=0 tx=0).stop 2024-05-11T13:33:20.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 -- 172.21.15.63:0/3759874448 shutdown_connections 2024-05-11T13:33:20.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 --2- 172.21.15.63:0/3759874448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f714806bbf0 0x7f71481103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:20.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 --2- 172.21.15.63:0/3759874448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481099a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:20.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 --2- 172.21.15.63:0/3759874448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714806b230 0x7f714806b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:20.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 -- 172.21.15.63:0/3759874448 >> 172.21.15.63:0/3759874448 conn(0x7f7148079400 msgr2=0x7f7148079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:20.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 -- 172.21.15.63:0/3759874448 shutdown_connections 2024-05-11T13:33:20.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.800+0000 7f714f000700 1 -- 172.21.15.63:0/3759874448 wait complete. 2024-05-11T13:33:20.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.801+0000 7f714f000700 1 Processor -- start 2024-05-11T13:33:20.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714f000700 1 -- start start 2024-05-11T13:33:20.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714f000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714806b230 0x7f714819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:20.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714f000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f714806bbf0 0x7f714819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714f000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714f000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71481135f0 con 0x7f71481095c0 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714f000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7148113470 con 0x7f714806bbf0 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714f000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7148113770 con 0x7f714806b230 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49338/0 (socket says 172.21.15.63:49338) 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.802+0000 7f714d000700 1 -- 172.21.15.63:0/344200346 learned_addr learned my addr 172.21.15.63:0/344200346 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f7147e00700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714806b230 0x7f714819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:20.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f713f400700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f714806bbf0 0x7f714819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f714d000700 1 -- 172.21.15.63:0/344200346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714806b230 msgr2=0x7f714819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f714d000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714806b230 0x7f714819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f714d000700 1 -- 172.21.15.63:0/344200346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f714806bbf0 msgr2=0x7f714819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f714d000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f714806bbf0 0x7f714819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f714d000700 1 -- 172.21.15.63:0/344200346 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71481a0d90 con 0x7f71481095c0 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f7147e00700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714806b230 0x7f714819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.803+0000 7f714d000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481a0650 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7140007ee0 tx=0x7f714000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.804+0000 7f7145600700 1 -- 172.21.15.63:0/344200346 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714000dbd0 con 0x7f71481095c0 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.804+0000 7f7145600700 1 -- 172.21.15.63:0/344200346 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f714000dd70 con 0x7f71481095c0 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.804+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71481a1080 con 0x7f71481095c0 2024-05-11T13:33:20.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.804+0000 7f7145600700 1 -- 172.21.15.63:0/344200346 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71400058d0 con 0x7f71481095c0 2024-05-11T13:33:20.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.804+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71481a8900 con 0x7f71481095c0 2024-05-11T13:33:20.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.805+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714806a240 con 0x7f71481095c0 2024-05-11T13:33:20.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.807+0000 7f7145600700 1 -- 172.21.15.63:0/344200346 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f714001a020 con 0x7f71481095c0 2024-05-11T13:33:20.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.809+0000 7f7145600700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f712c0b2900 0x7f712c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:20.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.809+0000 7f7145600700 1 -- 172.21.15.63:0/344200346 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f71400e1db0 con 0x7f71481095c0 2024-05-11T13:33:20.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.809+0000 7f7147e00700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f712c0b2900 0x7f712c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:20.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.810+0000 7f7147e00700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f712c0b2900 0x7f712c0b4dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f713800b5c0 tx=0x7f713801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:20.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:20.814+0000 7f7145600700 1 -- 172.21.15.63:0/344200346 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71400e6070 con 0x7f71481095c0 2024-05-11T13:33:21.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.054+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71481a8ad0 con 0x7f71481095c0 2024-05-11T13:33:21.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.056+0000 7f7145600700 1 -- 172.21.15.63:0/344200346 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24760 (secure 0 0 0) 0x7f7140010070 con 0x7f71481095c0 2024-05-11T13:33:21.058 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:21.058 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":90,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:30:41.007052+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.059+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f712c0b2900 msgr2=0x7f712c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.059+0000 7f714f000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f712c0b2900 0x7f712c0b4dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f713800b5c0 tx=0x7f713801a040 comp rx=0 tx=0).stop 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.059+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 msgr2=0x7f71481a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.059+0000 7f714f000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481a0650 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7140007ee0 tx=0x7f714000d3b0 comp rx=0 tx=0).stop 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 shutdown_connections 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f712c0b2900 0x7f712c0b4dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71481095c0 0x7f71481a0650 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714806b230 0x7f714819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 --2- 172.21.15.63:0/344200346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f714806bbf0 0x7f714819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 >> 172.21.15.63:0/344200346 conn(0x7f7148079400 msgr2=0x7f7148100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 shutdown_connections 2024-05-11T13:33:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:21.060+0000 7f714f000700 1 -- 172.21.15.63:0/344200346 wait complete. 2024-05-11T13:33:21.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:20 smithi063 ceph-mon[29809]: pgmap v520: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:21.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:20 smithi071 ceph-mon[33293]: pgmap v520: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:21.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:20 smithi165 ceph-mon[33235]: pgmap v520: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:22.013 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-11T13:28:38.143704+0000', 'modified': '2024-05-11T13:30:02.730662+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34472, 'mds_1': 34451, 'mds_2': 34427, 'mds_3': 34403, 'mds_4': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6839/3189418422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6838', 'nonce': 3189418422}, {'type': 'v1', 'addr': '172.21.15.63:6839', 'nonce': 3189418422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14577': {'gid': 14577, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6841/3763187067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6840', 'nonce': 3763187067}, {'type': 'v1', 'addr': '172.21.15.63:6841', 'nonce': 3763187067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/2703259087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 2703259087}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 2703259087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/4024086251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 4024086251}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 4024086251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2308228126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2308228126}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2308228126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.165:6839/1121013005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1121013005}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1121013005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34403': {'gid': 34403, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 37, 'addr': '172.21.15.71:6833/1870291573', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 1870291573}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 1870291573}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.71:6835/1316489545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 1316489545}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 1316489545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34451': {'gid': 34451, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.71:6837/136473158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 136473158}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 136473158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.71:6839/1525017608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1525017608}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1525017608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34472, 'qdb_cluster': [24472, 34403, 34427, 34451, 34472]} 2024-05-11T13:33:22.014 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-11T13:33:22.014 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-11T13:33:22.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:21 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/344200346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:33:22.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:21 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/344200346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:33:22.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:21 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/344200346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:33:22.487 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:23.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:22 smithi071 ceph-mon[33293]: pgmap v521: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:23.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:22 smithi165 ceph-mon[33235]: pgmap v521: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:23.401 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:22 smithi063 ceph-mon[29809]: pgmap v521: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:24.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 -- 172.21.15.63:0/4178552678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 msgr2=0x7ff94410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 --2- 172.21.15.63:0/4178552678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94410bb50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff938009990 tx=0x7ff938009ca0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 -- 172.21.15.63:0/4178552678 shutdown_connections 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 --2- 172.21.15.63:0/4178552678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff944106130 0x7ff944112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 --2- 172.21.15.63:0/4178552678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94410bb50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 --2- 172.21.15.63:0/4178552678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff944105770 0x7ff944105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 -- 172.21.15.63:0/4178552678 >> 172.21.15.63:0/4178552678 conn(0x7ff944101260 msgr2=0x7ff944103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.891+0000 7ff94ba00700 1 -- 172.21.15.63:0/4178552678 shutdown_connections 2024-05-11T13:33:24.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.892+0000 7ff94ba00700 1 -- 172.21.15.63:0/4178552678 wait complete. 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 Processor -- start 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 -- start start 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff944105770 0x7ff9441a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff944106130 0x7ff9441a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff944115820 con 0x7ff94410b770 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441156a0 con 0x7ff944106130 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.893+0000 7ff94ba00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441159a0 con 0x7ff944105770 2024-05-11T13:33:24.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94419bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49350/0 (socket says 172.21.15.63:49350) 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 -- 172.21.15.63:0/2652204791 learned_addr learned my addr 172.21.15.63:0/2652204791 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff943e00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff944106130 0x7ff9441a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949000700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff944105770 0x7ff9441a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 -- 172.21.15.63:0/2652204791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff944105770 msgr2=0x7ff9441a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff944105770 0x7ff9441a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 -- 172.21.15.63:0/2652204791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff944106130 msgr2=0x7ff9441a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff944106130 0x7ff9441a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949a00700 1 -- 172.21.15.63:0/2652204791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94419c260 con 0x7ff94410b770 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff949000700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff944105770 0x7ff9441a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:33:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.894+0000 7ff943e00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff944106130 0x7ff9441a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:33:24.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.895+0000 7ff949a00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94419bb00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00ada0 tx=0x7ff93c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:24.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.895+0000 7ff941600700 1 -- 172.21.15.63:0/2652204791 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c017020 con 0x7ff94410b770 2024-05-11T13:33:24.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.895+0000 7ff941600700 1 -- 172.21.15.63:0/2652204791 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff93c00ce10 con 0x7ff94410b770 2024-05-11T13:33:24.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.895+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94419c550 con 0x7ff94410b770 2024-05-11T13:33:24.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.895+0000 7ff941600700 1 -- 172.21.15.63:0/2652204791 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c005440 con 0x7ff94410b770 2024-05-11T13:33:24.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.895+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9441a87c0 con 0x7ff94410b770 2024-05-11T13:33:24.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.898+0000 7ff941600700 1 -- 172.21.15.63:0/2652204791 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff93c004ad0 con 0x7ff94410b770 2024-05-11T13:33:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.899+0000 7ff95646cb80 1 Processor -- start 2024-05-11T13:33:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff95646cb80 1 -- start start 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c45c30 0x55ff97c46010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 0x55ff97c40300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c40d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff95646cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ff97b929d0 con 0x55ff97c45c30 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff95646cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ff97b30710 con 0x55ff97c40920 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff95646cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ff97b8e990 con 0x55ff97c46550 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff941600700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff92c0b2900 0x7ff92c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.900+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c45c30 0x55ff97c46010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff941600700 1 -- 172.21.15.63:0/2652204791 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff93c0e1ce0 con 0x7ff94410b770 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff949a00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff92c0b2900 0x7ff92c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c45c30 0x55ff97c46010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49366/0 (socket says 172.21.15.63:49366) 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 -- 172.21.15.63:0/3276825863 learned_addr learned my addr 172.21.15.63:0/3276825863 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff949000700 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 0x55ff97c40300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c40d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 -- 172.21.15.63:0/3276825863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 msgr2=0x55ff97c40300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 0x55ff97c40300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 -- 172.21.15.63:0/3276825863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 msgr2=0x55ff97c40d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c40d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff943e00700 1 -- 172.21.15.63:0/3276825863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff97adf880 con 0x55ff97c45c30 2024-05-11T13:33:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.901+0000 7ff949a00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff92c0b2900 0x7ff92c0b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0a5000 tx=0x7ff93c0060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:24.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.902+0000 7ff943e00700 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c45c30 0x55ff97c46010 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff930005ea0 tx=0x7ff93000a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9119fa6e402298 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:24.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.902+0000 7ff936000700 1 -- 172.21.15.63:0/3276825863 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930018030 con 0x55ff97c45c30 2024-05-11T13:33:24.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.902+0000 7ff936000700 1 -- 172.21.15.63:0/3276825863 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff930010a80 con 0x55ff97c45c30 2024-05-11T13:33:24.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.902+0000 7ff936000700 1 -- 172.21.15.63:0/3276825863 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93000d880 con 0x55ff97c45c30 2024-05-11T13:33:24.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 -- 172.21.15.63:0/3276825863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c45c30 msgr2=0x55ff97c46010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c45c30 0x55ff97c46010 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff930005ea0 tx=0x7ff93000a780 comp rx=0 tx=0).stop 2024-05-11T13:33:24.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 -- 172.21.15.63:0/3276825863 shutdown_connections 2024-05-11T13:33:24.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c40d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c45c30 0x55ff97c46010 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3276825863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 0x55ff97c40300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 -- 172.21.15.63:0/3276825863 >> 172.21.15.63:0/3276825863 conn(0x55ff97b6eb10 msgr2=0x55ff97c3abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 -- 172.21.15.63:0/3276825863 shutdown_connections 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.903+0000 7ff95646cb80 1 -- 172.21.15.63:0/3276825863 wait complete. 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 -- start start 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 Processor -- start 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c4c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c46550 0x55ff97c49b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c4cfc0 con 0x55ff97c46550 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ff97af8ad0 con 0x55ff97c40920 2024-05-11T13:33:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.904+0000 7ff95646cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c4d140 con 0x55ff97c45c30 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff95646cb80 10 client.0 _ll_register_callbacks cb 0x55ff97b26330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c46550 0x55ff97c49b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c46550 0x55ff97c49b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49380/0 (socket says 172.21.15.63:49380) 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 -- 172.21.15.63:0/3125664337 learned_addr learned my addr 172.21.15.63:0/3125664337 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c4c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff949000700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 -- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 msgr2=0x55ff97c49630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c49630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 -- 172.21.15.63:0/3125664337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 msgr2=0x55ff97c4c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c4c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff943e00700 1 -- 172.21.15.63:0/3125664337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff97b583d0 con 0x55ff97c46550 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.905+0000 7ff949000700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:33:24.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff943e00700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c46550 0x55ff97c49b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff930010820 tx=0x7ff93000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5854ada2dd940b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:24.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff92aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff92aa00700 1 -- 172.21.15.63:0/3125664337 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930018070 con 0x55ff97c46550 2024-05-11T13:33:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff92aa00700 1 -- 172.21.15.63:0/3125664337 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff930004750 con 0x55ff97c46550 2024-05-11T13:33:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff930010a80 con 0x55ff97c46550 2024-05-11T13:33:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff92aa00700 1 -- 172.21.15.63:0/3125664337 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff930004950 con 0x55ff97c46550 2024-05-11T13:33:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff95646cb80 10 client.15180 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.906+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff97c57850 con 0x55ff97c46550 2024-05-11T13:33:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff92aa00700 1 -- 172.21.15.63:0/3125664337 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff930010a80 con 0x55ff97c46550 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff92aa00700 1 -- 172.21.15.63:0/3125664337 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff93000f040 con 0x55ff97c46550 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff95646cb80 10 client.15180 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff95646cb80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff95646cb80 20 client.15180 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97c59ab0 0x55ff97c5af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff95646cb80 4 client.15180 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.907+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 --> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff97c0d010 con 0x55ff97c59ab0 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.908+0000 7ff949000700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97c59ab0 0x55ff97c5af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.909+0000 7ff949000700 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97c59ab0 0x55ff97c5af20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25e0f8d1125b8991 server_cookie=2af890221d9bbf34 in_seq=0 out_seq=0 2024-05-11T13:33:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.909+0000 7ff92aa00700 10 client.15180 ms_handle_connect on v2:172.21.15.165:6838/1121013005 2024-05-11T13:33:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff92aa00700 1 -- 172.21.15.63:0/3125664337 <== mds.4 v2:172.21.15.165:6838/1121013005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff97c0d010 con 0x55ff97c59ab0 2024-05-11T13:33:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff92aa00700 10 client.15180 handle_command_reply: tid=0 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 client.15180 shutdown 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c46550 msgr2=0x55ff97c49b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c46550 0x55ff97c49b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff930010820 tx=0x7ff93000d460 comp rx=0 tx=0).stop 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 shutdown_connections 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97c59ab0 0x55ff97c5af20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c46550 0x55ff97c49b70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c40920 0x55ff97c4c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 --2- 172.21.15.63:0/3125664337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 >> 172.21.15.63:0/3125664337 conn(0x55ff97b6eb10 msgr2=0x55ff97c356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff92aa00700 0 client.15180 ms_handle_reset on v2:172.21.15.165:6838/1121013005 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 shutdown_connections 2024-05-11T13:33:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 1 -- 172.21.15.63:0/3125664337 wait complete. 2024-05-11T13:33:24.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.911+0000 7ff95646cb80 20 client.15180 trim_cache size 0 max 16384 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 Processor -- start 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 -- start start 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c46930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c46550 0x55ff97c4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c59260 con 0x55ff97c40920 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c59930 con 0x55ff97c46550 2024-05-11T13:33:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.934+0000 7ff95646cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c5a000 con 0x55ff97c45c30 2024-05-11T13:33:24.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c46930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c46930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49386/0 (socket says 172.21.15.63:49386) 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 -- 172.21.15.63:0/832294530 learned_addr learned my addr 172.21.15.63:0/832294530 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 -- 172.21.15.63:0/832294530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 msgr2=0x55ff97c46e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 --2- 172.21.15.63:0/832294530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 -- 172.21.15.63:0/832294530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c46550 msgr2=0x55ff97c4ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 --2- 172.21.15.63:0/832294530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c46550 0x55ff97c4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.935+0000 7ff943e00700 1 -- 172.21.15.63:0/832294530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff97c59750 con 0x55ff97c40920 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.936+0000 7ff943e00700 1 --2- 172.21.15.63:0/832294530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c46930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d550 tx=0x7ff93000afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8e78cee903a770 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.936+0000 7ff935600700 1 -- 172.21.15.63:0/832294530 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930018070 con 0x55ff97c40920 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.936+0000 7ff935600700 1 -- 172.21.15.63:0/832294530 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff93000a510 con 0x55ff97c40920 2024-05-11T13:33:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.936+0000 7ff935600700 1 -- 172.21.15.63:0/832294530 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930012080 con 0x55ff97c40920 2024-05-11T13:33:24.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.936+0000 7ff95646cb80 1 -- 172.21.15.63:0/832294530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 msgr2=0x55ff97c46930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.936+0000 7ff95646cb80 1 --2- 172.21.15.63:0/832294530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c46930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d550 tx=0x7ff93000afb0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 -- 172.21.15.63:0/832294530 shutdown_connections 2024-05-11T13:33:24.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 --2- 172.21.15.63:0/832294530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c46550 0x55ff97c4ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 --2- 172.21.15.63:0/832294530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c46930 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 --2- 172.21.15.63:0/832294530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c45c30 0x55ff97c46e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 -- 172.21.15.63:0/832294530 >> 172.21.15.63:0/832294530 conn(0x55ff97b6eb10 msgr2=0x55ff97c47f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:24.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 -- 172.21.15.63:0/832294530 shutdown_connections 2024-05-11T13:33:24.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 -- 172.21.15.63:0/832294530 wait complete. 2024-05-11T13:33:24.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 -- start start 2024-05-11T13:33:24.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.937+0000 7ff95646cb80 1 Processor -- start 2024-05-11T13:33:24.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c45c30 0x55ff97c648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff95646cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 0x55ff97c5f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff95646cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c5ada0 con 0x55ff97c40920 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff95646cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c5a6d0 con 0x55ff97c45c30 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff95646cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ff97c5b470 con 0x55ff97c46550 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff95646cb80 10 client.0 _ll_register_callbacks cb 0x55ff97b26330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff943e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49402/0 (socket says 172.21.15.63:49402) 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.938+0000 7ff943e00700 1 -- 172.21.15.63:0/2300808716 learned_addr learned my addr 172.21.15.63:0/2300808716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff949000700 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c45c30 0x55ff97c648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff943e00700 1 -- 172.21.15.63:0/2300808716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 msgr2=0x55ff97c5f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff943e00700 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 0x55ff97c5f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff943e00700 1 -- 172.21.15.63:0/2300808716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c45c30 msgr2=0x55ff97c648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff943e00700 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c45c30 0x55ff97c648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff943e00700 1 -- 172.21.15.63:0/2300808716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff97c5fb70 con 0x55ff97c40920 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff943e00700 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c643a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff930012390 tx=0x7ff93000a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46d56361abc5a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:24.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff92aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.939+0000 7ff92aa00700 1 -- 172.21.15.63:0/2300808716 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93003b060 con 0x55ff97c40920 2024-05-11T13:33:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.940+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff97ba03b0 con 0x55ff97c40920 2024-05-11T13:33:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.940+0000 7ff92aa00700 1 -- 172.21.15.63:0/2300808716 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff93000dcb0 con 0x55ff97c40920 2024-05-11T13:33:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.940+0000 7ff92aa00700 1 -- 172.21.15.63:0/2300808716 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff930018050 con 0x55ff97c40920 2024-05-11T13:33:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.940+0000 7ff95646cb80 10 client.15186 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.940+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff97cc59c0 con 0x55ff97c40920 2024-05-11T13:33:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.940+0000 7ff92aa00700 1 -- 172.21.15.63:0/2300808716 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ff97ba03b0 con 0x55ff97c40920 2024-05-11T13:33:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.941+0000 7ff92aa00700 1 -- 172.21.15.63:0/2300808716 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff93001d410 con 0x55ff97c40920 2024-05-11T13:33:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.941+0000 7ff95646cb80 10 client.15186 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.941+0000 7ff95646cb80 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-11T13:33:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.941+0000 7ff95646cb80 20 client.15186 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.941+0000 7ff95646cb80 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97cca1a0 0x55ff97cc8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.941+0000 7ff95646cb80 4 client.15186 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:33:24.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.941+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 --> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] -- command(tid 0: {"prefix": "status"}) -- 0x55ff97b9a450 con 0x55ff97cca1a0 2024-05-11T13:33:24.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.942+0000 7ff949000700 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97cca1a0 0x55ff97cc8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:24.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.942+0000 7ff949000700 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97cca1a0 0x55ff97cc8eb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d9d176832640035 server_cookie=bad9f4158aea5b91 in_seq=0 out_seq=0 2024-05-11T13:33:24.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.942+0000 7ff92aa00700 10 client.15186 ms_handle_connect on v2:172.21.15.165:6838/1121013005 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff92aa00700 1 -- 172.21.15.63:0/2300808716 <== mds.4 v2:172.21.15.165:6838/1121013005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ff97b9a450 con 0x55ff97cca1a0 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff92aa00700 10 client.15186 handle_command_reply: tid=0 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 client.15186 shutdown 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 msgr2=0x55ff97c643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c643a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff930012390 tx=0x7ff93000a3d0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 shutdown_connections 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55ff97cca1a0 0x55ff97cc8eb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff97c46550 0x55ff97c5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff97c40920 0x55ff97c643a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 --2- 172.21.15.63:0/2300808716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff97c45c30 0x55ff97c648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 >> 172.21.15.63:0/2300808716 conn(0x55ff97b6eb10 msgr2=0x55ff97c471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.943+0000 7ff92aa00700 0 client.15186 ms_handle_reset on v2:172.21.15.165:6838/1121013005 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.944+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 shutdown_connections 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.944+0000 7ff95646cb80 1 -- 172.21.15.63:0/2300808716 wait complete. 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.944+0000 7ff95646cb80 20 client.15186 trim_cache size 0 max 16384 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 4, 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 24472, 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:33:24.946 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:33:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:33:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 207.188504521, 2024-05-11T13:33:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:33:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:33:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:33:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 319.25605159999998 2024-05-11T13:33:24.947 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff92c0b2900 msgr2=0x7ff92c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff92c0b2900 0x7ff92c0b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0a5000 tx=0x7ff93c0060c0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 msgr2=0x7ff94419bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94419bb00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00ada0 tx=0x7ff93c00c330 comp rx=0 tx=0).stop 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 shutdown_connections 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7ff92c0b2900 0x7ff92c0b4dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94410b770 0x7ff94419bb00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff944105770 0x7ff9441a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 --2- 172.21.15.63:0/2652204791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff944106130 0x7ff9441a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 >> 172.21.15.63:0/2652204791 conn(0x7ff944101260 msgr2=0x7ff944101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 shutdown_connections 2024-05-11T13:33:24.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:24.945+0000 7ff94ba00700 1 -- 172.21.15.63:0/2652204791 wait complete. 2024-05-11T13:33:25.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:24 smithi071 ceph-mon[33293]: pgmap v522: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:25.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:24 smithi165 ceph-mon[33235]: pgmap v522: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:25.366 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:24 smithi063 ceph-mon[29809]: pgmap v522: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:25.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.k status 2024-05-11T13:33:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:27.097 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:26 smithi063 ceph-mon[29809]: pgmap v523: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:27.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:26 smithi071 ceph-mon[33293]: pgmap v523: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:27.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:26 smithi165 ceph-mon[33235]: pgmap v523: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:28.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 -- 172.21.15.63:0/479767191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198105770 msgr2=0x7fa198105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 --2- 172.21.15.63:0/479767191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198105770 0x7fa198105bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa184009990 tx=0x7fa184009ca0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 -- 172.21.15.63:0/479767191 shutdown_connections 2024-05-11T13:33:28.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 --2- 172.21.15.63:0/479767191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa198106130 0x7fa198112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 --2- 172.21.15.63:0/479767191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19810b770 0x7fa19810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 --2- 172.21.15.63:0/479767191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198105770 0x7fa198105bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 -- 172.21.15.63:0/479767191 >> 172.21.15.63:0/479767191 conn(0x7fa198101260 msgr2=0x7fa198103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:28.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 -- 172.21.15.63:0/479767191 shutdown_connections 2024-05-11T13:33:28.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.814+0000 7fa19ec00700 1 -- 172.21.15.63:0/479767191 wait complete. 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.815+0000 7fa19ec00700 1 Processor -- start 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa19ec00700 1 -- start start 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa19ec00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa198105770 0x7fa1981a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa19ec00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198106130 0x7fa1981a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa19ec00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19810b770 0x7fa19819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa19ec00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa198115870 con 0x7fa198106130 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa19ec00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981156f0 con 0x7fa19810b770 2024-05-11T13:33:28.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa19ec00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981159f0 con 0x7fa198105770 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa197400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198106130 0x7fa1981a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.816+0000 7fa197400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198106130 0x7fa1981a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49422/0 (socket says 172.21.15.63:49422) 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197400700 1 -- 172.21.15.63:0/860129733 learned_addr learned my addr 172.21.15.63:0/860129733 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197e00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa198105770 0x7fa1981a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa19cc00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19810b770 0x7fa19819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197400700 1 -- 172.21.15.63:0/860129733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa198105770 msgr2=0x7fa1981a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197400700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa198105770 0x7fa1981a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197400700 1 -- 172.21.15.63:0/860129733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19810b770 msgr2=0x7fa19819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197400700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19810b770 0x7fa19819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197400700 1 -- 172.21.15.63:0/860129733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19819c2b0 con 0x7fa198106130 2024-05-11T13:33:28.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.817+0000 7fa197400700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198106130 0x7fa1981a1e80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa184000c00 tx=0x7fa184005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:28.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.818+0000 7fa194c00700 1 -- 172.21.15.63:0/860129733 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184027070 con 0x7fa198106130 2024-05-11T13:33:28.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.818+0000 7fa194c00700 1 -- 172.21.15.63:0/860129733 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa18400fb80 con 0x7fa198106130 2024-05-11T13:33:28.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.818+0000 7fa194c00700 1 -- 172.21.15.63:0/860129733 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184022c30 con 0x7fa198106130 2024-05-11T13:33:28.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.818+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19819c540 con 0x7fa198106130 2024-05-11T13:33:28.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.818+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1981a8810 con 0x7fa198106130 2024-05-11T13:33:28.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.822+0000 7fa194c00700 1 -- 172.21.15.63:0/860129733 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa18402a070 con 0x7fa198106130 2024-05-11T13:33:28.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 Processor -- start 2024-05-11T13:33:28.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 -- start start 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x563764284500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x563764288550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764288a90 0x563764288ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5637641d69d0 con 0x563764284a40 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa194c00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa17c0b2900 0x7fa17c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563764174710 con 0x563764284120 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637641d2990 con 0x563764288a90 2024-05-11T13:33:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.823+0000 7fa194c00700 1 -- 172.21.15.63:0/860129733 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa1840e34f0 con 0x7fa198106130 2024-05-11T13:33:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa19cc00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa17c0b2900 0x7fa17c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x563764288550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x563764288550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49424/0 (socket says 172.21.15.63:49424) 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 -- 172.21.15.63:0/2027138156 learned_addr learned my addr 172.21.15.63:0/2027138156 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197400700 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764288a90 0x563764288ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa19cc00700 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x563764284500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 -- 172.21.15.63:0/2027138156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764288a90 msgr2=0x563764288ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764288a90 0x563764288ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 -- 172.21.15.63:0/2027138156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 msgr2=0x563764284500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x563764284500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.824+0000 7fa197e00700 1 -- 172.21.15.63:0/2027138156 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563764123880 con 0x563764284a40 2024-05-11T13:33:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.825+0000 7fa19cc00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa17c0b2900 0x7fa17c0b4dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa190009fd0 tx=0x7fa190009c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.825+0000 7fa197e00700 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x563764288550 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa188008bd0 tx=0x7fa188008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff3900c7f1fc431 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.825+0000 7fa18d600700 1 -- 172.21.15.63:0/2027138156 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188014070 con 0x563764284a40 2024-05-11T13:33:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.825+0000 7fa18d600700 1 -- 172.21.15.63:0/2027138156 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa18800ed10 con 0x563764284a40 2024-05-11T13:33:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.825+0000 7fa18d600700 1 -- 172.21.15.63:0/2027138156 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188005570 con 0x563764284a40 2024-05-11T13:33:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 -- 172.21.15.63:0/2027138156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 msgr2=0x563764288550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x563764288550 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa188008bd0 tx=0x7fa188008ee0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 -- 172.21.15.63:0/2027138156 shutdown_connections 2024-05-11T13:33:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764288a90 0x563764288ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x563764284500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/2027138156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x563764288550 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 -- 172.21.15.63:0/2027138156 >> 172.21.15.63:0/2027138156 conn(0x5637641b2b10 msgr2=0x56376427e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 -- 172.21.15.63:0/2027138156 shutdown_connections 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.826+0000 7fa1a9500b80 1 -- 172.21.15.63:0/2027138156 wait complete. 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.827+0000 7fa1a9500b80 1 -- start start 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.827+0000 7fa1a9500b80 1 Processor -- start 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.827+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.827+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284a40 0x56376428d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.827+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x56376428d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563764290ca0 con 0x563764288a90 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56376413cad0 con 0x563764284a40 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563764290e20 con 0x563764284120 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa1a9500b80 10 client.0 _ll_register_callbacks cb 0x56376417dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa197e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa197e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:38570/0 (socket says 172.21.15.63:38570) 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa197e00700 1 -- 172.21.15.63:0/501244947 learned_addr learned my addr 172.21.15.63:0/501244947 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa197400700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284a40 0x56376428d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.828+0000 7fa19cc00700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x56376428d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa197400700 1 -- 172.21.15.63:0/501244947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 msgr2=0x56376428ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa197400700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa197400700 1 -- 172.21.15.63:0/501244947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 msgr2=0x56376428d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa197400700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x56376428d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa197400700 1 -- 172.21.15.63:0/501244947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376419c3d0 con 0x563764284a40 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa19cc00700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x56376428d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa197400700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284a40 0x56376428d340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1840e0be0 tx=0x7fa1840aa150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a17ea3c07c7283b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.829+0000 7fa182000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:33:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.830+0000 7fa182000700 1 -- 172.21.15.63:0/501244947 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840aa2a0 con 0x563764284a40 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.830+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa18800ed10 con 0x563764284a40 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.830+0000 7fa182000700 1 -- 172.21.15.63:0/501244947 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa1840e3190 con 0x563764284a40 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.830+0000 7fa182000700 1 -- 172.21.15.63:0/501244947 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1840aae90 con 0x563764284a40 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.830+0000 7fa1a9500b80 10 client.34922 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.830+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56376429b5a0 con 0x563764284a40 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa182000700 1 -- 172.21.15.63:0/501244947 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa18800ed10 con 0x563764284a40 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa182000700 1 -- 172.21.15.63:0/501244947 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa1840b30d0 con 0x563764284a40 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa1a9500b80 10 client.34922 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa1a9500b80 10 client.34922 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa1a9500b80 20 client.34922 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376429dc40 0x56376429f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa1a9500b80 4 client.34922 mds_command: new command op to 34403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:33:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.831+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 --> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563764251160 con 0x56376429dc40 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.832+0000 7fa197e00700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376429dc40 0x56376429f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.833+0000 7fa197e00700 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376429dc40 0x56376429f0b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=210c97f9ff698177 server_cookie=20bd19d63ae29ebd in_seq=0 out_seq=0 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.833+0000 7fa182000700 10 client.34922 ms_handle_connect on v2:172.21.15.71:6832/1870291573 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa182000700 1 -- 172.21.15.63:0/501244947 <== mds.3 v2:172.21.15.71:6832/1870291573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563764251160 con 0x56376429dc40 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa182000700 10 client.34922 handle_command_reply: tid=0 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 client.34922 shutdown 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284a40 msgr2=0x56376428d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284a40 0x56376428d340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1840e0be0 tx=0x7fa1840aa150 comp rx=0 tx=0).stop 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 shutdown_connections 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x56376428d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376429dc40 0x56376429f0b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/501244947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284a40 0x56376428d340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa182000700 0 client.34922 ms_handle_reset on v2:172.21.15.71:6832/1870291573 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 >> 172.21.15.63:0/501244947 conn(0x5637641b2b10 msgr2=0x563764279370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 shutdown_connections 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.835+0000 7fa1a9500b80 1 -- 172.21.15.63:0/501244947 wait complete. 2024-05-11T13:33:28.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.836+0000 7fa1a9500b80 20 client.34922 trim_cache size 0 max 16384 2024-05-11T13:33:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 Processor -- start 2024-05-11T13:33:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 -- start start 2024-05-11T13:33:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x56376428afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284a40 0x563764278ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x563764279020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56376429d3f0 con 0x563764288a90 2024-05-11T13:33:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56376429dac0 con 0x563764284120 2024-05-11T13:33:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56376429e190 con 0x563764284a40 2024-05-11T13:33:28.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa19cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x563764279020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa19cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x563764279020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49434/0 (socket says 172.21.15.63:49434) 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.859+0000 7fa19cc00700 1 -- 172.21.15.63:0/306951484 learned_addr learned my addr 172.21.15.63:0/306951484 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa197400700 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284a40 0x563764278ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa197e00700 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x56376428afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa19cc00700 1 -- 172.21.15.63:0/306951484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284a40 msgr2=0x563764278ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa19cc00700 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284a40 0x563764278ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa19cc00700 1 -- 172.21.15.63:0/306951484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 msgr2=0x56376428afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa19cc00700 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x56376428afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa19cc00700 1 -- 172.21.15.63:0/306951484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376427b4b0 con 0x563764288a90 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.860+0000 7fa19cc00700 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x563764279020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa190009320 tx=0x7fa19001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eae7751257e99c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.861+0000 7fa18cc00700 1 -- 172.21.15.63:0/306951484 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190004bb0 con 0x563764288a90 2024-05-11T13:33:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.861+0000 7fa18cc00700 1 -- 172.21.15.63:0/306951484 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa190004d50 con 0x563764288a90 2024-05-11T13:33:28.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.861+0000 7fa18cc00700 1 -- 172.21.15.63:0/306951484 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19001a440 con 0x563764288a90 2024-05-11T13:33:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.861+0000 7fa1a9500b80 1 -- 172.21.15.63:0/306951484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 msgr2=0x563764279020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.861+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x563764279020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa190009320 tx=0x7fa19001b000 comp rx=0 tx=0).stop 2024-05-11T13:33:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 -- 172.21.15.63:0/306951484 shutdown_connections 2024-05-11T13:33:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764288a90 0x563764279020 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764284120 0x56376428afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/306951484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284a40 0x563764278ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 -- 172.21.15.63:0/306951484 >> 172.21.15.63:0/306951484 conn(0x5637641b2b10 msgr2=0x56376428bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 -- 172.21.15.63:0/306951484 shutdown_connections 2024-05-11T13:33:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 -- 172.21.15.63:0/306951484 wait complete. 2024-05-11T13:33:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 -- start start 2024-05-11T13:33:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.862+0000 7fa1a9500b80 1 Processor -- start 2024-05-11T13:33:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x56376428a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa1a9500b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764288a90 0x56376428ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56376429ef30 con 0x563764284a40 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56376429e860 con 0x563764288a90 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa1a9500b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56376429f600 con 0x563764284120 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa1a9500b80 10 client.0 _ll_register_callbacks cb 0x56376417dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa197400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x56376428a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa197400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x56376428a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49442/0 (socket says 172.21.15.63:49442) 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa197400700 1 -- 172.21.15.63:0/3076123565 learned_addr learned my addr 172.21.15.63:0/3076123565 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.863+0000 7fa197e00700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa19cc00700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764288a90 0x56376428ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa197400700 1 -- 172.21.15.63:0/3076123565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 msgr2=0x56376428a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa197400700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa197400700 1 -- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764288a90 msgr2=0x56376428ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa197400700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764288a90 0x56376428ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa197400700 1 -- 172.21.15.63:0/3076123565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637642a6740 con 0x563764284a40 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa19cc00700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764288a90 0x56376428ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa197400700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x56376428a5e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa1840b1210 tx=0x7fa18400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364d494e3edb1731 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa182000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa182000700 1 -- 172.21.15.63:0/3076123565 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840aa2a0 con 0x563764284a40 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa182000700 1 -- 172.21.15.63:0/3076123565 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa18400f6f0 con 0x563764284a40 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.864+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa190004d50 con 0x563764284a40 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa182000700 1 -- 172.21.15.63:0/3076123565 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1840aa790 con 0x563764284a40 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 10 client.15204 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5637642a6740 con 0x563764284a40 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa182000700 1 -- 172.21.15.63:0/3076123565 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa190004d50 con 0x563764284a40 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa182000700 1 -- 172.21.15.63:0/3076123565 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa1840bfb50 con 0x563764284a40 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 10 client.15204 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 20 client.15204 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376430db80 0x56376430c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 4 client.15204 mds_command: new command op to 34403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.865+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 --> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] -- command(tid 0: {"prefix": "status"}) -- 0x5637641e43b0 con 0x56376430db80 2024-05-11T13:33:28.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.866+0000 7fa197e00700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376430db80 0x56376430c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:28.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.867+0000 7fa197e00700 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376430db80 0x56376430c890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9db0216236017b44 server_cookie=2d17fd3a4f84d3f7 in_seq=0 out_seq=0 2024-05-11T13:33:28.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.867+0000 7fa182000700 10 client.15204 ms_handle_connect on v2:172.21.15.71:6832/1870291573 2024-05-11T13:33:28.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.867+0000 7fa182000700 1 -- 172.21.15.63:0/3076123565 <== mds.3 v2:172.21.15.71:6832/1870291573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5637641e43b0 con 0x56376430db80 2024-05-11T13:33:28.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.867+0000 7fa182000700 10 client.15204 handle_command_reply: tid=0 2024-05-11T13:33:28.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.867+0000 7fa1a9500b80 1 client.15204 shutdown 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 msgr2=0x56376428a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x56376428a5e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa1840b1210 tx=0x7fa18400ff60 comp rx=0 tx=0).stop 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 shutdown_connections 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563764288a90 0x56376428ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x56376430db80 0x56376430c890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563764284120 0x56376428a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 --2- 172.21.15.63:0/3076123565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563764284a40 0x56376428a5e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 >> 172.21.15.63:0/3076123565 conn(0x5637641b2b10 msgr2=0x563764278ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa182000700 0 client.15204 ms_handle_reset on v2:172.21.15.71:6832/1870291573 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 shutdown_connections 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 1 -- 172.21.15.63:0/3076123565 wait complete. 2024-05-11T13:33:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.868+0000 7fa1a9500b80 20 client.15204 trim_cache size 0 max 16384 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 3, 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34403, 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 212.129216633, 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:33:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:33:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:33:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 355.05388643200001 2024-05-11T13:33:28.875 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:33:28.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa17c0b2900 msgr2=0x7fa17c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa17c0b2900 0x7fa17c0b4dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa190009fd0 tx=0x7fa190009c60 comp rx=0 tx=0).stop 2024-05-11T13:33:28.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198106130 msgr2=0x7fa1981a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198106130 0x7fa1981a1e80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa184000c00 tx=0x7fa184005ef0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 shutdown_connections 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fa17c0b2900 0x7fa17c0b4dc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19810b770 0x7fa19819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa198105770 0x7fa1981a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 --2- 172.21.15.63:0/860129733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa198106130 0x7fa1981a1e80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 >> 172.21.15.63:0/860129733 conn(0x7fa198101260 msgr2=0x7fa1981018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 shutdown_connections 2024-05-11T13:33:28.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:28.869+0000 7fa19ec00700 1 -- 172.21.15.63:0/860129733 wait complete. 2024-05-11T13:33:29.301 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:28 smithi063 ceph-mon[29809]: pgmap v524: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:29.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:28 smithi165 ceph-mon[33235]: pgmap v524: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:29.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:28 smithi071 ceph-mon[33293]: pgmap v524: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:29.842 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.e status 2024-05-11T13:33:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:31.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:30 smithi063 ceph-mon[29809]: pgmap v525: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:31.256 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:30 smithi071 ceph-mon[33293]: pgmap v525: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:31.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:30 smithi165 ceph-mon[33235]: pgmap v525: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.577+0000 7f4aefe00700 1 -- 172.21.15.63:0/4093338164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0103490 msgr2=0x7f4af0103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.577+0000 7f4aefe00700 1 --2- 172.21.15.63:0/4093338164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0103490 0x7f4af0103910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0099c0 tx=0x7f4adc009cd0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.577+0000 7f4aefe00700 1 -- 172.21.15.63:0/4093338164 shutdown_connections 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.577+0000 7f4aefe00700 1 --2- 172.21.15.63:0/4093338164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af0103e50 0x7f4af0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.577+0000 7f4aefe00700 1 --2- 172.21.15.63:0/4093338164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4af0109540 0x7f4af0109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.577+0000 7f4aefe00700 1 --2- 172.21.15.63:0/4093338164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0103490 0x7f4af0103910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.577+0000 7f4aefe00700 1 -- 172.21.15.63:0/4093338164 >> 172.21.15.63:0/4093338164 conn(0x7f4af00fef60 msgr2=0x7f4af0101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.578+0000 7f4aefe00700 1 -- 172.21.15.63:0/4093338164 shutdown_connections 2024-05-11T13:33:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.578+0000 7f4aefe00700 1 -- 172.21.15.63:0/4093338164 wait complete. 2024-05-11T13:33:32.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.579+0000 7f4aefe00700 1 Processor -- start 2024-05-11T13:33:32.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.579+0000 7f4aefe00700 1 -- start start 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aefe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4af0103490 0x7f4af0075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aefe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af0103e50 0x7f4af0075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aefe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0109540 0x7f4af00762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aefe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4af0113520 con 0x7f4af0109540 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aefe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4af01133a0 con 0x7f4af0103e50 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aefe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4af01136a0 con 0x7f4af0103490 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0109540 0x7f4af00762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0109540 0x7f4af00762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37884/0 (socket says 172.21.15.63:37884) 2024-05-11T13:33:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.580+0000 7f4aef400700 1 -- 172.21.15.63:0/2662932290 learned_addr learned my addr 172.21.15.63:0/2662932290 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:32.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aee000700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af0103e50 0x7f4af0075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aeea00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4af0103490 0x7f4af0075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aef400700 1 -- 172.21.15.63:0/2662932290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4af0103490 msgr2=0x7f4af0075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aef400700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4af0103490 0x7f4af0075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aef400700 1 -- 172.21.15.63:0/2662932290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af0103e50 msgr2=0x7f4af0075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aef400700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af0103e50 0x7f4af0075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aef400700 1 -- 172.21.15.63:0/2662932290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af01a7e90 con 0x7f4af0109540 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4aef400700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0109540 0x7f4af00762d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8007ee0 tx=0x7f4ae800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.581+0000 7f4ae7400700 1 -- 172.21.15.63:0/2662932290 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800dbd0 con 0x7f4af0109540 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.582+0000 7f4ae7400700 1 -- 172.21.15.63:0/2662932290 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ae800dd70 con 0x7f4af0109540 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.582+0000 7f4ae7400700 1 -- 172.21.15.63:0/2662932290 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80058d0 con 0x7f4af0109540 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.582+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af01a8180 con 0x7f4af0109540 2024-05-11T13:33:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.582+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af01a8640 con 0x7f4af0109540 2024-05-11T13:33:32.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.585+0000 7f4ae7400700 1 -- 172.21.15.63:0/2662932290 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4ae801a020 con 0x7f4af0109540 2024-05-11T13:33:32.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.586+0000 7f4b002e1b80 1 Processor -- start 2024-05-11T13:33:32.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.587+0000 7f4b002e1b80 1 -- start start 2024-05-11T13:33:32.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.587+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb1e40 0x560203fb2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.587+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.587+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203facd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.587+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560203efe9d0 con 0x560203fac8d0 2024-05-11T13:33:32.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.587+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560203e9c710 con 0x560203fb1e40 2024-05-11T13:33:32.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.587+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560203efa990 con 0x560203fb2760 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203facd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203facd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37892/0 (socket says 172.21.15.63:37892) 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 -- 172.21.15.63:0/1964435343 learned_addr learned my addr 172.21.15.63:0/1964435343 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aeea00700 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb1e40 0x560203fb2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 -- 172.21.15.63:0/1964435343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 msgr2=0x560203fac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 -- 172.21.15.63:0/1964435343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb1e40 msgr2=0x560203fb2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb1e40 0x560203fb2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.588+0000 7f4aee000700 1 -- 172.21.15.63:0/1964435343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560203e4b880 con 0x560203fac8d0 2024-05-11T13:33:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.589+0000 7f4aee000700 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203facd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4adc005cd0 tx=0x7f4adc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c48c3e5e948b7be server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.589+0000 7f4adbe00700 1 -- 172.21.15.63:0/1964435343 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0053e0 con 0x560203fac8d0 2024-05-11T13:33:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.589+0000 7f4adbe00700 1 -- 172.21.15.63:0/1964435343 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4adc01c070 con 0x560203fac8d0 2024-05-11T13:33:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.589+0000 7f4adbe00700 1 -- 172.21.15.63:0/1964435343 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0038e0 con 0x560203fac8d0 2024-05-11T13:33:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.589+0000 7f4ae7400700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4ad40b2900 0x7f4ad40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4aeea00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4ad40b2900 0x7f4ad40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4ae7400700 1 -- 172.21.15.63:0/2662932290 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4ae80e1df0 con 0x7f4af0109540 2024-05-11T13:33:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1964435343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 msgr2=0x560203facd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203facd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4adc005cd0 tx=0x7f4adc005c00 comp rx=0 tx=0).stop 2024-05-11T13:33:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1964435343 shutdown_connections 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203facd50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb1e40 0x560203fb2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1964435343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1964435343 >> 172.21.15.63:0/1964435343 conn(0x560203edab10 msgr2=0x560203fa6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1964435343 shutdown_connections 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.590+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1964435343 wait complete. 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.591+0000 7f4aeea00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4ad40b2900 0x7f4ad40b4dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0005fd0 tx=0x7f4ae00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.591+0000 7f4b002e1b80 1 -- start start 2024-05-11T13:33:32.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.591+0000 7f4b002e1b80 1 Processor -- start 2024-05-11T13:33:32.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203fb8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fb57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb2760 0x560203fb5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560203fb9150 con 0x560203fac8d0 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560203e64ad0 con 0x560203fb2760 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560203fb92d0 con 0x560203fb1e40 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4b002e1b80 10 client.0 _ll_register_callbacks cb 0x560203e92330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4aeea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203fb8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4aeea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203fb8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37898/0 (socket says 172.21.15.63:37898) 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.592+0000 7f4aeea00700 1 -- 172.21.15.63:0/3795924241 learned_addr learned my addr 172.21.15.63:0/3795924241 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aee000700 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fb57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aef400700 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb2760 0x560203fb5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aeea00700 1 -- 172.21.15.63:0/3795924241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 msgr2=0x560203fb57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aeea00700 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fb57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aeea00700 1 -- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb2760 msgr2=0x560203fb5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aeea00700 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb2760 0x560203fb5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aeea00700 1 -- 172.21.15.63:0/3795924241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560203ec43d0 con 0x560203fac8d0 2024-05-11T13:33:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4aeea00700 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203fb8450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0010ec0 tx=0x7f4ae0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b2376359deb1c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.593+0000 7f4ad8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4ad8c00700 1 -- 172.21.15.63:0/3795924241 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011440 con 0x560203fac8d0 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4ad8c00700 1 -- 172.21.15.63:0/3795924241 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4ae00115e0 con 0x560203fac8d0 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4adc01c070 con 0x560203fac8d0 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4ad8c00700 1 -- 172.21.15.63:0/3795924241 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ae002d050 con 0x560203fac8d0 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4b002e1b80 10 client.15216 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560203fc3a20 con 0x560203fac8d0 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4ad8c00700 1 -- 172.21.15.63:0/3795924241 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4adc01c070 con 0x560203fac8d0 2024-05-11T13:33:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.594+0000 7f4ad8c00700 1 -- 172.21.15.63:0/3795924241 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ae001f7f0 con 0x560203fac8d0 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.595+0000 7f4b002e1b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.595+0000 7f4b002e1b80 10 client.15216 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.595+0000 7f4b002e1b80 20 client.15216 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.595+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x560203fc60f0 0x560203fc7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.595+0000 7f4b002e1b80 4 client.15216 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.595+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 --> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560203f79010 con 0x560203fc60f0 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.595+0000 7f4aee000700 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x560203fc60f0 0x560203fc7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.596+0000 7f4aee000700 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x560203fc60f0 0x560203fc7560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ede0ef355d714c1b server_cookie=861691afcf154050 in_seq=0 out_seq=0 2024-05-11T13:33:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.596+0000 7f4ad8c00700 10 client.15216 ms_handle_connect on v2:172.21.15.71:6834/1316489545 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.598+0000 7f4ad8c00700 1 -- 172.21.15.63:0/3795924241 <== mds.2 v2:172.21.15.71:6834/1316489545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560203f79010 con 0x560203fc60f0 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.598+0000 7f4ad8c00700 10 client.15216 handle_command_reply: tid=0 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 client.15216 shutdown 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 msgr2=0x560203fb8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203fb8450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0010ec0 tx=0x7f4ae0004750 comp rx=0 tx=0).stop 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 shutdown_connections 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fb2760 0x560203fb5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x560203fc60f0 0x560203fc7560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fac8d0 0x560203fb8450 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3795924241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fb57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4ad8c00700 0 client.15216 ms_handle_reset on v2:172.21.15.71:6834/1316489545 2024-05-11T13:33:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 >> 172.21.15.63:0/3795924241 conn(0x560203edab10 msgr2=0x560203fab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 shutdown_connections 2024-05-11T13:33:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3795924241 wait complete. 2024-05-11T13:33:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.599+0000 7f4b002e1b80 20 client.15216 trim_cache size 0 max 16384 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 Processor -- start 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 -- start start 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fa78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb1e40 0x560203fabc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560203fc58a0 con 0x560203fb1e40 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560203fc5f70 con 0x560203fac8d0 2024-05-11T13:33:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.623+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560203fc6640 con 0x560203fb2760 2024-05-11T13:33:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb1e40 0x560203fabc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb1e40 0x560203fabc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37902/0 (socket says 172.21.15.63:37902) 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 -- 172.21.15.63:0/3993255953 learned_addr learned my addr 172.21.15.63:0/3993255953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aeea00700 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fa78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 -- 172.21.15.63:0/3993255953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 msgr2=0x560203fac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aef400700 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 -- 172.21.15.63:0/3993255953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 msgr2=0x560203fa78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fa78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.624+0000 7f4aee000700 1 -- 172.21.15.63:0/3993255953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560203fb92e0 con 0x560203fb1e40 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.625+0000 7f4aee000700 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb1e40 0x560203fabc80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0058e0 tx=0x7f4adc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eb158e0bfc9044 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.625+0000 7f4adb400700 1 -- 172.21.15.63:0/3993255953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc005580 con 0x560203fb1e40 2024-05-11T13:33:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.625+0000 7f4adb400700 1 -- 172.21.15.63:0/3993255953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4adc01c070 con 0x560203fb1e40 2024-05-11T13:33:32.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.625+0000 7f4adb400700 1 -- 172.21.15.63:0/3993255953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0317a0 con 0x560203fb1e40 2024-05-11T13:33:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3993255953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb1e40 msgr2=0x560203fabc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb1e40 0x560203fabc80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0058e0 tx=0x7f4adc01a040 comp rx=0 tx=0).stop 2024-05-11T13:33:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3993255953 shutdown_connections 2024-05-11T13:33:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb2760 0x560203fac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fa78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/3993255953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb1e40 0x560203fabc80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3993255953 >> 172.21.15.63:0/3993255953 conn(0x560203edab10 msgr2=0x560203fc4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3993255953 shutdown_connections 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.626+0000 7f4b002e1b80 1 -- 172.21.15.63:0/3993255953 wait complete. 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.627+0000 7f4b002e1b80 1 -- start start 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.627+0000 7f4b002e1b80 1 Processor -- start 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.627+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fcdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4b002e1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb2760 0x560203fce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560203fc73e0 con 0x560203fb2760 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560203fc6d10 con 0x560203fac8d0 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4b002e1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560203fc7ab0 con 0x560203fb1e40 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4b002e1b80 10 client.0 _ll_register_callbacks cb 0x560203e92330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:32.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb2760 0x560203fce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb2760 0x560203fce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37918/0 (socket says 172.21.15.63:37918) 2024-05-11T13:33:32.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 -- 172.21.15.63:0/1373787451 learned_addr learned my addr 172.21.15.63:0/1373787451 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:32.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aeea00700 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fcdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aee000700 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 -- 172.21.15.63:0/1373787451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 msgr2=0x560203fce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 -- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 msgr2=0x560203fcdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fcdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.628+0000 7f4aef400700 1 -- 172.21.15.63:0/1373787451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560203fb6e20 con 0x560203fb2760 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4aef400700 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb2760 0x560203fce770 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80df4e0 tx=0x7f4ae80a8c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eb0dc047fe9607 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4ad8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4ad8c00700 1 -- 172.21.15.63:0/1373787451 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80c3060 con 0x560203fb2760 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4adc01c070 con 0x560203fb2760 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4ad8c00700 1 -- 172.21.15.63:0/1373787451 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4ae80a9470 con 0x560203fb2760 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4ad8c00700 1 -- 172.21.15.63:0/1373787451 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ae80ca050 con 0x560203fb2760 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4b002e1b80 10 client.15222 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:32.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.629+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56020403a120 con 0x560203fb2760 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4ad8c00700 1 -- 172.21.15.63:0/1373787451 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4adc01c070 con 0x560203fb2760 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4ad8c00700 1 -- 172.21.15.63:0/1373787451 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ae80c5ce0 con 0x560203fb2760 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4b002e1b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4b002e1b80 10 client.15222 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4b002e1b80 20 client.15222 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x56020403ecf0 0x56020403da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4b002e1b80 4 client.15222 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.630+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 --> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] -- command(tid 0: {"prefix": "status"}) -- 0x560203f0c3b0 con 0x56020403ecf0 2024-05-11T13:33:32.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.631+0000 7f4aeea00700 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x56020403ecf0 0x56020403da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:32.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4aeea00700 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x56020403ecf0 0x56020403da00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c1eb4e357731ff2 server_cookie=bf4c1b523dfdeec3 in_seq=0 out_seq=0 2024-05-11T13:33:32.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4ad8c00700 10 client.15222 ms_handle_connect on v2:172.21.15.71:6834/1316489545 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4ad8c00700 1 -- 172.21.15.63:0/1373787451 <== mds.2 v2:172.21.15.71:6834/1316489545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560203f0c3b0 con 0x56020403ecf0 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4ad8c00700 10 client.15222 handle_command_reply: tid=0 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 client.15222 shutdown 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb2760 msgr2=0x560203fce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb2760 0x560203fce770 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80df4e0 tx=0x7f4ae80a8c30 comp rx=0 tx=0).stop 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 shutdown_connections 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x56020403ecf0 0x56020403da00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560203fb2760 0x560203fce770 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560203fac8d0 0x560203fcdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 --2- 172.21.15.63:0/1373787451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560203fb1e40 0x560203fce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.632+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 >> 172.21.15.63:0/1373787451 conn(0x560203edab10 msgr2=0x560203fabba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.633+0000 7f4ad8c00700 0 client.15222 ms_handle_reset on v2:172.21.15.71:6834/1316489545 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.633+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 shutdown_connections 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.633+0000 7f4b002e1b80 1 -- 172.21.15.63:0/1373787451 wait complete. 2024-05-11T13:33:32.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.633+0000 7f4b002e1b80 20 client.15222 trim_cache size 0 max 16384 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 2, 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34427, 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 216.907214211, 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:33:32.636 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:33:32.637 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:33:32.637 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 343.43027176200002 2024-05-11T13:33:32.637 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:33:32.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4ad40b2900 msgr2=0x7f4ad40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4ad40b2900 0x7f4ad40b4dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0005fd0 tx=0x7f4ae00058e0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0109540 msgr2=0x7f4af00762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:32.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0109540 0x7f4af00762d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8007ee0 tx=0x7f4ae800d3b0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 shutdown_connections 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f4ad40b2900 0x7f4ad40b4dc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4af0109540 0x7f4af00762d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4af0103490 0x7f4af0075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 --2- 172.21.15.63:0/2662932290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af0103e50 0x7f4af0075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 >> 172.21.15.63:0/2662932290 conn(0x7f4af00fef60 msgr2=0x7f4af0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 shutdown_connections 2024-05-11T13:33:32.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:32.634+0000 7f4aefe00700 1 -- 172.21.15.63:0/2662932290 wait complete. 2024-05-11T13:33:33.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:32 smithi165 ceph-mon[33235]: pgmap v526: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:33.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:32 smithi063 ceph-mon[29809]: pgmap v526: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:33.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:32 smithi071 ceph-mon[33293]: pgmap v526: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-11T13:33:33.602 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-11T13:33:34.084 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:35.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:34 smithi063 ceph-mon[29809]: pgmap v527: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:35.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:34 smithi165 ceph-mon[33235]: pgmap v527: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:35.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:34 smithi071 ceph-mon[33293]: pgmap v527: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 -- 172.21.15.63:0/295538179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06c480 msgr2=0x7f5cac1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/295538179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06c480 0x7f5cac1103d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b0a0 tx=0x7f5ca400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 -- 172.21.15.63:0/295538179 shutdown_connections 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/295538179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06c480 0x7f5cac1103d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/295538179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cac1095b0 0x7f5cac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/295538179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cac06bac0 0x7f5cac06bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 -- 172.21.15.63:0/295538179 >> 172.21.15.63:0/295538179 conn(0x7f5cac0793c0 msgr2=0x7f5cac0797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 -- 172.21.15.63:0/295538179 shutdown_connections 2024-05-11T13:33:36.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.542+0000 7f5cb0e00700 1 -- 172.21.15.63:0/295538179 wait complete. 2024-05-11T13:33:36.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.543+0000 7f5cb0e00700 1 Processor -- start 2024-05-11T13:33:36.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.543+0000 7f5cb0e00700 1 -- start start 2024-05-11T13:33:36.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5cb0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06bac0 0x7f5cac19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5cb0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cac06c480 0x7f5cac19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5cb0e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cac1095b0 0x7f5cac1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5cb0e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1135f0 con 0x7f5cac06bac0 2024-05-11T13:33:36.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5cb0e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac113470 con 0x7f5cac06c480 2024-05-11T13:33:36.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5cb0e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac113770 con 0x7f5cac1095b0 2024-05-11T13:33:36.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5ca9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06bac0 0x7f5cac19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5ca9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06bac0 0x7f5cac19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37936/0 (socket says 172.21.15.63:37936) 2024-05-11T13:33:36.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.544+0000 7f5ca9e00700 1 -- 172.21.15.63:0/1280336406 learned_addr learned my addr 172.21.15.63:0/1280336406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5caa800700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cac1095b0 0x7f5cac1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5ca9400700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cac06c480 0x7f5cac19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5ca9e00700 1 -- 172.21.15.63:0/1280336406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cac1095b0 msgr2=0x7f5cac1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cac1095b0 0x7f5cac1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5ca9e00700 1 -- 172.21.15.63:0/1280336406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cac06c480 msgr2=0x7f5cac19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cac06c480 0x7f5cac19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5ca9e00700 1 -- 172.21.15.63:0/1280336406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac1a0d80 con 0x7f5cac06bac0 2024-05-11T13:33:36.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.545+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06bac0 0x7f5cac19bdc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00d950 tx=0x7f5c9c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:36.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.546+0000 7f5ca2a00700 1 -- 172.21.15.63:0/1280336406 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c016070 con 0x7f5cac06bac0 2024-05-11T13:33:36.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.546+0000 7f5ca2a00700 1 -- 172.21.15.63:0/1280336406 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c9c010460 con 0x7f5cac06bac0 2024-05-11T13:33:36.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.546+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac1a1070 con 0x7f5cac06bac0 2024-05-11T13:33:36.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.546+0000 7f5ca2a00700 1 -- 172.21.15.63:0/1280336406 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00f620 con 0x7f5cac06bac0 2024-05-11T13:33:36.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.546+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1a88f0 con 0x7f5cac06bac0 2024-05-11T13:33:36.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.549+0000 7f5ca2a00700 1 -- 172.21.15.63:0/1280336406 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5c9c019070 con 0x7f5cac06bac0 2024-05-11T13:33:36.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 Processor -- start 2024-05-11T13:33:36.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 -- start start 2024-05-11T13:33:36.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5ca2a00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5c900b2880 0x7f5c900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e08f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08fa40 0x557a9e093550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e093a90 0x557a9e093ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557a9dfe19d0 con 0x557a9e093a90 2024-05-11T13:33:36.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557a9df7f710 con 0x557a9e08fa40 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557a9dfdd990 con 0x557a9e08f120 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.551+0000 7f5ca2a00700 1 -- 172.21.15.63:0/1280336406 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5c9c0e21e0 con 0x7f5cac06bac0 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.552+0000 7f5caa800700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5c900b2880 0x7f5c900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.552+0000 7f5ca9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e08f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.552+0000 7f5ca9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e08f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:34706/0 (socket says 172.21.15.63:34706) 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.552+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2347604497 learned_addr learned my addr 172.21.15.63:0/2347604497 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.552+0000 7f5ca9400700 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08fa40 0x557a9e093550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.552+0000 7f5caa800700 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e093a90 0x557a9e093ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.552+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2347604497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08fa40 msgr2=0x557a9e093550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.553+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08fa40 0x557a9e093550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.553+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2347604497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e093a90 msgr2=0x557a9e093ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.553+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e093a90 0x557a9e093ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.553+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2347604497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9df2e880 con 0x557a9e08f120 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.553+0000 7f5caa800700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5c900b2880 0x7f5c900b4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009850 tx=0x7f5ca4009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.553+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e08f500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00bfd0 tx=0x7f5c9c0a8be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1143dcc5b79734f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.553+0000 7f5c97400700 1 -- 172.21.15.63:0/2347604497 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c020030 con 0x557a9e08f120 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.554+0000 7f5c97400700 1 -- 172.21.15.63:0/2347604497 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c9c0a8f00 con 0x557a9e08f120 2024-05-11T13:33:36.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.554+0000 7f5c97400700 1 -- 172.21.15.63:0/2347604497 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0c99e0 con 0x557a9e08f120 2024-05-11T13:33:36.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.554+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2347604497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 msgr2=0x557a9e08f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.554+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e08f500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00bfd0 tx=0x7f5c9c0a8be0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.555+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2347604497 shutdown_connections 2024-05-11T13:33:36.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.555+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e093a90 0x557a9e093ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.555+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e08f500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.555+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2347604497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08fa40 0x557a9e093550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.555+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2347604497 >> 172.21.15.63:0/2347604497 conn(0x557a9dfbdb10 msgr2=0x557a9e089860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:36.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.555+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2347604497 shutdown_connections 2024-05-11T13:33:36.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.555+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2347604497 wait complete. 2024-05-11T13:33:36.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 -- start start 2024-05-11T13:33:36.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 Processor -- start 2024-05-11T13:33:36.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08f120 0x557a9e09af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e0982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e093a90 0x557a9e098830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557a9e09bca0 con 0x557a9e08fa40 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557a9df47ad0 con 0x557a9e08f120 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557a9e09be20 con 0x557a9e093a90 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.556+0000 7f5cbb799b80 10 client.0 _ll_register_callbacks cb 0x557a9df88c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e0982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e0982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37952/0 (socket says 172.21.15.63:37952) 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 -- 172.21.15.63:0/1640074934 learned_addr learned my addr 172.21.15.63:0/1640074934 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08f120 0x557a9e09af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5caa800700 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e093a90 0x557a9e098830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 -- 172.21.15.63:0/1640074934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e093a90 msgr2=0x557a9e098830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e093a90 0x557a9e098830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 -- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08f120 msgr2=0x557a9e09af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08f120 0x557a9e09af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.557+0000 7f5ca9400700 1 -- 172.21.15.63:0/1640074934 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9dfa73d0 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5ca9400700 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e0982f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c98005c50 tx=0x7f5c98012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86d372df2d1ebc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5c8be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5c8be00700 1 -- 172.21.15.63:0/1640074934 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980105e0 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5c8be00700 1 -- 172.21.15.63:0/1640074934 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5c98010780 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9c0a8f00 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5c8be00700 1 -- 172.21.15.63:0/1640074934 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c98010980 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5cbb799b80 10 client.15234 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.558+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a9e0a65b0 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5c8be00700 1 -- 172.21.15.63:0/1640074934 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c9c0a8f00 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5c8be00700 1 -- 172.21.15.63:0/1640074934 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c98019070 con 0x557a9e08fa40 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5cbb799b80 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5cbb799b80 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T13:33:36.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5cbb799b80 20 client.15234 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:36.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e0aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5cbb799b80 4 client.15234 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:33:36.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.559+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 --> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a9e05c160 con 0x557a9e0a8c50 2024-05-11T13:33:36.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.560+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e0aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.561+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e0aa0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5970d7be1905f943 server_cookie=39b6ca203845cd65 in_seq=0 out_seq=0 2024-05-11T13:33:36.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.561+0000 7f5c8be00700 10 client.15234 ms_handle_connect on v2:172.21.15.71:6836/136473158 2024-05-11T13:33:36.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5c8be00700 1 -- 172.21.15.63:0/1640074934 <== mds.1 v2:172.21.15.71:6836/136473158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a9e05c160 con 0x557a9e0a8c50 2024-05-11T13:33:36.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5c8be00700 10 client.15234 handle_command_reply: tid=0 2024-05-11T13:33:36.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 client.15234 shutdown 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 msgr2=0x557a9e0982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e0982f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c98005c50 tx=0x7f5c98012730 comp rx=0 tx=0).stop 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 shutdown_connections 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e093a90 0x557a9e098830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e0aa0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5c8be00700 0 client.15234 ms_handle_reset on v2:172.21.15.71:6836/136473158 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e08f120 0x557a9e09af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/1640074934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e0982f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.563+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 >> 172.21.15.63:0/1640074934 conn(0x557a9dfbdb10 msgr2=0x557a9e084370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:36.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.564+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 shutdown_connections 2024-05-11T13:33:36.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.564+0000 7f5cbb799b80 1 -- 172.21.15.63:0/1640074934 wait complete. 2024-05-11T13:33:36.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.564+0000 7f5cbb799b80 20 client.15234 trim_cache size 0 max 16384 2024-05-11T13:33:36.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.587+0000 7f5cbb799b80 1 Processor -- start 2024-05-11T13:33:36.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.587+0000 7f5cbb799b80 1 -- start start 2024-05-11T13:33:36.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e083ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e084020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e084560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0a8400 con 0x557a9e08fa40 2024-05-11T13:33:36.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0a8ad0 con 0x557a9e093a90 2024-05-11T13:33:36.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0a91a0 con 0x557a9e08f120 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5ca9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e084020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5ca9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e084020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37956/0 (socket says 172.21.15.63:37956) 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5ca9400700 1 -- 172.21.15.63:0/897341528 learned_addr learned my addr 172.21.15.63:0/897341528 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5caa800700 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e084560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.588+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e083ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5ca9400700 1 -- 172.21.15.63:0/897341528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 msgr2=0x557a9e083ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5ca9400700 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e083ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5ca9400700 1 -- 172.21.15.63:0/897341528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 msgr2=0x557a9e084560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5ca9400700 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e084560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5ca9400700 1 -- 172.21.15.63:0/897341528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9e08a950 con 0x557a9e08fa40 2024-05-11T13:33:36.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e083ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:33:36.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5ca9400700 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e084020 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b770 tx=0x7f5c9801a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ea08f4ee3c9efc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:36.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5c96a00700 1 -- 172.21.15.63:0/897341528 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9802e070 con 0x557a9e08fa40 2024-05-11T13:33:36.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5c96a00700 1 -- 172.21.15.63:0/897341528 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c980105e0 con 0x557a9e08fa40 2024-05-11T13:33:36.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.589+0000 7f5c96a00700 1 -- 172.21.15.63:0/897341528 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9801ba40 con 0x557a9e08fa40 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 -- 172.21.15.63:0/897341528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 msgr2=0x557a9e084020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e084020 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b770 tx=0x7f5c9801a3e0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 -- 172.21.15.63:0/897341528 shutdown_connections 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e084560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08f120 0x557a9e083ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/897341528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08fa40 0x557a9e084020 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 -- 172.21.15.63:0/897341528 >> 172.21.15.63:0/897341528 conn(0x557a9dfbdb10 msgr2=0x557a9e083360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 -- 172.21.15.63:0/897341528 shutdown_connections 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.590+0000 7f5cbb799b80 1 -- 172.21.15.63:0/897341528 wait complete. 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.591+0000 7f5cbb799b80 1 -- start start 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.591+0000 7f5cbb799b80 1 Processor -- start 2024-05-11T13:33:36.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.591+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08f120 0x557a9e0ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.591+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08fa40 0x557a9e0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5cbb799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e0adc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0a9f40 con 0x557a9e08f120 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0a9870 con 0x557a9e093a90 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5cbb799b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0aa610 con 0x557a9e08fa40 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5cbb799b80 10 client.0 _ll_register_callbacks cb 0x557a9df88c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08f120 0x557a9e0ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08f120 0x557a9e0ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37962/0 (socket says 172.21.15.63:37962) 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2909856367 learned_addr learned my addr 172.21.15.63:0/2909856367 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9400700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08fa40 0x557a9e0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5caa800700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e0adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2909856367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08fa40 msgr2=0x557a9e0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08fa40 0x557a9e0ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 msgr2=0x557a9e0adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e0adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9e00700 1 -- 172.21.15.63:0/2909856367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9e0b1b50 con 0x557a9e08f120 2024-05-11T13:33:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5caa800700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e0adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:33:36.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.592+0000 7f5ca9400700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08fa40 0x557a9e0ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:33:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5ca9e00700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08f120 0x557a9e0ad1d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0a5470 tx=0x7f5c9c0a52b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f787592bd14f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5c8be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5c8be00700 1 -- 172.21.15.63:0/2909856367 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c020040 con 0x557a9e08f120 2024-05-11T13:33:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5c8be00700 1 -- 172.21.15.63:0/2909856367 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5c9c010930 con 0x557a9e08f120 2024-05-11T13:33:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c980105e0 con 0x557a9e08f120 2024-05-11T13:33:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5c8be00700 1 -- 172.21.15.63:0/2909856367 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0a8f00 con 0x557a9e08f120 2024-05-11T13:33:36.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5cbb799b80 10 client.15240 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.593+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a9e1144e0 con 0x557a9e08f120 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5c8be00700 1 -- 172.21.15.63:0/2909856367 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c980105e0 con 0x557a9e08f120 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5c8be00700 1 -- 172.21.15.63:0/2909856367 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c9c0b1e50 con 0x557a9e08f120 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5cbb799b80 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5cbb799b80 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5cbb799b80 20 client.15240 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e1179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5cbb799b80 4 client.15240 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.594+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 --> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] -- command(tid 0: {"prefix": "status"}) -- 0x557a9dfef3b0 con 0x557a9e0a8c50 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.595+0000 7f5ca9400700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e1179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.595+0000 7f5ca9400700 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e1179d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc255faa22c50708 server_cookie=754e413ff4045d12 in_seq=0 out_seq=0 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.595+0000 7f5c8be00700 10 client.15240 ms_handle_connect on v2:172.21.15.71:6836/136473158 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5c8be00700 1 -- 172.21.15.63:0/2909856367 <== mds.1 v2:172.21.15.71:6836/136473158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557a9dfef3b0 con 0x557a9e0a8c50 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5c8be00700 10 client.15240 handle_command_reply: tid=0 2024-05-11T13:33:36.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 client.15240 shutdown 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08f120 msgr2=0x557a9e0ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08f120 0x557a9e0ad1d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0a5470 tx=0x7f5c9c0a52b0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 shutdown_connections 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x557a9e0a8c50 0x557a9e1179d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a9e093a90 0x557a9e0adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a9e08f120 0x557a9e0ad1d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 --2- 172.21.15.63:0/2909856367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9e08fa40 0x557a9e0ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5c8be00700 0 client.15240 ms_handle_reset on v2:172.21.15.71:6836/136473158 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 >> 172.21.15.63:0/2909856367 conn(0x557a9dfbdb10 msgr2=0x557a9e095e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.596+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 shutdown_connections 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.597+0000 7f5cbb799b80 1 -- 172.21.15.63:0/2909856367 wait complete. 2024-05-11T13:33:36.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.597+0000 7f5cbb799b80 20 client.15240 trim_cache size 0 max 16384 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 1, 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34451, 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 221.895752456, 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:33:36.600 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 335.15850749499998 2024-05-11T13:33:36.601 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:33:36.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.597+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5c900b2880 msgr2=0x7f5c900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.597+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5c900b2880 0x7f5c900b4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009850 tx=0x7f5ca4009470 comp rx=0 tx=0).stop 2024-05-11T13:33:36.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.597+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06bac0 msgr2=0x7f5cac19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:36.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.597+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06bac0 0x7f5cac19bdc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00d950 tx=0x7f5c9c00dd10 comp rx=0 tx=0).stop 2024-05-11T13:33:36.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 shutdown_connections 2024-05-11T13:33:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f5c900b2880 0x7f5c900b4d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cac1095b0 0x7f5cac1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cac06bac0 0x7f5cac19bdc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 --2- 172.21.15.63:0/1280336406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cac06c480 0x7f5cac19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 >> 172.21.15.63:0/1280336406 conn(0x7f5cac0793c0 msgr2=0x7f5cac101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 shutdown_connections 2024-05-11T13:33:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:36.598+0000 7f5cb0e00700 1 -- 172.21.15.63:0/1280336406 wait complete. 2024-05-11T13:33:37.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:36 smithi165 ceph-mon[33235]: pgmap v528: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:37.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:36 smithi063 ceph-mon[29809]: pgmap v528: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:37.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:36 smithi071 ceph-mon[33293]: pgmap v528: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-11T13:33:37.573 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-11T13:33:38.067 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:39.263 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:38 smithi063 ceph-mon[29809]: pgmap v529: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:39.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:38 smithi165 ceph-mon[33235]: pgmap v529: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:39.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:38 smithi071 ceph-mon[33293]: pgmap v529: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-11T13:33:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.426+0000 7f1708000700 1 -- 172.21.15.63:0/981182908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 msgr2=0x7f17001081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.426+0000 7f1708000700 1 --2- 172.21.15.63:0/981182908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f17001081d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0099c0 tx=0x7f16ec009cd0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.427+0000 7f1708000700 1 -- 172.21.15.63:0/981182908 shutdown_connections 2024-05-11T13:33:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.427+0000 7f1708000700 1 --2- 172.21.15.63:0/981182908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1700108900 0x7f1700110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.427+0000 7f1708000700 1 --2- 172.21.15.63:0/981182908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1700103800 0x7f1700103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.427+0000 7f1708000700 1 --2- 172.21.15.63:0/981182908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f17001081d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.427+0000 7f1708000700 1 -- 172.21.15.63:0/981182908 >> 172.21.15.63:0/981182908 conn(0x7f170007a6b0 msgr2=0x7f170007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.427+0000 7f1708000700 1 -- 172.21.15.63:0/981182908 shutdown_connections 2024-05-11T13:33:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.428+0000 7f1708000700 1 -- 172.21.15.63:0/981182908 wait complete. 2024-05-11T13:33:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.429+0000 7f1708000700 1 Processor -- start 2024-05-11T13:33:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.429+0000 7f1708000700 1 -- start start 2024-05-11T13:33:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.429+0000 7f1708000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1700103800 0x7f170019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.429+0000 7f1708000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f170019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.429+0000 7f1708000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1700108900 0x7f17001a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1708000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17001135d0 con 0x7f17001041b0 2024-05-11T13:33:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1708000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1700113450 con 0x7f1700103800 2024-05-11T13:33:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1708000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1700113750 con 0x7f1700108900 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f170019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f170019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50640/0 (socket says 172.21.15.63:50640) 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 -- 172.21.15.63:0/3909255613 learned_addr learned my addr 172.21.15.63:0/3909255613 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1705600700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1700103800 0x7f170019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1706000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1700108900 0x7f17001a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 -- 172.21.15.63:0/3909255613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1700108900 msgr2=0x7f17001a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1700108900 0x7f17001a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 -- 172.21.15.63:0/3909255613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1700103800 msgr2=0x7f170019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1700103800 0x7f170019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.430+0000 7f1704c00700 1 -- 172.21.15.63:0/3909255613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17001a0d40 con 0x7f17001041b0 2024-05-11T13:33:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.431+0000 7f1704c00700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f170019c270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.431+0000 7f16fe000700 1 -- 172.21.15.63:0/3909255613 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec027070 con 0x7f17001041b0 2024-05-11T13:33:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.431+0000 7f16fe000700 1 -- 172.21.15.63:0/3909255613 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ec00fda0 con 0x7f17001041b0 2024-05-11T13:33:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.431+0000 7f16fe000700 1 -- 172.21.15.63:0/3909255613 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec022e50 con 0x7f17001041b0 2024-05-11T13:33:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.431+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17001a0fd0 con 0x7f17001041b0 2024-05-11T13:33:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.431+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17001a88b0 con 0x7f17001041b0 2024-05-11T13:33:40.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.435+0000 7f16fe000700 1 -- 172.21.15.63:0/3909255613 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f16ec02a070 con 0x7f17001041b0 2024-05-11T13:33:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 Processor -- start 2024-05-11T13:33:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 -- start start 2024-05-11T13:33:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf60010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561aadeac9d0 con 0x561aadf5a920 2024-05-11T13:33:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561aade4a710 con 0x561aadf60550 2024-05-11T13:33:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561aadea8990 con 0x561aadf5fc30 2024-05-11T13:33:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f16fe000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f16e80b2a20 0x7f16e80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1704c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f16fe000700 1 -- 172.21.15.63:0/3909255613 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f16ec0e31c0 con 0x7f17001041b0 2024-05-11T13:33:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1704c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50646/0 (socket says 172.21.15.63:50646) 2024-05-11T13:33:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1706000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f16e80b2a20 0x7f16e80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.437+0000 7f1704c00700 1 -- 172.21.15.63:0/162364780 learned_addr learned my addr 172.21.15.63:0/162364780 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1706000700 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf60010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1705600700 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1704c00700 1 -- 172.21.15.63:0/162364780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 msgr2=0x561aadf60010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1704c00700 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf60010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1704c00700 1 -- 172.21.15.63:0/162364780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 msgr2=0x561aadf5a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1704c00700 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf5a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1704c00700 1 -- 172.21.15.63:0/162364780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aaddf9880 con 0x561aadf5a920 2024-05-11T13:33:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.438+0000 7f1704c00700 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5ad80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0a64e0 tx=0x7f16ec09ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b6599abe9093d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.439+0000 7f1706000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f16e80b2a20 0x7f16e80b4ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f16f80060b0 tx=0x7f16f8009120 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.439+0000 7f16f2a00700 1 -- 172.21.15.63:0/162364780 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec031040 con 0x561aadf5a920 2024-05-11T13:33:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.439+0000 7f16f2a00700 1 -- 172.21.15.63:0/162364780 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ec0a9f70 con 0x561aadf5a920 2024-05-11T13:33:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.439+0000 7f16f2a00700 1 -- 172.21.15.63:0/162364780 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0aa460 con 0x561aadf5a920 2024-05-11T13:33:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 -- 172.21.15.63:0/162364780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 msgr2=0x561aadf5ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5ad80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0a64e0 tx=0x7f16ec09ed30 comp rx=0 tx=0).stop 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 -- 172.21.15.63:0/162364780 shutdown_connections 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5ad80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf60010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/162364780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf5a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 -- 172.21.15.63:0/162364780 >> 172.21.15.63:0/162364780 conn(0x561aade88b10 msgr2=0x561aadf54bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 -- 172.21.15.63:0/162364780 shutdown_connections 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 -- 172.21.15.63:0/162364780 wait complete. 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.440+0000 7f1712a0bb80 1 -- start start 2024-05-11T13:33:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 1 Processor -- start 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf63560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf63aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561aadf66d80 con 0x561aadf5a920 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561aade12ad0 con 0x561aadf60550 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561aadf66f00 con 0x561aadf5fc30 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.441+0000 7f1712a0bb80 10 client.0 _ll_register_callbacks cb 0x561aade40330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50662/0 (socket says 172.21.15.63:50662) 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 -- 172.21.15.63:0/2945045670 learned_addr learned my addr 172.21.15.63:0/2945045670 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1706000700 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf63aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1704c00700 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf63560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 -- 172.21.15.63:0/2945045670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 msgr2=0x561aadf63560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf63560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 -- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 msgr2=0x561aadf63aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf63aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.442+0000 7f1705600700 1 -- 172.21.15.63:0/2945045670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aade723d0 con 0x561aadf5a920 2024-05-11T13:33:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f1705600700 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf661c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f16f400b960 tx=0x7f16f400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d13c16183df7a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f16e7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f16e7400700 1 -- 172.21.15.63:0/2945045670 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4003ca0 con 0x561aadf5a920 2024-05-11T13:33:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f16e7400700 1 -- 172.21.15.63:0/2945045670 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f16f4011440 con 0x561aadf5a920 2024-05-11T13:33:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16ec0a9f70 con 0x561aadf5a920 2024-05-11T13:33:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f16e7400700 1 -- 172.21.15.63:0/2945045670 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16f4003e40 con 0x561aadf5a920 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f1712a0bb80 10 client.15252 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.443+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561aadf71500 con 0x561aadf5a920 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f16e7400700 1 -- 172.21.15.63:0/2945045670 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ec0a9f70 con 0x561aadf5a920 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f16e7400700 1 -- 172.21.15.63:0/2945045670 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f16f401b650 con 0x561aadf5a920 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f1712a0bb80 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f1712a0bb80 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f1712a0bb80 20 client.15252 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadf73c00 0x561aadf75070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f1712a0bb80 4 client.15252 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.444+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aadf27010 con 0x561aadf73c00 2024-05-11T13:33:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.445+0000 7f1704c00700 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadf73c00 0x561aadf75070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.446+0000 7f16e7400700 10 client.15252 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:33:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.446+0000 7f1704c00700 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadf73c00 0x561aadf75070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15c976ac6429b1d server_cookie=f0c179b719399cfd in_seq=0 out_seq=0 2024-05-11T13:33:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f16e7400700 1 -- 172.21.15.63:0/2945045670 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561aadf27010 con 0x561aadf73c00 2024-05-11T13:33:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f16e7400700 10 client.15252 handle_command_reply: tid=0 2024-05-11T13:33:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 client.15252 shutdown 2024-05-11T13:33:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 msgr2=0x561aadf661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf661c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f16f400b960 tx=0x7f16f400bd20 comp rx=0 tx=0).stop 2024-05-11T13:33:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 shutdown_connections 2024-05-11T13:33:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadf73c00 0x561aadf75070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf63aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf661c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/2945045670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf63560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 >> 172.21.15.63:0/2945045670 conn(0x561aade88b10 msgr2=0x561aadf4f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f16e7400700 0 client.15252 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:33:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 shutdown_connections 2024-05-11T13:33:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.448+0000 7f1712a0bb80 1 -- 172.21.15.63:0/2945045670 wait complete. 2024-05-11T13:33:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.449+0000 7f1712a0bb80 20 client.15252 trim_cache size 0 max 16384 2024-05-11T13:33:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.472+0000 7f1712a0bb80 1 Processor -- start 2024-05-11T13:33:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.472+0000 7f1712a0bb80 1 -- start start 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf62390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561aadf733b0 con 0x561aadf5a920 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561aadf73a80 con 0x561aadf60550 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561aadf74150 con 0x561aadf5fc30 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1705600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1705600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50672/0 (socket says 172.21.15.63:50672) 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.473+0000 7f1705600700 1 -- 172.21.15.63:0/1725254608 learned_addr learned my addr 172.21.15.63:0/1725254608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1704c00700 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1706000700 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf62390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1705600700 1 -- 172.21.15.63:0/1725254608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 msgr2=0x561aadf61e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1705600700 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf61e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1705600700 1 -- 172.21.15.63:0/1725254608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 msgr2=0x561aadf62390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1705600700 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf62390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1705600700 1 -- 172.21.15.63:0/1725254608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aadf517c0 con 0x561aadf5a920 2024-05-11T13:33:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f1705600700 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5a300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005720 tx=0x7f16f4005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb81951602d6ed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f16f2000700 1 -- 172.21.15.63:0/1725254608 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400b480 con 0x561aadf5a920 2024-05-11T13:33:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.474+0000 7f16f2000700 1 -- 172.21.15.63:0/1725254608 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16f402e070 con 0x561aadf5a920 2024-05-11T13:33:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.475+0000 7f16f2000700 1 -- 172.21.15.63:0/1725254608 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4029c90 con 0x561aadf5a920 2024-05-11T13:33:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.475+0000 7f1712a0bb80 1 -- 172.21.15.63:0/1725254608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 msgr2=0x561aadf5a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.475+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5a300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005720 tx=0x7f16f4005750 comp rx=0 tx=0).stop 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 -- 172.21.15.63:0/1725254608 shutdown_connections 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf60550 0x561aadf62390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5a920 0x561aadf5a300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/1725254608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf5fc30 0x561aadf61e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 -- 172.21.15.63:0/1725254608 >> 172.21.15.63:0/1725254608 conn(0x561aade88b10 msgr2=0x561aadf5d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 -- 172.21.15.63:0/1725254608 shutdown_connections 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 -- 172.21.15.63:0/1725254608 wait complete. 2024-05-11T13:33:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.476+0000 7f1712a0bb80 1 -- start start 2024-05-11T13:33:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 1 Processor -- start 2024-05-11T13:33:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf5a920 0x561aadf62d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5fc30 0x561aadf632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf60550 0x561aadf63800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561aadf74ef0 con 0x561aadf5fc30 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561aadf74820 con 0x561aadf5a920 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561aadf755c0 con 0x561aadf60550 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.477+0000 7f1712a0bb80 10 client.0 _ll_register_callbacks cb 0x561aade40330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5fc30 0x561aadf632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5fc30 0x561aadf632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50682/0 (socket says 172.21.15.63:50682) 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 -- 172.21.15.63:0/886210275 learned_addr learned my addr 172.21.15.63:0/886210275 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1705600700 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf5a920 0x561aadf62d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 -- 172.21.15.63:0/886210275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf60550 msgr2=0x561aadf63800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf60550 0x561aadf63800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 -- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf5a920 msgr2=0x561aadf62d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf5a920 0x561aadf62d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.478+0000 7f1704c00700 1 -- 172.21.15.63:0/886210275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aadf7ca40 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f1704c00700 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5fc30 0x561aadf632c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0aa3f0 tx=0x7f16ec0e3440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923ede0ef1d03ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f16e7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f16e7400700 1 -- 172.21.15.63:0/886210275 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0c4060 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f402e070 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f16e7400700 1 -- 172.21.15.63:0/886210275 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f16ec0a9950 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f16e7400700 1 -- 172.21.15.63:0/886210275 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16ec0ae810 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f1712a0bb80 10 client.15258 fetch_fsmap learned FSMap version 28 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561aadfe7c90 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.479+0000 7f16e7400700 1 -- 172.21.15.63:0/886210275 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16f402e070 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.480+0000 7f16e7400700 1 -- 172.21.15.63:0/886210275 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f16ec0e0930 con 0x561aadf5fc30 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.480+0000 7f1712a0bb80 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.480+0000 7f1712a0bb80 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:33:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.480+0000 7f1712a0bb80 20 client.15258 populate_metadata read hostname 'smithi063' 2024-05-11T13:33:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.480+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadfec6d0 0x561aadfeb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.480+0000 7f1712a0bb80 4 client.15258 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:33:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.480+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "status"}) -- 0x561aadeba3b0 con 0x561aadfec6d0 2024-05-11T13:33:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.481+0000 7f1705600700 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadfec6d0 0x561aadfeb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f1705600700 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadfec6d0 0x561aadfeb3e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7157f3c0c52c443 server_cookie=fd74ebc7987755ef in_seq=0 out_seq=0 2024-05-11T13:33:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f16e7400700 10 client.15258 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:33:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f16e7400700 1 -- 172.21.15.63:0/886210275 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561aadeba3b0 con 0x561aadfec6d0 2024-05-11T13:33:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f16e7400700 10 client.15258 handle_command_reply: tid=0 2024-05-11T13:33:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f1712a0bb80 1 client.15258 shutdown 2024-05-11T13:33:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5fc30 msgr2=0x561aadf632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5fc30 0x561aadf632c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0aa3f0 tx=0x7f16ec0e3440 comp rx=0 tx=0).stop 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.482+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 shutdown_connections 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561aadf60550 0x561aadf63800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561aadfec6d0 0x561aadfeb3e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561aadf5a920 0x561aadf62d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 1 --2- 172.21.15.63:0/886210275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561aadf5fc30 0x561aadf632c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 >> 172.21.15.63:0/886210275 conn(0x561aade88b10 msgr2=0x561aadf61dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f16e7400700 0 client.15258 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 shutdown_connections 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 1 -- 172.21.15.63:0/886210275 wait complete. 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.483+0000 7f1712a0bb80 20 client.15258 trim_cache size 0 max 16384 2024-05-11T13:33:40.484 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 0, 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34472, 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 302.26221138900002, 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 28, 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 90, 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 88, 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 321.44018734700001 2024-05-11T13:33:40.485 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:33:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f16e80b2a20 msgr2=0x7f16e80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f16e80b2a20 0x7f16e80b4ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f16f80060b0 tx=0x7f16f8009120 comp rx=0 tx=0).stop 2024-05-11T13:33:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 msgr2=0x7f170019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f170019c270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec00f460 comp rx=0 tx=0).stop 2024-05-11T13:33:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 shutdown_connections 2024-05-11T13:33:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f16e80b2a20 0x7f16e80b4ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1700108900 0x7f17001a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1700103800 0x7f170019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 --2- 172.21.15.63:0/3909255613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17001041b0 0x7f170019c270 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 >> 172.21.15.63:0/3909255613 conn(0x7f170007a6b0 msgr2=0x7f1700101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.484+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 shutdown_connections 2024-05-11T13:33:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:40.485+0000 7f1708000700 1 -- 172.21.15.63:0/3909255613 wait complete. 2024-05-11T13:33:41.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:40 smithi063 ceph-mon[29809]: pgmap v530: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:41.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:40 smithi165 ceph-mon[33235]: pgmap v530: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:41.431 INFO:teuthology.run_tasks:Running task sequential... 2024-05-11T13:33:41.442 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-11T13:33:41.442 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-11T13:33:41.443 INFO:teuthology.task.print:Enabling mgr modules 2024-05-11T13:33:41.443 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-11T13:33:41.444 INFO:teuthology.task.exec:Executing custom commands... 2024-05-11T13:33:41.444 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:33:41.444 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-11T13:33:41.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:40 smithi071 ceph-mon[33293]: pgmap v530: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-11T13:33:41.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 -- 172.21.15.63:0/1880108967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8108660 msgr2=0x7fd4a810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/1880108967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8108660 0x7fd4a810aa50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009ca0 tx=0x7fd4a002a8c0 comp rx=0 tx=0).stop 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 -- 172.21.15.63:0/1880108967 shutdown_connections 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/1880108967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8108660 0x7fd4a810aa50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/1880108967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4a8105d30 0x7fd4a8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/1880108967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8103400 0x7fd4a81057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 -- 172.21.15.63:0/1880108967 >> 172.21.15.63:0/1880108967 conn(0x7fd4a80fcd50 msgr2=0x7fd4a80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 -- 172.21.15.63:0/1880108967 shutdown_connections 2024-05-11T13:33:41.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.648+0000 7fd4a7e00640 1 -- 172.21.15.63:0/1880108967 wait complete. 2024-05-11T13:33:41.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 Processor -- start 2024-05-11T13:33:41.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 -- start start 2024-05-11T13:33:41.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8103400 0x7fd4a8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:41.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8105d30 0x7fd4a8198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:41.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4a8108660 0x7fd4a819ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:41.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810e330 con 0x7fd4a8103400 2024-05-11T13:33:41.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810e1b0 con 0x7fd4a8105d30 2024-05-11T13:33:41.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.649+0000 7fd4a7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810e4b0 con 0x7fd4a8108660 2024-05-11T13:33:41.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8103400 0x7fd4a8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:41.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8103400 0x7fd4a8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50702/0 (socket says 172.21.15.63:50702) 2024-05-11T13:33:41.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 -- 172.21.15.63:0/2037826492 learned_addr learned my addr 172.21.15.63:0/2037826492 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:41.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6000640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8105d30 0x7fd4a8198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:41.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a7400640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4a8108660 0x7fd4a819ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:41.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 -- 172.21.15.63:0/2037826492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4a8108660 msgr2=0x7fd4a819ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:41.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4a8108660 0x7fd4a819ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:41.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 -- 172.21.15.63:0/2037826492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8105d30 msgr2=0x7fd4a8198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:41.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8105d30 0x7fd4a8198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:41.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.650+0000 7fd4a6a00640 1 -- 172.21.15.63:0/2037826492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a81a0650 con 0x7fd4a8103400 2024-05-11T13:33:41.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.651+0000 7fd4a6a00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8103400 0x7fd4a8198a30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd49800d950 tx=0x7fd49800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:41.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.651+0000 7fd49f400640 1 -- 172.21.15.63:0/2037826492 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498017030 con 0x7fd4a8103400 2024-05-11T13:33:41.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.651+0000 7fd49f400640 1 -- 172.21.15.63:0/2037826492 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd498004020 con 0x7fd4a8103400 2024-05-11T13:33:41.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.651+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a81a0940 con 0x7fd4a8103400 2024-05-11T13:33:41.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.651+0000 7fd49f400640 1 -- 172.21.15.63:0/2037826492 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980055d0 con 0x7fd4a8103400 2024-05-11T13:33:41.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.651+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a81a0e90 con 0x7fd4a8103400 2024-05-11T13:33:41.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.652+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a80fe840 con 0x7fd4a8103400 2024-05-11T13:33:41.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.656+0000 7fd49f400640 1 -- 172.21.15.63:0/2037826492 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd498009a30 con 0x7fd4a8103400 2024-05-11T13:33:41.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.658+0000 7fd49f400640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd48c0b27e0 0x7fd48c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:41.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.658+0000 7fd49f400640 1 -- 172.21.15.63:0/2037826492 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd4980e27b0 con 0x7fd4a8103400 2024-05-11T13:33:41.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.658+0000 7fd4a6000640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd48c0b27e0 0x7fd48c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:41.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.659+0000 7fd4a6000640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd48c0b27e0 0x7fd48c0b4ca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8199f40 tx=0x7fd494005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:41.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.661+0000 7fd49f400640 1 -- 172.21.15.63:0/2037826492 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4980aa0c0 con 0x7fd4a8103400 2024-05-11T13:33:41.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:41.924+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fd4a8063660 con 0x7fd4a8103400 2024-05-11T13:33:42.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.012+0000 7fd49f400640 1 -- 172.21.15.63:0/2037826492 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fd4980a99c0 con 0x7fd4a8103400 2024-05-11T13:33:42.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd48c0b27e0 msgr2=0x7fd48c0b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:42.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd48c0b27e0 0x7fd48c0b4ca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8199f40 tx=0x7fd494005f90 comp rx=0 tx=0).stop 2024-05-11T13:33:42.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8103400 msgr2=0x7fd4a8198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:42.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8103400 0x7fd4a8198a30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd49800d950 tx=0x7fd49800de10 comp rx=0 tx=0).stop 2024-05-11T13:33:42.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 shutdown_connections 2024-05-11T13:33:42.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7fd48c0b27e0 0x7fd48c0b4ca0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:42.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4a8108660 0x7fd4a819ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:42.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8105d30 0x7fd4a8198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:42.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.015+0000 7fd4a7e00640 1 --2- 172.21.15.63:0/2037826492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a8103400 0x7fd4a8198a30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:42.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.016+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 >> 172.21.15.63:0/2037826492 conn(0x7fd4a80fcd50 msgr2=0x7fd4a8103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:42.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.016+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 shutdown_connections 2024-05-11T13:33:42.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:42.016+0000 7fd4a7e00640 1 -- 172.21.15.63:0/2037826492 wait complete. 2024-05-11T13:33:42.047 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-11T13:33:42.061 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-11T13:33:42.061 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-11T13:33:42.061 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-11T13:33:42.061 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T13:33:42.061 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-11T13:33:42.061 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-11T13:33:42.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:42.062 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-11T13:33:42.105 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-11T13:33:42.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:42.106 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T13:33:42.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:42.215 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-11T13:33:42.269 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:42 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2037826492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-11T13:33:42.269 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: ignoring --setuser ceph since I am not root 2024-05-11T13:33:42.270 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: ignoring --setgroup ceph since I am not root 2024-05-11T13:33:42.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2037826492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-11T13:33:42.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: ignoring --setuser ceph since I am not root 2024-05-11T13:33:42.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: ignoring --setgroup ceph since I am not root 2024-05-11T13:33:42.316 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:42 smithi165 ceph-mgr[35110]: -- 172.21.15.165:0/79356748 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e83b645a0 con 0x555e83b38800 2024-05-11T13:33:42.325 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-11T13:33:42.326 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-11T13:33:42.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:42 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2037826492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-11T13:33:42.327 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: ignoring --setuser ceph since I am not root 2024-05-11T13:33:42.327 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: ignoring --setgroup ceph since I am not root 2024-05-11T13:33:42.327 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:42 smithi071 ceph-mgr[34670]: -- 172.21.15.71:0/314803568 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580feba65a0 con 0x5580feb7b000 2024-05-11T13:33:42.327 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:42.326+0000 7f856e4d9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:33:42.592 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:42.334+0000 7fc6ff068200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:33:42.628 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:42.340+0000 7f17c4007200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T13:33:42.628 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:42.627+0000 7f17c4007200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:33:42.777 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:42.933 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:42.781+0000 7f17c4007200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:33:42.953 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:42.585+0000 7f856e4d9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:33:42.953 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:42.714+0000 7f856e4d9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:33:43.013 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:42.591+0000 7fc6ff068200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:33:43.014 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:42.722+0000 7fc6ff068200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T13:33:43.014 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:43 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2037826492' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-11T13:33:43.014 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:43 smithi165 ceph-mon[33235]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-11T13:33:43.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:43 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2037826492' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-11T13:33:43.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:43 smithi063 ceph-mon[29809]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-11T13:33:43.315 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:43.120+0000 7fc6ff068200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:33:43.315 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:43.235+0000 7fc6ff068200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:33:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:43 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2037826492' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-11T13:33:43.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:43 smithi071 ceph-mon[33293]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-11T13:33:43.454 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:43.110+0000 7f856e4d9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:33:43.455 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:43.224+0000 7f856e4d9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:33:43.457 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:43.211+0000 7f17c4007200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T13:33:43.457 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:43.332+0000 7f17c4007200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T13:33:45.065 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:44 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:44.775+0000 7fc6ff068200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:33:45.065 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:45.003+0000 7fc6ff068200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 -- 172.21.15.63:0/2424507113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 msgr2=0x7f6260112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 --2- 172.21.15.63:0/2424507113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f6260112290 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f625800b0a0 tx=0x7f625800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 -- 172.21.15.63:0/2424507113 shutdown_connections 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 --2- 172.21.15.63:0/2424507113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f6260112290 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 --2- 172.21.15.63:0/2424507113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260104460 0x7f626006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 --2- 172.21.15.63:0/2424507113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f626006c160 0x7f626006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 -- 172.21.15.63:0/2424507113 >> 172.21.15.63:0/2424507113 conn(0x7f6260079400 msgr2=0x7f6260079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 -- 172.21.15.63:0/2424507113 shutdown_connections 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.182+0000 7f6265200700 1 -- 172.21.15.63:0/2424507113 wait complete. 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.183+0000 7f6265200700 1 Processor -- start 2024-05-11T13:33:45.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f6265200700 1 -- start start 2024-05-11T13:33:45.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f6265200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f626006c160 0x7f626019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:45.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f6265200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6260104460 0x7f626019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f6265200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f62601a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f6265200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6260115820 con 0x7f626010fea0 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f6265200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62601156a0 con 0x7f6260104460 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f6265200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62601159a0 con 0x7f626006c160 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f625e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f62601a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f625e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f62601a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50744/0 (socket says 172.21.15.63:50744) 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.184+0000 7f625e800700 1 -- 172.21.15.63:0/2181196720 learned_addr learned my addr 172.21.15.63:0/2181196720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f625de00700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f626006c160 0x7f626019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f6257e00700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6260104460 0x7f626019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f625e800700 1 -- 172.21.15.63:0/2181196720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f626006c160 msgr2=0x7f626019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f625e800700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f626006c160 0x7f626019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f625e800700 1 -- 172.21.15.63:0/2181196720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6260104460 msgr2=0x7f626019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f625e800700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6260104460 0x7f626019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f625e800700 1 -- 172.21.15.63:0/2181196720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62601a0d20 con 0x7f626010fea0 2024-05-11T13:33:45.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.185+0000 7f625e800700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f62601a05e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6258006070 tx=0x7f6258004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:45.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.186+0000 7f6256a00700 1 -- 172.21.15.63:0/2181196720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258021070 con 0x7f626010fea0 2024-05-11T13:33:45.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.186+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62601a0fb0 con 0x7f626010fea0 2024-05-11T13:33:45.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.186+0000 7f6256a00700 1 -- 172.21.15.63:0/2181196720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6258007b80 con 0x7f626010fea0 2024-05-11T13:33:45.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.186+0000 7f6256a00700 1 -- 172.21.15.63:0/2181196720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258004700 con 0x7f626010fea0 2024-05-11T13:33:45.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.186+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62601a8890 con 0x7f626010fea0 2024-05-11T13:33:45.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.187+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6260102a90 con 0x7f626010fea0 2024-05-11T13:33:45.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.189+0000 7f6256a00700 1 -- 172.21.15.63:0/2181196720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f6258003a00 con 0x7f626010fea0 2024-05-11T13:33:45.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.191+0000 7f6256a00700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 0x7f62440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:45.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.191+0000 7f625de00700 1 -- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 msgr2=0x7f62440b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:45.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.191+0000 7f6256a00700 1 -- 172.21.15.63:0/2181196720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f625800f020 con 0x7f626010fea0 2024-05-11T13:33:45.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.191+0000 7f625de00700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 0x7f62440b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:33:45.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.196+0000 7f6256a00700 1 -- 172.21.15.63:0/2181196720 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f625802a020 con 0x7f626010fea0 2024-05-11T13:33:45.203 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:44.771+0000 7f856e4d9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:33:45.203 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:44.996+0000 7f856e4d9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:33:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.392+0000 7f625de00700 1 -- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 msgr2=0x7f62440b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.392+0000 7f625de00700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 0x7f62440b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:33:45.393 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:44 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:44.951+0000 7f17c4007200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T13:33:45.393 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:45.194+0000 7f17c4007200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T13:33:45.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.483+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62601a8a60 con 0x7f626010fea0 2024-05-11T13:33:45.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.484+0000 7f6256a00700 1 -- 172.21.15.63:0/2181196720 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f62580ab4f0 con 0x7f626010fea0 2024-05-11T13:33:45.484 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T13:33:45.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.486+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 msgr2=0x7f62440b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:33:45.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 0x7f62440b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 msgr2=0x7f62601a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:45.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f62601a05e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6258006070 tx=0x7f6258004d50 comp rx=0 tx=0).stop 2024-05-11T13:33:45.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 shutdown_connections 2024-05-11T13:33:45.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f62440b28d0 0x7f62440b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f626010fea0 0x7f62601a05e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f626006c160 0x7f626019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 --2- 172.21.15.63:0/2181196720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6260104460 0x7f626019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:45.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 >> 172.21.15.63:0/2181196720 conn(0x7f6260079400 msgr2=0x7f6260101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:45.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 shutdown_connections 2024-05-11T13:33:45.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:45.487+0000 7f6265200700 1 -- 172.21.15.63:0/2181196720 wait complete. 2024-05-11T13:33:45.537 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:45.221+0000 7fc6ff068200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:33:45.537 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:45.211+0000 7f856e4d9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:33:45.683 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:45 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2181196720' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T13:33:45.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:45.420+0000 7f17c4007200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T13:33:45.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:45 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2181196720' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T13:33:45.815 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:45.605+0000 7fc6ff068200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:33:45.815 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:45.731+0000 7fc6ff068200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:33:45.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:45 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2181196720' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T13:33:45.953 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:45.591+0000 7f856e4d9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:33:45.953 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:45.718+0000 7f856e4d9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:33:46.183 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:45.832+0000 7f17c4007200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T13:33:46.184 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:45.964+0000 7f17c4007200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T13:33:46.223 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:45.972+0000 7f856e4d9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:33:46.243 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:45.987+0000 7fc6ff068200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T13:33:46.441 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-11T13:33:46.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:46.442 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-11T13:33:46.486 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T13:33:46.486 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T13:33:46.486 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T13:33:46.486 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 41623sec preferred_lft 41623sec 2024-05-11T13:33:46.487 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-11T13:33:46.488 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-11T13:33:46.488 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T13:33:46.488 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:33:46.488 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:33:46.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-11T13:33:46.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-11T13:33:46.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-11T13:33:46.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T13:33:46.488 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:33:46.500 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:46.225+0000 7f17c4007200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T13:33:46.565 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:46.243+0000 7fc6ff068200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:33:46.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:33:46.702 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:46.223+0000 7f856e4d9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:33:46.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:33:46.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:46.728 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T13:33:46.779 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:46.499+0000 7f17c4007200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T13:33:46.786 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-11T13:33:46.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:46.789 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-11T13:33:46.828 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-11T13:33:46.828 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T13:33:46.828 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T13:33:46.828 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T13:33:46.828 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T13:33:46.828 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:33:46.828 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:33:46.828 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T13:33:46.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T13:33:46.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T13:33:46.829 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:33:46.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:33:47.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:33:47.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:47.070 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-11T13:33:47.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:47.113 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-11T13:33:47.209 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:33:47.209 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:33:47.209 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-11T13:33:47.209 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-11T13:33:47.209 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:33:47.297 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:47.006+0000 7f856e4d9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:33:47.297 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:47.296+0000 7f856e4d9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:33:47.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:33:47.314 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:47.034+0000 7fc6ff068200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:33:47.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:33:47.379 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-11T13:33:47.379 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:33:47.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:33:47.581 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:47.380+0000 7f17c4007200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T13:33:47.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:33:47.620 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T13:33:47.620 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-11T13:33:47.620 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-11T13:33:47.621 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T13:33:47.621 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-11T13:33:47.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T13:33:47.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:33:47.809 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-11T13:33:47.809 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T13:33:47.809 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /tmp/cephfs 2024-05-11T13:33:47.814 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:47.328+0000 7fc6ff068200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:33:47.850 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-11T13:33:47.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T13:33:47.851 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /tmp/cephfs 2024-05-11T13:33:47.933 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:47.691+0000 7f17c4007200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T13:33:47.935 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-11T13:33:48.056 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T13:33:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-11T13:33:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-11T13:33:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-11T13:33:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-11T13:33:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-11T13:33:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-11T13:33:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T13:33:48.104 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/066ac03fb5ca3e1a8dadc84d5f3ab52a45c9209b1c80d02c848ede682c25ff0d/merged 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/72df2fc077412c4e4af695ef9f198bb31d7e3c052c9e160f9bd0d4dc6cd70b33/merged 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/40150c9075cef353a4533debcf913c7d6bd23da52bd9ce25a594c656f983a864/merged 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1719d935211d22b6bd790b1ddba7dfc6d15f8a773beb336cedbdecf76e6562a0/merged 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/4fe368d0473a68a1e52e321a508fa25ac11afa55517a21df5d6b356061efcbdd/merged 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/cf2a505ce6db6359598171dad10a47d6b558f1cb809c02159e9df5a8ee193c11/merged 2024-05-11T13:33:48.105 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8616b01baaa8532458654530a421e33810f567d1d8d4cbc136006fe087780e92/merged 2024-05-11T13:33:48.106 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a6fabb9e7c1965dee1e2844d69bec59f189c2c9c82b01d0ec04b6bbe043a49f/merged 2024-05-11T13:33:48.106 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/01b094eceb45788c05f1b4ff90dc262a42dc5269b472183a2f97b3b780d66a0f/merged 2024-05-11T13:33:48.106 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/03a8bd5cbd061b90d08d64b40852822224f9263e682bd0b20ca549b7b7123159/merged 2024-05-11T13:33:48.106 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-11T13:33:48.106 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-11T13:33:48.106 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-11T13:33:48.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:48.107 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-11T13:33:48.156 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T13:33:48.156 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/de410bc9-fc99-432f-8537-cb4b491e8ce2 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-05-11T13:33:48.161 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-11T13:33:48.203 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:47.842+0000 7f856e4d9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:33:48.203 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:47.968+0000 7f856e4d9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:33:48.227 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-11T13:33:48.266 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T13:33:48.267 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/066ac03fb5ca3e1a8dadc84d5f3ab52a45c9209b1c80d02c848ede682c25ff0d/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/72df2fc077412c4e4af695ef9f198bb31d7e3c052c9e160f9bd0d4dc6cd70b33/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/40150c9075cef353a4533debcf913c7d6bd23da52bd9ce25a594c656f983a864/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1719d935211d22b6bd790b1ddba7dfc6d15f8a773beb336cedbdecf76e6562a0/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/4fe368d0473a68a1e52e321a508fa25ac11afa55517a21df5d6b356061efcbdd/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/cf2a505ce6db6359598171dad10a47d6b558f1cb809c02159e9df5a8ee193c11/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8616b01baaa8532458654530a421e33810f567d1d8d4cbc136006fe087780e92/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a6fabb9e7c1965dee1e2844d69bec59f189c2c9c82b01d0ec04b6bbe043a49f/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/01b094eceb45788c05f1b4ff90dc262a42dc5269b472183a2f97b3b780d66a0f/merged 2024-05-11T13:33:48.268 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/03a8bd5cbd061b90d08d64b40852822224f9263e682bd0b20ca549b7b7123159/merged 2024-05-11T13:33:48.269 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-11T13:33:48.269 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-11T13:33:48.269 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-11T13:33:48.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:48.270 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-11T13:33:48.314 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:47.884+0000 7fc6ff068200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:33:48.314 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:48.013+0000 7fc6ff068200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:33:48.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:2024-05-11T13:33:48.322+0000 7f91e932b480 -1 init, newargv = 0x558be2b703c0 newargc=17 2024-05-11T13:33:48.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[87027]: starting ceph client 2024-05-11T13:33:48.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[87027]: starting fuse 2024-05-11T13:33:48.433 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:48.290+0000 7f17c4007200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T13:33:48.702 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:48.435+0000 7f856e4d9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:33:48.815 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:48.485+0000 7fc6ff068200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:33:48.921 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:48.435+0000 7f17c4007200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T13:33:49.183 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:48.921+0000 7f17c4007200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T13:33:49.202 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:48.893+0000 7f856e4d9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:33:49.203 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:49.015+0000 7f856e4d9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:33:49.314 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:48.937+0000 7fc6ff068200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:33:49.315 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:49.062+0000 7fc6ff068200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:33:49.456 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-11T13:33:49.540 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T13:33:49.591 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-11T13:33:49.591 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-11T13:33:49.591 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-11T13:33:49.591 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-11T13:33:49.591 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-11T13:33:49.591 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-11T13:33:49.592 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-11T13:33:49.593 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/066ac03fb5ca3e1a8dadc84d5f3ab52a45c9209b1c80d02c848ede682c25ff0d/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/72df2fc077412c4e4af695ef9f198bb31d7e3c052c9e160f9bd0d4dc6cd70b33/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/40150c9075cef353a4533debcf913c7d6bd23da52bd9ce25a594c656f983a864/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1719d935211d22b6bd790b1ddba7dfc6d15f8a773beb336cedbdecf76e6562a0/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/4fe368d0473a68a1e52e321a508fa25ac11afa55517a21df5d6b356061efcbdd/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/cf2a505ce6db6359598171dad10a47d6b558f1cb809c02159e9df5a8ee193c11/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8616b01baaa8532458654530a421e33810f567d1d8d4cbc136006fe087780e92/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a6fabb9e7c1965dee1e2844d69bec59f189c2c9c82b01d0ec04b6bbe043a49f/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/01b094eceb45788c05f1b4ff90dc262a42dc5269b472183a2f97b3b780d66a0f/merged 2024-05-11T13:33:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/03a8bd5cbd061b90d08d64b40852822224f9263e682bd0b20ca549b7b7123159/merged 2024-05-11T13:33:49.595 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-11T13:33:49.595 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-11T13:33:49.595 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-11T13:33:49.595 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs 2024-05-11T13:33:49.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:49.596 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-11T13:33:49.642 INFO:teuthology.orchestra.run.smithi063.stdout:114 2024-05-11T13:33:49.643 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-11T13:33:49.643 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/tmp/cephfs" 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-05-11T13:33:49.644 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.0")) 2024-05-11T13:33:49.645 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-05-11T13:33:49.647 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:49.340+0000 7f856e4d9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:33:49.647 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:49.519+0000 7f856e4d9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:33:49.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:49.400+0000 7f17c4007200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T13:33:49.684 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:49.530+0000 7f17c4007200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T13:33:49.699 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:49.389+0000 7fc6ff068200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:33:49.699 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:49.571+0000 7fc6ff068200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:33:49.699 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:49.698+0000 7fc6ff068200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:33:49.847 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.0.87027.asok 2024-05-11T13:33:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T13:33:49.864 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87027.asok 2024-05-11T13:33:49.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:49.865 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87027.asok status 2024-05-11T13:33:49.952 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:49.646+0000 7f856e4d9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:33:50.067 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:49.874+0000 7f17c4007200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "308506802b0bdea4df7050cfc1fcbdc04fea6a6e", 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "0", 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/tmp/cephfs", 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "87027", 2024-05-11T13:33:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/volumes/qa/sv_0/de410bc9-fc99-432f-8537-cb4b491e8ce2" 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 15291, 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 15291 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1337369332 2024-05-11T13:33:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1337369332 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.15291 192.168.0.1:0/1337369332", 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.1:0/1337369332", 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 5, 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 28, 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 90, 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-05-11T13:33:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-05-11T13:33:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-05-11T13:33:50.091 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:33:50.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:50.103 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-05-11T13:33:50.103 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /tmp/cephfs 2024-05-11T13:33:50.156 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-05-11T13:33:50.156 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-05-11T13:33:50.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:33:50.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /tmp/cephfs 2024-05-11T13:33:50.230 INFO:teuthology.run_tasks:Running task exec... 2024-05-11T13:33:50.239 INFO:teuthology.task.exec:Executing custom commands... 2024-05-11T13:33:50.240 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:33:50.240 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-11T13:33:50.433 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:50.066+0000 7f17c4007200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T13:33:50.433 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:50.202+0000 7f17c4007200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T13:33:50.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 -- 172.21.15.63:0/1551384687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc1087a0 msgr2=0x7f57cc10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:50.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 --2- 172.21.15.63:0/1551384687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc1087a0 0x7f57cc10abf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009ca0 tx=0x7f57c802a8c0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 -- 172.21.15.63:0/1551384687 shutdown_connections 2024-05-11T13:33:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 --2- 172.21.15.63:0/1551384687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc1087a0 0x7f57cc10abf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 --2- 172.21.15.63:0/1551384687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57cc105e70 0x7f57cc108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 --2- 172.21.15.63:0/1551384687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc103540 0x7f57cc105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 -- 172.21.15.63:0/1551384687 >> 172.21.15.63:0/1551384687 conn(0x7f57cc0fcf30 msgr2=0x7f57cc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 -- 172.21.15.63:0/1551384687 shutdown_connections 2024-05-11T13:33:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.436+0000 7f57d4800640 1 -- 172.21.15.63:0/1551384687 wait complete. 2024-05-11T13:33:50.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.437+0000 7f57d4800640 1 Processor -- start 2024-05-11T13:33:50.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.437+0000 7f57d4800640 1 -- start start 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d4800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc103540 0x7f57cc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d4800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc105e70 0x7f57cc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d4800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57cc1087a0 0x7f57cc1a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d4800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc101d70 con 0x7f57cc103540 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d4800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc101bf0 con 0x7f57cc105e70 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d4800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc101ef0 con 0x7f57cc1087a0 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc103540 0x7f57cc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc103540 0x7f57cc19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32834/0 (socket says 172.21.15.63:32834) 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d1e00640 1 -- 172.21.15.63:0/3726837284 learned_addr learned my addr 172.21.15.63:0/3726837284 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d2800640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57cc1087a0 0x7f57cc1a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d1400640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc105e70 0x7f57cc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d1e00640 1 -- 172.21.15.63:0/3726837284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57cc1087a0 msgr2=0x7f57cc1a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.438+0000 7f57d1e00640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57cc1087a0 0x7f57cc1a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57d1e00640 1 -- 172.21.15.63:0/3726837284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc105e70 msgr2=0x7f57cc19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:50.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57d1e00640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc105e70 0x7f57cc19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57d1e00640 1 -- 172.21.15.63:0/3726837284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc1a29d0 con 0x7f57cc103540 2024-05-11T13:33:50.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57d1400640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc105e70 0x7f57cc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:33:50.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57d1e00640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc103540 0x7f57cc19adb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57c000e9f0 tx=0x7f57c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:50.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57c6a00640 1 -- 172.21.15.63:0/3726837284 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c000cd80 con 0x7f57cc103540 2024-05-11T13:33:50.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57c6a00640 1 -- 172.21.15.63:0/3726837284 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57c00108c0 con 0x7f57cc103540 2024-05-11T13:33:50.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57c6a00640 1 -- 172.21.15.63:0/3726837284 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00187c0 con 0x7f57cc103540 2024-05-11T13:33:50.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc1a2cc0 con 0x7f57cc103540 2024-05-11T13:33:50.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.439+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc1a31c0 con 0x7f57cc103540 2024-05-11T13:33:50.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.441+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc1a3390 con 0x7f57cc103540 2024-05-11T13:33:50.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.443+0000 7f57c6a00640 1 -- 172.21.15.63:0/3726837284 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f57c0010430 con 0x7f57cc103540 2024-05-11T13:33:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.446+0000 7f57c6a00640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 0x7f57b40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.446+0000 7f57d1400640 1 -- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 msgr2=0x7f57b40b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.446+0000 7f57c6a00640 1 -- 172.21.15.63:0/3726837284 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f57c00e2b50 con 0x7f57cc103540 2024-05-11T13:33:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.446+0000 7f57d1400640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 0x7f57b40b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:33:50.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.451+0000 7f57c6a00640 1 -- 172.21.15.63:0/3726837284 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57c00aa460 con 0x7f57cc103540 2024-05-11T13:33:50.452 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:50.047+0000 7f856e4d9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:33:50.452 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:50.171+0000 7f856e4d9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:33:50.565 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:50.104+0000 7fc6ff068200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:33:50.565 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:50.231+0000 7fc6ff068200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:33:50.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.647+0000 7f57d1400640 1 -- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 msgr2=0x7f57b40b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:50.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.647+0000 7f57d1400640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 0x7f57b40b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:33:50.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.705+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f57cc1a3660 con 0x7f57cc103540 2024-05-11T13:33:50.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.760+0000 7f57c6a00640 1 -- 172.21.15.63:0/3726837284 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f57c00a9d60 con 0x7f57cc103540 2024-05-11T13:33:50.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 msgr2=0x7f57b40b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:33:50.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 0x7f57b40b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc103540 msgr2=0x7f57cc19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:50.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc103540 0x7f57cc19adb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57c000e9f0 tx=0x7f57c000eeb0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 shutdown_connections 2024-05-11T13:33:50.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f57b40b28f0 0x7f57b40b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57cc1087a0 0x7f57cc1a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc105e70 0x7f57cc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 --2- 172.21.15.63:0/3726837284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc103540 0x7f57cc19adb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 >> 172.21.15.63:0/3726837284 conn(0x7f57cc0fcf30 msgr2=0x7f57cc108b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:50.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 shutdown_connections 2024-05-11T13:33:50.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.764+0000 7f57d4800640 1 -- 172.21.15.63:0/3726837284 wait complete. 2024-05-11T13:33:50.783 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-11T13:33:50.794 INFO:teuthology.run_tasks:Running task exec... 2024-05-11T13:33:50.802 INFO:teuthology.task.exec:Executing custom commands... 2024-05-11T13:33:50.802 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:33:50.802 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-11T13:33:50.934 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:50 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3726837284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-11T13:33:50.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:50.625+0000 7f17c4007200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T13:33:50.934 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:50.760+0000 7f17c4007200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 -- 172.21.15.63:0/193007275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 msgr2=0x7f133c075940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 --2- 172.21.15.63:0/193007275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c075940 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1324009900 tx=0x7f132402a3f0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 -- 172.21.15.63:0/193007275 shutdown_connections 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 --2- 172.21.15.63:0/193007275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c10a7b0 0x7f133c10cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 --2- 172.21.15.63:0/193007275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c075940 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 --2- 172.21.15.63:0/193007275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133c076aa0 0x7f133c074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 -- 172.21.15.63:0/193007275 >> 172.21.15.63:0/193007275 conn(0x7f133c0fde60 msgr2=0x7f133c1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.990+0000 7f133be00640 1 -- 172.21.15.63:0/193007275 shutdown_connections 2024-05-11T13:33:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.991+0000 7f133be00640 1 -- 172.21.15.63:0/193007275 wait complete. 2024-05-11T13:33:50.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.991+0000 7f133be00640 1 Processor -- start 2024-05-11T13:33:50.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133be00640 1 -- start start 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133c076aa0 0x7f133c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c10a7b0 0x7f133c1a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f133c110500 con 0x7f133c0754c0 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f133c110380 con 0x7f133c10a7b0 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f133c110680 con 0x7f133c076aa0 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32852/0 (socket says 172.21.15.63:32852) 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133aa00640 1 -- 172.21.15.63:0/3508743199 learned_addr learned my addr 172.21.15.63:0/3508743199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.992+0000 7f133b400640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c10a7b0 0x7f133c1a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f1333e00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133c076aa0 0x7f133c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133aa00640 1 -- 172.21.15.63:0/3508743199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133c076aa0 msgr2=0x7f133c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:50.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133aa00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133c076aa0 0x7f133c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133aa00640 1 -- 172.21.15.63:0/3508743199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c10a7b0 msgr2=0x7f133c1a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133aa00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c10a7b0 0x7f133c1a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133aa00640 1 -- 172.21.15.63:0/3508743199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1a28d0 con 0x7f133c0754c0 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f1333e00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133c076aa0 0x7f133c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133aa00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c19ac90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f132c00d770 tx=0x7f132c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f1333400640 1 -- 172.21.15.63:0/3508743199 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c017070 con 0x7f133c0754c0 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1a2bc0 con 0x7f133c0754c0 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f1333400640 1 -- 172.21.15.63:0/3508743199 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f132c0054b0 con 0x7f133c0754c0 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f1333400640 1 -- 172.21.15.63:0/3508743199 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c004c60 con 0x7f133c0754c0 2024-05-11T13:33:50.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.993+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1a30c0 con 0x7f133c0754c0 2024-05-11T13:33:50.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.995+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c1094e0 con 0x7f133c0754c0 2024-05-11T13:33:50.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.997+0000 7f1333400640 1 -- 172.21.15.63:0/3508743199 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f132c004e00 con 0x7f133c0754c0 2024-05-11T13:33:50.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.999+0000 7f1333400640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 0x7f13200b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:50.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.999+0000 7f1333e00640 1 -- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 msgr2=0x7f13200b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:50.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.999+0000 7f1333400640 1 -- 172.21.15.63:0/3508743199 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f132c0e2130 con 0x7f133c0754c0 2024-05-11T13:33:50.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:50.999+0000 7f1333e00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 0x7f13200b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:33:51.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.004+0000 7f1333400640 1 -- 172.21.15.63:0/3508743199 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f132c0a9aa0 con 0x7f133c0754c0 2024-05-11T13:33:51.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:50 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3726837284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-11T13:33:51.065 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:51.003+0000 7fc6ff068200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:33:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.200+0000 7f1333e00640 1 -- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 msgr2=0x7f13200b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.200+0000 7f1333e00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 0x7f13200b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:33:51.202 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:50.931+0000 7f856e4d9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:33:51.203 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:50 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3726837284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-11T13:33:51.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.212+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f133c0636d0 con 0x7f133c0754c0 2024-05-11T13:33:51.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.219+0000 7f1333400640 1 -- 172.21.15.63:0/3508743199 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f132c0a93a0 con 0x7f133c0754c0 2024-05-11T13:33:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 msgr2=0x7f13200b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:33:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 0x7f13200b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 msgr2=0x7f133c19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c19ac90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f132c00d770 tx=0x7f132c00dc30 comp rx=0 tx=0).stop 2024-05-11T13:33:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 shutdown_connections 2024-05-11T13:33:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f13200b2830 0x7f13200b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c10a7b0 0x7f133c1a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133c076aa0 0x7f133c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 --2- 172.21.15.63:0/3508743199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f133c0754c0 0x7f133c19ac90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.222+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 >> 172.21.15.63:0/3508743199 conn(0x7f133c0fde60 msgr2=0x7f133c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.223+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 shutdown_connections 2024-05-11T13:33:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.223+0000 7f133be00640 1 -- 172.21.15.63:0/3508743199 wait complete. 2024-05-11T13:33:51.241 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-11T13:33:51.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.433+0000 7f2f84a00640 1 -- 172.21.15.63:0/1300051692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80108660 msgr2=0x7f2f8010aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.433+0000 7f2f84a00640 1 --2- 172.21.15.63:0/1300051692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80108660 0x7f2f8010aa50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009ca0 tx=0x7f2f7802a8c0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.435+0000 7f2f84a00640 1 -- 172.21.15.63:0/1300051692 shutdown_connections 2024-05-11T13:33:51.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.435+0000 7f2f84a00640 1 --2- 172.21.15.63:0/1300051692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80108660 0x7f2f8010aa50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.435+0000 7f2f84a00640 1 --2- 172.21.15.63:0/1300051692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f80105d30 0x7f2f80108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.436+0000 7f2f84a00640 1 --2- 172.21.15.63:0/1300051692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f80103400 0x7f2f801057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.436+0000 7f2f84a00640 1 -- 172.21.15.63:0/1300051692 >> 172.21.15.63:0/1300051692 conn(0x7f2f800fcd50 msgr2=0x7f2f800ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:51.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.436+0000 7f2f84a00640 1 -- 172.21.15.63:0/1300051692 shutdown_connections 2024-05-11T13:33:51.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.437+0000 7f2f84a00640 1 -- 172.21.15.63:0/1300051692 wait complete. 2024-05-11T13:33:51.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 Processor -- start 2024-05-11T13:33:51.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 -- start start 2024-05-11T13:33:51.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f80103400 0x7f2f8019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80105d30 0x7f2f8019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f80108660 0x7f2f801a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010e310 con 0x7f2f80105d30 2024-05-11T13:33:51.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010e190 con 0x7f2f80103400 2024-05-11T13:33:51.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.439+0000 7f2f84a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010e490 con 0x7f2f80108660 2024-05-11T13:33:51.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80105d30 0x7f2f8019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80105d30 0x7f2f8019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32864/0 (socket says 172.21.15.63:32864) 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 -- 172.21.15.63:0/198958911 learned_addr learned my addr 172.21.15.63:0/198958911 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7f400640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f80103400 0x7f2f8019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7fe00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f80108660 0x7f2f801a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 -- 172.21.15.63:0/198958911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f80108660 msgr2=0x7f2f801a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f80108660 0x7f2f801a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 -- 172.21.15.63:0/198958911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f80103400 msgr2=0x7f2f8019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f80103400 0x7f2f8019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.440+0000 7f2f7ea00640 1 -- 172.21.15.63:0/198958911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f801a2800 con 0x7f2f80105d30 2024-05-11T13:33:51.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.441+0000 7f2f7ea00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80105d30 0x7f2f8019b120 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00d770 tx=0x7f2f6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:51.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.441+0000 7f2f77e00640 1 -- 172.21.15.63:0/198958911 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c017070 con 0x7f2f80105d30 2024-05-11T13:33:51.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.441+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f801a2af0 con 0x7f2f80105d30 2024-05-11T13:33:51.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.441+0000 7f2f77e00640 1 -- 172.21.15.63:0/198958911 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f6c009c30 con 0x7f2f80105d30 2024-05-11T13:33:51.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.441+0000 7f2f77e00640 1 -- 172.21.15.63:0/198958911 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00f620 con 0x7f2f80105d30 2024-05-11T13:33:51.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.441+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f801a3090 con 0x7f2f80105d30 2024-05-11T13:33:51.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.443+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8010a150 con 0x7f2f80105d30 2024-05-11T13:33:51.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.445+0000 7f2f77e00640 1 -- 172.21.15.63:0/198958911 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f2f6c01a070 con 0x7f2f80105d30 2024-05-11T13:33:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.447+0000 7f2f77e00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 0x7f2f640b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.447+0000 7f2f7f400640 1 -- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 msgr2=0x7f2f640b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.447+0000 7f2f77e00640 1 -- 172.21.15.63:0/198958911 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2f6c0e2230 con 0x7f2f80105d30 2024-05-11T13:33:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.447+0000 7f2f7f400640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 0x7f2f640b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:33:51.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.452+0000 7f2f77e00640 1 -- 172.21.15.63:0/198958911 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f6c009de0 con 0x7f2f80105d30 2024-05-11T13:33:51.565 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:33:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z[35106]: 2024-05-11T13:33:51.296+0000 7fc6ff068200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:33:51.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.648+0000 7f2f7f400640 1 -- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 msgr2=0x7f2f640b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:51.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.648+0000 7f2f7f400640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 0x7f2f640b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T13:33:51.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.661+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f2f80063660 con 0x7f2f80105d30 2024-05-11T13:33:51.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.668+0000 7f2f77e00640 1 -- 172.21.15.63:0/198958911 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f2f6c0a9b40 con 0x7f2f80105d30 2024-05-11T13:33:51.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 msgr2=0x7f2f640b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:33:51.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 0x7f2f640b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80105d30 msgr2=0x7f2f8019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80105d30 0x7f2f8019b120 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00d770 tx=0x7f2f6c00dc30 comp rx=0 tx=0).stop 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 shutdown_connections 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f2f640b2830 0x7f2f640b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f80108660 0x7f2f801a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f80105d30 0x7f2f8019b120 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 --2- 172.21.15.63:0/198958911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f80103400 0x7f2f8019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 >> 172.21.15.63:0/198958911 conn(0x7f2f800fcd50 msgr2=0x7f2f80102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 shutdown_connections 2024-05-11T13:33:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.671+0000 7f2f84a00640 1 -- 172.21.15.63:0/198958911 wait complete. 2024-05-11T13:33:51.690 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-11T13:33:51.702 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:33:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y[34666]: 2024-05-11T13:33:51.217+0000 7f856e4d9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3726837284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3508743199' entity='client.admin' 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: Standby manager daemon y restarted 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: Standby manager daemon y started 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: Standby manager daemon z restarted 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: Standby manager daemon z started 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T13:33:51.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:33:51.869 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T13:33:51.869 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:33:51.869 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:51 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/198958911' entity='client.admin' 2024-05-11T13:33:51.869 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:51 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:51.594+0000 7f17c4007200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T13:33:51.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.881+0000 7f3100600640 1 -- 172.21.15.63:0/3901381525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8105ef0 msgr2=0x7f30f81082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.881+0000 7f3100600640 1 --2- 172.21.15.63:0/3901381525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8105ef0 0x7f30f81082e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009900 tx=0x7f30e402a3f0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.882+0000 7f3100600640 1 -- 172.21.15.63:0/3901381525 shutdown_connections 2024-05-11T13:33:51.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.882+0000 7f3100600640 1 --2- 172.21.15.63:0/3901381525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f8108820 0x7f30f810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.882+0000 7f3100600640 1 --2- 172.21.15.63:0/3901381525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8105ef0 0x7f30f81082e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.882+0000 7f3100600640 1 --2- 172.21.15.63:0/3901381525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30f81035c0 0x7f30f81059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.882+0000 7f3100600640 1 -- 172.21.15.63:0/3901381525 >> 172.21.15.63:0/3901381525 conn(0x7f30f80fcf30 msgr2=0x7f30f80ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:51.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.882+0000 7f3100600640 1 -- 172.21.15.63:0/3901381525 shutdown_connections 2024-05-11T13:33:51.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.882+0000 7f3100600640 1 -- 172.21.15.63:0/3901381525 wait complete. 2024-05-11T13:33:51.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 Processor -- start 2024-05-11T13:33:51.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 -- start start 2024-05-11T13:33:51.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30f81035c0 0x7f30f819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f8105ef0 0x7f30f819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8108820 0x7f30f81a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30f810e4f0 con 0x7f30f8108820 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30f810e370 con 0x7f30f81035c0 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f3100600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30f810e670 con 0x7f30f8105ef0 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f30fe600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8108820 0x7f30f81a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f30fe600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8108820 0x7f30f81a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32884/0 (socket says 172.21.15.63:32884) 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.883+0000 7f30fe600640 1 -- 172.21.15.63:0/3633895458 learned_addr learned my addr 172.21.15.63:0/3633895458 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fd200640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f8105ef0 0x7f30f819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fdc00640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30f81035c0 0x7f30f819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fe600640 1 -- 172.21.15.63:0/3633895458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f8105ef0 msgr2=0x7f30f819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fe600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f8105ef0 0x7f30f819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fe600640 1 -- 172.21.15.63:0/3633895458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30f81035c0 msgr2=0x7f30f819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fe600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30f81035c0 0x7f30f819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fe600640 1 -- 172.21.15.63:0/3633895458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f81a2a60 con 0x7f30f8108820 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30fe600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8108820 0x7f30f81a23f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f30f000cad0 tx=0x7f30f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f000ae20 con 0x7f30f8108820 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30f0005020 con 0x7f30f8108820 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.884+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f81a2d50 con 0x7f30f8108820 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.885+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f000f6b0 con 0x7f30f8108820 2024-05-11T13:33:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.885+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f81a3260 con 0x7f30f8108820 2024-05-11T13:33:51.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.886+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f810a310 con 0x7f30f8108820 2024-05-11T13:33:51.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.888+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f30f00051d0 con 0x7f30f8108820 2024-05-11T13:33:51.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.889+0000 7f30f6000640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f30e00b2970 0x7f30e00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:51.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.890+0000 7f30fdc00640 1 -- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f30e00b2970 msgr2=0x7f30e00b4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1742389159 2024-05-11T13:33:51.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.890+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f30f00e2a40 con 0x7f30f8108820 2024-05-11T13:33:51.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.890+0000 7f30fdc00640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f30e00b2970 0x7f30e00b4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T13:33:51.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.894+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30f0014030 con 0x7f30f8108820 2024-05-11T13:33:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.957+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7f30f00a5ee0 con 0x7f30f8108820 2024-05-11T13:33:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.957+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f30e00b2970 msgr2=0x7f30e00b4e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T13:33:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:51.957+0000 7f30f6000640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f30e00b2970 0x7f30e00b4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3726837284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3508743199' entity='client.admin' 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: Standby manager daemon y restarted 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: Standby manager daemon y started 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: Standby manager daemon z restarted 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: Standby manager daemon z started 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T13:33:52.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:33:52.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:51 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/198958911' entity='client.admin' 2024-05-11T13:33:52.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.142+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f30f802dbb0 con 0x7f30f8108820 2024-05-11T13:33:52.147 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:51 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:51.930+0000 7f17c4007200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T13:33:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.148+0000 7f30f6000640 1 -- 172.21.15.63:0/3633895458 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f30f00aa350 con 0x7f30f8108820 2024-05-11T13:33:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.151+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8108820 msgr2=0x7f30f81a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.151+0000 7f3100600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8108820 0x7f30f81a23f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f30f000cad0 tx=0x7f30f000cf90 comp rx=0 tx=0).stop 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.151+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 shutdown_connections 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.152+0000 7f3100600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:6800/1742389159,v1:172.21.15.63:6801/1742389159] conn(0x7f30e00b2970 0x7f30e00b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.152+0000 7f3100600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30f8108820 0x7f30f81a23f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.152+0000 7f3100600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f8105ef0 0x7f30f819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.152+0000 7f3100600640 1 --2- 172.21.15.63:0/3633895458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30f81035c0 0x7f30f819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.152+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 >> 172.21.15.63:0/3633895458 conn(0x7f30f80fcf30 msgr2=0x7f30f8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.152+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 shutdown_connections 2024-05-11T13:33:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.152+0000 7f3100600640 1 -- 172.21.15.63:0/3633895458 wait complete. 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3726837284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3508743199' entity='client.admin' 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: Standby manager daemon y restarted 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: Standby manager daemon y started 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.71:0/3492534593' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: Standby manager daemon z restarted 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: Standby manager daemon z started 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T13:33:52.154 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T13:33:52.155 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T13:33:52.155 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='mgr.? 172.21.15.165:0/1207778202' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T13:33:52.155 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:51 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/198958911' entity='client.admin' 2024-05-11T13:33:52.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-11T13:33:52.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 -- 172.21.15.63:0/4082940799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664104b20 msgr2=0x7fd664104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 --2- 172.21.15.63:0/4082940799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664104b20 0x7fd664104f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd654009860 tx=0x7fd65402a340 comp rx=0 tx=0).stop 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 -- 172.21.15.63:0/4082940799 shutdown_connections 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 --2- 172.21.15.63:0/4082940799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6641066d0 0x7fd66410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 --2- 172.21.15.63:0/4082940799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664105d10 0x7fd664106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 --2- 172.21.15.63:0/4082940799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664104b20 0x7fd664104f40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 -- 172.21.15.63:0/4082940799 >> 172.21.15.63:0/4082940799 conn(0x7fd664100090 msgr2=0x7fd6641024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 -- 172.21.15.63:0/4082940799 shutdown_connections 2024-05-11T13:33:52.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.385+0000 7fd668a00640 1 -- 172.21.15.63:0/4082940799 wait complete. 2024-05-11T13:33:52.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.387+0000 7fd668a00640 1 Processor -- start 2024-05-11T13:33:52.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.388+0000 7fd668a00640 1 -- start start 2024-05-11T13:33:52.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.388+0000 7fd668a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664104b20 0x7fd664071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:52.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.388+0000 7fd668a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664105d10 0x7fd664071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:52.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.388+0000 7fd668a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6641066d0 0x7fd6641aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:52.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.388+0000 7fd668a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd664110520 con 0x7fd6641066d0 2024-05-11T13:33:52.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.388+0000 7fd668a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641103a0 con 0x7fd664104b20 2024-05-11T13:33:52.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.388+0000 7fd668a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641106a0 con 0x7fd664105d10 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd663200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6641066d0 0x7fd6641aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664104b20 0x7fd664071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664104b20 0x7fd664071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:39664/0 (socket says 172.21.15.63:39664) 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 -- 172.21.15.63:0/3430737352 learned_addr learned my addr 172.21.15.63:0/3430737352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd661e00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664105d10 0x7fd664071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 -- 172.21.15.63:0/3430737352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664105d10 msgr2=0x7fd664071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664105d10 0x7fd664071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 -- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6641066d0 msgr2=0x7fd6641aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6641066d0 0x7fd6641aac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:52.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.389+0000 7fd662800640 1 -- 172.21.15.63:0/3430737352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6641ab170 con 0x7fd664104b20 2024-05-11T13:33:52.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.390+0000 7fd662800640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664104b20 0x7fd664071560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd654009780 tx=0x7fd65402ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:52.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.390+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654037040 con 0x7fd664104b20 2024-05-11T13:33:52.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.390+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6540305b0 con 0x7fd664104b20 2024-05-11T13:33:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.390+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654003850 con 0x7fd664104b20 2024-05-11T13:33:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.390+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6641ab400 con 0x7fd664104b20 2024-05-11T13:33:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.390+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6641ab7b0 con 0x7fd664104b20 2024-05-11T13:33:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.392+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66410bf50 con 0x7fd664104b20 2024-05-11T13:33:52.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.396+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7fd654043020 con 0x7fd664104b20 2024-05-11T13:33:52.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.399+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd654101520 con 0x7fd664104b20 2024-05-11T13:33:52.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.399+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd65404a020 con 0x7fd664104b20 2024-05-11T13:33:52.780 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.565+0000 7f1719000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.565+0000 7f1719000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.565+0000 7f1719000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.565+0000 7f1719000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.565+0000 7f1719000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.566+0000 7f1715400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.566+0000 7f1715400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.566+0000 7f1715400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.566+0000 7f1715400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.781 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:33:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:33:52.566+0000 7f1715400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: Active manager daemon x restarted 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: Activating manager daemon x 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: osdmap e91: 12 total, 12 up, 12 in 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: mgrmap e20: x(active, starting, since 0.0250177s), standbys: y, z 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:33:52.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T13:33:52.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:33:52.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:33:52.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: Manager daemon x is now available 2024-05-11T13:33:52.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:33:52.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T13:33:52.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T13:33:52.957 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:52 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3633895458' entity='client.admin' 2024-05-11T13:33:52.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.980+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 21) ==== 149464+0+0 (secure 0 0 0) 0x7fd6540c4400 con 0x7fd664104b20 2024-05-11T13:33:52.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.981+0000 7fd65b400640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd6480baee0 0x7fd6480bd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:52.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.981+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd654042e70 con 0x7fd6480baee0 2024-05-11T13:33:52.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.985+0000 7fd661e00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd6480baee0 0x7fd6480bd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:52.992+0000 7fd661e00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd6480baee0 0x7fd6480bd2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd664072a70 tx=0x7fd650009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: Active manager daemon x restarted 2024-05-11T13:33:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: Activating manager daemon x 2024-05-11T13:33:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: osdmap e91: 12 total, 12 up, 12 in 2024-05-11T13:33:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: mgrmap e20: x(active, starting, since 0.0250177s), standbys: y, z 2024-05-11T13:33:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:33:53.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:33:53.067 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:33:53.068 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: Manager daemon x is now available 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T13:33:53.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:52 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3633895458' entity='client.admin' 2024-05-11T13:33:53.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.136+0000 7fd65b400640 1 -- 172.21.15.63:0/3430737352 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd654042e70 con 0x7fd6480baee0 2024-05-11T13:33:53.136 INFO:teuthology.orchestra.run.smithi063.stdout:Schedule set for path / 2024-05-11T13:33:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd6480baee0 msgr2=0x7fd6480bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd6480baee0 0x7fd6480bd2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd664072a70 tx=0x7fd650009260 comp rx=0 tx=0).stop 2024-05-11T13:33:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664104b20 msgr2=0x7fd664071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664104b20 0x7fd664071560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd654009780 tx=0x7fd65402ae50 comp rx=0 tx=0).stop 2024-05-11T13:33:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 shutdown_connections 2024-05-11T13:33:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd6480baee0 0x7fd6480bd2d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6641066d0 0x7fd6641aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd664105d10 0x7fd664071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 --2- 172.21.15.63:0/3430737352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd664104b20 0x7fd664071560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 >> 172.21.15.63:0/3430737352 conn(0x7fd664100090 msgr2=0x7fd664109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 shutdown_connections 2024-05-11T13:33:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.139+0000 7fd668a00640 1 -- 172.21.15.63:0/3430737352 wait complete. 2024-05-11T13:33:53.153 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: Active manager daemon x restarted 2024-05-11T13:33:53.153 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: Activating manager daemon x 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: osdmap e91: 12 total, 12 up, 12 in 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: mgrmap e20: x(active, starting, since 0.0250177s), standbys: y, z 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:33:53.154 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T13:33:53.155 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: Manager daemon x is now available 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T13:33:53.156 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:52 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3633895458' entity='client.admin' 2024-05-11T13:33:53.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 -- 172.21.15.63:0/1764005637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 msgr2=0x7f651c10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 --2- 172.21.15.63:0/1764005637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 0x7f651c10abf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6514009ca0 tx=0x7f651402a8c0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 -- 172.21.15.63:0/1764005637 shutdown_connections 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 --2- 172.21.15.63:0/1764005637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 0x7f651c10abf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 --2- 172.21.15.63:0/1764005637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f651c105e70 0x7f651c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 --2- 172.21.15.63:0/1764005637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f651c103540 0x7f651c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 -- 172.21.15.63:0/1764005637 >> 172.21.15.63:0/1764005637 conn(0x7f651c0fcf30 msgr2=0x7f651c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.342+0000 7f6522600640 1 -- 172.21.15.63:0/1764005637 shutdown_connections 2024-05-11T13:33:53.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.343+0000 7f6522600640 1 -- 172.21.15.63:0/1764005637 wait complete. 2024-05-11T13:33:53.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 Processor -- start 2024-05-11T13:33:53.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 -- start start 2024-05-11T13:33:53.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f651c103540 0x7f651c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:53.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f651c105e70 0x7f651c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:53.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 0x7f651c1a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:53.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10e510 con 0x7f651c1087a0 2024-05-11T13:33:53.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10e390 con 0x7f651c105e70 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f6522600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10e690 con 0x7f651c103540 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f651aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f651c105e70 0x7f651c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f651be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 0x7f651c1a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f651aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f651c105e70 0x7f651c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:39708/0 (socket says 172.21.15.63:39708) 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.344+0000 7f651aa00640 1 -- 172.21.15.63:0/2454974541 learned_addr learned my addr 172.21.15.63:0/2454974541 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.345+0000 7f651b400640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f651c103540 0x7f651c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.345+0000 7f651be00640 1 -- 172.21.15.63:0/2454974541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f651c103540 msgr2=0x7f651c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.345+0000 7f651be00640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f651c103540 0x7f651c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.345+0000 7f651be00640 1 -- 172.21.15.63:0/2454974541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f651c105e70 msgr2=0x7f651c19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.345+0000 7f651be00640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f651c105e70 0x7f651c19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.345+0000 7f651be00640 1 -- 172.21.15.63:0/2454974541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c1a2960 con 0x7f651c1087a0 2024-05-11T13:33:53.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.345+0000 7f651be00640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 0x7f651c1a22f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6514031040 tx=0x7f6514003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:53.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.346+0000 7f6513e00640 1 -- 172.21.15.63:0/2454974541 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514032e90 con 0x7f651c1087a0 2024-05-11T13:33:53.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.346+0000 7f6513e00640 1 -- 172.21.15.63:0/2454974541 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65140044d0 con 0x7f651c1087a0 2024-05-11T13:33:53.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.346+0000 7f6513e00640 1 -- 172.21.15.63:0/2454974541 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514030ce0 con 0x7f651c1087a0 2024-05-11T13:33:53.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.346+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c1a2bf0 con 0x7f651c1087a0 2024-05-11T13:33:53.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.346+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c1a3160 con 0x7f651c1087a0 2024-05-11T13:33:53.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.348+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651c10a290 con 0x7f651c1087a0 2024-05-11T13:33:53.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.350+0000 7f6513e00640 1 -- 172.21.15.63:0/2454974541 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 21) ==== 149464+0+0 (secure 0 0 0) 0x7f6514004040 con 0x7f651c1087a0 2024-05-11T13:33:53.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.351+0000 7f6513e00640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65000b2710 0x7f65000b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:53.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.352+0000 7f6513e00640 1 -- 172.21.15.63:0/2454974541 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f65141038d0 con 0x7f651c1087a0 2024-05-11T13:33:53.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.352+0000 7f651b400640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65000b2710 0x7f65000b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:53.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.353+0000 7f651b400640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65000b2710 0x7f65000b4bd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f650c005d00 tx=0x7f650c00b410 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:53.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.357+0000 7f6513e00640 1 -- 172.21.15.63:0/2454974541 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65140cb3f0 con 0x7f651c1087a0 2024-05-11T13:33:53.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.623+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f651c062f40 con 0x7f65000b2710 2024-05-11T13:33:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.790+0000 7f6513e00640 1 -- 172.21.15.63:0/2454974541 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f651c062f40 con 0x7f65000b2710 2024-05-11T13:33:53.790 INFO:teuthology.orchestra.run.smithi063.stdout:Retention added to path / 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.793+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65000b2710 msgr2=0x7f65000b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.793+0000 7f6522600640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65000b2710 0x7f65000b4bd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f650c005d00 tx=0x7f650c00b410 comp rx=0 tx=0).stop 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.793+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 msgr2=0x7f651c1a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.793+0000 7f6522600640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 0x7f651c1a22f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6514031040 tx=0x7f6514003ba0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 shutdown_connections 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65000b2710 0x7f65000b4bd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f651c1087a0 0x7f651c1a22f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f651c105e70 0x7f651c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 --2- 172.21.15.63:0/2454974541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f651c103540 0x7f651c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 >> 172.21.15.63:0/2454974541 conn(0x7f651c0fcf30 msgr2=0x7f651c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 shutdown_connections 2024-05-11T13:33:53.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:53.794+0000 7f6522600640 1 -- 172.21.15.63:0/2454974541 wait complete. 2024-05-11T13:33:53.814 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-11T13:33:54.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.005+0000 7efcbd000640 1 -- 172.21.15.63:0/2481703807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 msgr2=0x7efcb8106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.005+0000 7efcbd000640 1 --2- 172.21.15.63:0/2481703807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb8106330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efca0009900 tx=0x7efca002a410 comp rx=0 tx=0).stop 2024-05-11T13:33:54.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.006+0000 7efcbd000640 1 -- 172.21.15.63:0/2481703807 shutdown_connections 2024-05-11T13:33:54.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.006+0000 7efcbd000640 1 --2- 172.21.15.63:0/2481703807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcb8106870 0x7efcb810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.006+0000 7efcbd000640 1 --2- 172.21.15.63:0/2481703807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb8106330 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.006+0000 7efcbd000640 1 --2- 172.21.15.63:0/2481703807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8104cc0 0x7efcb81050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.006+0000 7efcbd000640 1 -- 172.21.15.63:0/2481703807 >> 172.21.15.63:0/2481703807 conn(0x7efcb8100250 msgr2=0x7efcb8102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:54.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.006+0000 7efcbd000640 1 -- 172.21.15.63:0/2481703807 shutdown_connections 2024-05-11T13:33:54.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.006+0000 7efcbd000640 1 -- 172.21.15.63:0/2481703807 wait complete. 2024-05-11T13:33:54.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 Processor -- start 2024-05-11T13:33:54.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 -- start start 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcb8104cc0 0x7efcb819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8106870 0x7efcb81a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81106f0 con 0x7efcb8105eb0 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8110570 con 0x7efcb8106870 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcbd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8110870 con 0x7efcb8104cc0 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.007+0000 7efcb5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33064/0 (socket says 172.21.15.63:33064) 2024-05-11T13:33:54.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 -- 172.21.15.63:0/2988520498 learned_addr learned my addr 172.21.15.63:0/2988520498 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb6800640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8106870 0x7efcb81a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5e00640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcb8104cc0 0x7efcb819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 -- 172.21.15.63:0/2988520498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcb8104cc0 msgr2=0x7efcb819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcb8104cc0 0x7efcb819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 -- 172.21.15.63:0/2988520498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8106870 msgr2=0x7efcb81a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8106870 0x7efcb81a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 -- 172.21.15.63:0/2988520498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb81a2980 con 0x7efcb8105eb0 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb6800640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8106870 0x7efcb81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.008+0000 7efcb5400640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb819b2a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca0031460 tx=0x7efca0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.009+0000 7efcaea00640 1 -- 172.21.15.63:0/2988520498 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca003f070 con 0x7efcb8105eb0 2024-05-11T13:33:54.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.009+0000 7efcaea00640 1 -- 172.21.15.63:0/2988520498 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efca0031da0 con 0x7efcb8105eb0 2024-05-11T13:33:54.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.009+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81a2c10 con 0x7efcb8105eb0 2024-05-11T13:33:54.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.009+0000 7efcaea00640 1 -- 172.21.15.63:0/2988520498 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0035030 con 0x7efcb8105eb0 2024-05-11T13:33:54.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.009+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81a3150 con 0x7efcb8105eb0 2024-05-11T13:33:54.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.010+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb810c000 con 0x7efcb8105eb0 2024-05-11T13:33:54.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.012+0000 7efcaea00640 1 -- 172.21.15.63:0/2988520498 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 22) ==== 149772+0+0 (secure 0 0 0) 0x7efca0042070 con 0x7efcb8105eb0 2024-05-11T13:33:54.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.014+0000 7efcaea00640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efc9c0b2a90 0x7efc9c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.014+0000 7efcaea00640 1 -- 172.21.15.63:0/2988520498 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7efca0103210 con 0x7efcb8105eb0 2024-05-11T13:33:54.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.014+0000 7efcb5e00640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efc9c0b2a90 0x7efc9c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.015+0000 7efcb5e00640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efc9c0b2a90 0x7efc9c0b4f50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efcb8105d10 tx=0x7efca800b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:54.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.018+0000 7efcaea00640 1 -- 172.21.15.63:0/2988520498 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efca00ca1b0 con 0x7efcb8105eb0 2024-05-11T13:33:54.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:53 smithi063 ceph-mon[29809]: mgrmap e21: x(active, since 1.04734s), standbys: y, z 2024-05-11T13:33:54.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:53 smithi063 ceph-mon[29809]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:54.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:53 smithi063 ceph-mon[29809]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.266+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efcb8062ea0 con 0x7efc9c0b2a90 2024-05-11T13:33:54.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.283+0000 7efcaea00640 1 -- 172.21.15.63:0/2988520498 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7efcb8062ea0 con 0x7efc9c0b2a90 2024-05-11T13:33:54.282 INFO:teuthology.orchestra.run.smithi063.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T13:33:52", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efc9c0b2a90 msgr2=0x7efc9c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efc9c0b2a90 0x7efc9c0b4f50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efcb8105d10 tx=0x7efca800b390 comp rx=0 tx=0).stop 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 msgr2=0x7efcb819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb819b2a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca0031460 tx=0x7efca0003c80 comp rx=0 tx=0).stop 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 shutdown_connections 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efc9c0b2a90 0x7efc9c0b4f50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8106870 0x7efcb81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.285+0000 7efcbd000640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcb8105eb0 0x7efcb819b2a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.286+0000 7efcbd000640 1 --2- 172.21.15.63:0/2988520498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcb8104cc0 0x7efcb819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.286+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 >> 172.21.15.63:0/2988520498 conn(0x7efcb8100250 msgr2=0x7efcb81091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.286+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 shutdown_connections 2024-05-11T13:33:54.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.286+0000 7efcbd000640 1 -- 172.21.15.63:0/2988520498 wait complete. 2024-05-11T13:33:54.306 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-11T13:33:54.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:53 smithi165 ceph-mon[33235]: mgrmap e21: x(active, since 1.04734s), standbys: y, z 2024-05-11T13:33:54.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:53 smithi165 ceph-mon[33235]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:54.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:53 smithi165 ceph-mon[33235]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:54.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:53 smithi071 ceph-mon[33293]: mgrmap e21: x(active, since 1.04734s), standbys: y, z 2024-05-11T13:33:54.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:53 smithi071 ceph-mon[33293]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:54.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:53 smithi071 ceph-mon[33293]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 -- 172.21.15.63:0/3215310444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910103cb0 msgr2=0x7f5910104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 --2- 172.21.15.63:0/3215310444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910103cb0 0x7f5910104130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009900 tx=0x7f58f802a3f0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 -- 172.21.15.63:0/3215310444 shutdown_connections 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 --2- 172.21.15.63:0/3215310444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5910104670 0x7f591010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 --2- 172.21.15.63:0/3215310444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910103cb0 0x7f5910104130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 --2- 172.21.15.63:0/3215310444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5910102ac0 0x7f5910102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 -- 172.21.15.63:0/3215310444 >> 172.21.15.63:0/3215310444 conn(0x7f59100fe050 msgr2=0x7f5910100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 -- 172.21.15.63:0/3215310444 shutdown_connections 2024-05-11T13:33:54.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.502+0000 7f5915600640 1 -- 172.21.15.63:0/3215310444 wait complete. 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.503+0000 7f5915600640 1 Processor -- start 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5915600640 1 -- start start 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5915600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5910102ac0 0x7f5910076810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5915600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5910103cb0 0x7f5910074e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5915600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910104670 0x7f5910075390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5915600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f591010e4a0 con 0x7f5910104670 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5915600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f591010e320 con 0x7f5910102ac0 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5915600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f591010e620 con 0x7f5910103cb0 2024-05-11T13:33:54.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f590f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910104670 0x7f5910075390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f5907e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5910103cb0 0x7f5910074e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f590f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910104670 0x7f5910075390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33090/0 (socket says 172.21.15.63:33090) 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.504+0000 7f590f400640 1 -- 172.21.15.63:0/895931961 learned_addr learned my addr 172.21.15.63:0/895931961 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f590ea00640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5910102ac0 0x7f5910076810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f590f400640 1 -- 172.21.15.63:0/895931961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5910103cb0 msgr2=0x7f5910074e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f590f400640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5910103cb0 0x7f5910074e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f590f400640 1 -- 172.21.15.63:0/895931961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5910102ac0 msgr2=0x7f5910076810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f590f400640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5910102ac0 0x7f5910076810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f590f400640 1 -- 172.21.15.63:0/895931961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5910075c20 con 0x7f5910104670 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f5907e00640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5910103cb0 0x7f5910074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:33:54.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.505+0000 7f590f400640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910104670 0x7f5910075390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f590800eaa0 tx=0x7f590800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:54.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.506+0000 7f5907400640 1 -- 172.21.15.63:0/895931961 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590800cdb0 con 0x7f5910104670 2024-05-11T13:33:54.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.506+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59101a2cb0 con 0x7f5910104670 2024-05-11T13:33:54.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.506+0000 7f5907400640 1 -- 172.21.15.63:0/895931961 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5908005020 con 0x7f5910104670 2024-05-11T13:33:54.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.506+0000 7f5907400640 1 -- 172.21.15.63:0/895931961 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59080116d0 con 0x7f5910104670 2024-05-11T13:33:54.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.506+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59101a31d0 con 0x7f5910104670 2024-05-11T13:33:54.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.507+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59100507f0 con 0x7f5910104670 2024-05-11T13:33:54.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.511+0000 7f5907400640 1 -- 172.21.15.63:0/895931961 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 22) ==== 149772+0+0 (secure 0 0 0) 0x7f59080051d0 con 0x7f5910104670 2024-05-11T13:33:54.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.513+0000 7f5907400640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f58f40b2a10 0x7f58f40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:54.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.513+0000 7f5907400640 1 -- 172.21.15.63:0/895931961 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f59080e2fe0 con 0x7f5910104670 2024-05-11T13:33:54.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.513+0000 7f590ea00640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f58f40b2a10 0x7f58f40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:54.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.514+0000 7f5907400640 1 -- 172.21.15.63:0/895931961 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5908016030 con 0x7f5910104670 2024-05-11T13:33:54.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.514+0000 7f590ea00640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f58f40b2a10 0x7f58f40b4ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5910103b10 tx=0x7f5900005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.783+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5910062f60 con 0x7f58f40b2a10 2024-05-11T13:33:54.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.793+0000 7f5907400640 1 -- 172.21.15.63:0/895931961 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5910062f60 con 0x7f58f40b2a10 2024-05-11T13:33:54.792 INFO:teuthology.orchestra.run.smithi063.stdout:/ 1m 6m3h 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f58f40b2a10 msgr2=0x7f58f40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f58f40b2a10 0x7f58f40b4ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5910103b10 tx=0x7f5900005f90 comp rx=0 tx=0).stop 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910104670 msgr2=0x7f5910075390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910104670 0x7f5910075390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f590800eaa0 tx=0x7f590800ef60 comp rx=0 tx=0).stop 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 shutdown_connections 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f58f40b2a10 0x7f58f40b4ed0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5910104670 0x7f5910075390 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5910103cb0 0x7f5910074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 --2- 172.21.15.63:0/895931961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5910102ac0 0x7f5910076810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.796+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 >> 172.21.15.63:0/895931961 conn(0x7f59100fe050 msgr2=0x7f5910107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.797+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 shutdown_connections 2024-05-11T13:33:54.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:54.797+0000 7f5915600640 1 -- 172.21.15.63:0/895931961 wait complete. 2024-05-11T13:33:54.817 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-11T13:33:54.890 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-11T13:33:54.905 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-11T13:33:54.940 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-11T13:33:54.940 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T13:33:55.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:54 smithi063 ceph-mon[29809]: [11/May/2024:13:33:53] ENGINE Bus STARTING 2024-05-11T13:33:55.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:54 smithi063 ceph-mon[29809]: [11/May/2024:13:33:53] ENGINE Serving on https://172.21.15.63:7150 2024-05-11T13:33:55.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:54 smithi063 ceph-mon[29809]: [11/May/2024:13:33:53] ENGINE Serving on http://172.21.15.63:8765 2024-05-11T13:33:55.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:54 smithi063 ceph-mon[29809]: [11/May/2024:13:33:53] ENGINE Bus STARTED 2024-05-11T13:33:55.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:54 smithi063 ceph-mon[29809]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:55.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:54 smithi063 ceph-mon[29809]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:55.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:54 smithi063 ceph-mon[29809]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-11T13:33:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:54 smithi165 ceph-mon[33235]: [11/May/2024:13:33:53] ENGINE Bus STARTING 2024-05-11T13:33:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:54 smithi165 ceph-mon[33235]: [11/May/2024:13:33:53] ENGINE Serving on https://172.21.15.63:7150 2024-05-11T13:33:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:54 smithi165 ceph-mon[33235]: [11/May/2024:13:33:53] ENGINE Serving on http://172.21.15.63:8765 2024-05-11T13:33:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:54 smithi165 ceph-mon[33235]: [11/May/2024:13:33:53] ENGINE Bus STARTED 2024-05-11T13:33:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:54 smithi165 ceph-mon[33235]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:54 smithi165 ceph-mon[33235]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:55.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:54 smithi165 ceph-mon[33235]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-11T13:33:55.374 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:55.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:54 smithi071 ceph-mon[33293]: [11/May/2024:13:33:53] ENGINE Bus STARTING 2024-05-11T13:33:55.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:54 smithi071 ceph-mon[33293]: [11/May/2024:13:33:53] ENGINE Serving on https://172.21.15.63:7150 2024-05-11T13:33:55.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:54 smithi071 ceph-mon[33293]: [11/May/2024:13:33:53] ENGINE Serving on http://172.21.15.63:8765 2024-05-11T13:33:55.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:54 smithi071 ceph-mon[33293]: [11/May/2024:13:33:53] ENGINE Bus STARTED 2024-05-11T13:33:55.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:54 smithi071 ceph-mon[33293]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:55.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:54 smithi071 ceph-mon[33293]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:55.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:54 smithi071 ceph-mon[33293]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-11T13:33:56.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:56.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:56.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.184 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.185 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:55 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.314 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:56.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:56.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:55 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:56.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:33:56.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:56.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:55 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:57.247 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:56 smithi165 ceph-mon[33235]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:57.248 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:56 smithi165 ceph-mon[33235]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-11T13:33:57.258 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:56 smithi063 ceph-mon[29809]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:57.258 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:56 smithi063 ceph-mon[29809]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-11T13:33:57.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:56 smithi071 ceph-mon[33293]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:57.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:56 smithi071 ceph-mon[33293]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-11T13:33:57.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 -- 172.21.15.63:0/2653730167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 msgr2=0x7f712c110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 --2- 172.21.15.63:0/2653730167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c110440 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f712400b0a0 tx=0x7f712400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 -- 172.21.15.63:0/2653730167 shutdown_connections 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 --2- 172.21.15.63:0/2653730167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c110440 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 --2- 172.21.15.63:0/2653730167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1096b0 0x7f712c109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 --2- 172.21.15.63:0/2653730167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1035a0 0x7f712c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 -- 172.21.15.63:0/2653730167 >> 172.21.15.63:0/2653730167 conn(0x7f712c07a700 msgr2=0x7f712c07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 -- 172.21.15.63:0/2653730167 shutdown_connections 2024-05-11T13:33:57.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.622+0000 7f7131a00700 1 -- 172.21.15.63:0/2653730167 wait complete. 2024-05-11T13:33:57.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.623+0000 7f7131a00700 1 Processor -- start 2024-05-11T13:33:57.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.623+0000 7f7131a00700 1 -- start start 2024-05-11T13:33:57.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f7131a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1035a0 0x7f712c1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f7131a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f7131a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1096b0 0x7f712c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f7131a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f712c113630 con 0x7f712c103f60 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f7131a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1134b0 con 0x7f712c1035a0 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f7131a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1137b0 con 0x7f712c1096b0 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f712a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.624+0000 7f712aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1035a0 0x7f712c1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:57.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c1a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33116/0 (socket says 172.21.15.63:33116) 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 -- 172.21.15.63:0/1154594447 learned_addr learned my addr 172.21.15.63:0/1154594447 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712b400700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1096b0 0x7f712c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 -- 172.21.15.63:0/1154594447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1096b0 msgr2=0x7f712c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1096b0 0x7f712c19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 -- 172.21.15.63:0/1154594447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1035a0 msgr2=0x7f712c1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1035a0 0x7f712c1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 -- 172.21.15.63:0/1154594447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c19c1b0 con 0x7f712c103f60 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.625+0000 7f712a000700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c1a1dd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f711800d950 tx=0x7f711800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.626+0000 7f7123400700 1 -- 172.21.15.63:0/1154594447 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118016070 con 0x7f712c103f60 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.626+0000 7f7123400700 1 -- 172.21.15.63:0/1154594447 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f711800fd20 con 0x7f712c103f60 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.626+0000 7f7123400700 1 -- 172.21.15.63:0/1154594447 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711800e3b0 con 0x7f712c103f60 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.626+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c19c440 con 0x7f712c103f60 2024-05-11T13:33:57.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.627+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1a8760 con 0x7f712c103f60 2024-05-11T13:33:57.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.628+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c06a240 con 0x7f712c103f60 2024-05-11T13:33:57.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.630+0000 7f7123400700 1 -- 172.21.15.63:0/1154594447 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f7118019070 con 0x7f712c103f60 2024-05-11T13:33:57.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.632+0000 7f7123400700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b2bd0 0x7f71100b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:33:57.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.632+0000 7f7123400700 1 -- 172.21.15.63:0/1154594447 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f71180e2550 con 0x7f712c103f60 2024-05-11T13:33:57.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.632+0000 7f712aa00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b2bd0 0x7f71100b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:33:57.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.633+0000 7f712aa00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b2bd0 0x7f71100b5090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f712c19cb90 tx=0x7f711c00a380 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:33:57.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.637+0000 7f7123400700 1 -- 172.21.15.63:0/1154594447 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71180a9a30 con 0x7f712c103f60 2024-05-11T13:33:57.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.884+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f712c1100e0 con 0x7f71100b2bd0 2024-05-11T13:33:57.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.908+0000 7f7123400700 1 -- 172.21.15.63:0/1154594447 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339525 (secure 0 0 0) 0x7f712c1100e0 con 0x7f71100b2bd0 2024-05-11T13:33:57.914 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.915+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b2bd0 msgr2=0x7f71100b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.915+0000 7f7131a00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b2bd0 0x7f71100b5090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f712c19cb90 tx=0x7f711c00a380 comp rx=0 tx=0).stop 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.915+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 msgr2=0x7f712c1a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.915+0000 7f7131a00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c1a1dd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f711800d950 tx=0x7f711800dc60 comp rx=0 tx=0).stop 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.916+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 shutdown_connections 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.916+0000 7f7131a00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b2bd0 0x7f71100b5090 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.916+0000 7f7131a00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1096b0 0x7f712c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.916+0000 7f7131a00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1035a0 0x7f712c1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.916+0000 7f7131a00700 1 --2- 172.21.15.63:0/1154594447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c103f60 0x7f712c1a1dd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:33:57.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.916+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 >> 172.21.15.63:0/1154594447 conn(0x7f712c07a700 msgr2=0x7f712c10e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:33:57.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.916+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 shutdown_connections 2024-05-11T13:33:57.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:33:57.917+0000 7f7131a00700 1 -- 172.21.15.63:0/1154594447 wait complete. 2024-05-11T13:33:57.918 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-11T13:33:58.728 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.728 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.728 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.728 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.728 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.728 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi071 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.729 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:33:58.730 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:33:58.731 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:58 smithi063 ceph-mon[29809]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:58.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi071 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:33:58.817 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:33:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:33:58.818 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:58 smithi165 ceph-mon[33235]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:58.940 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-11T13:33:55.969149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195246,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2563,"num_read_kb":1424,"num_write":515,"num_write_kb":3632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":909,"ondisk_log_size":909,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389968,"kb_used_data":15652,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551856,"statfs":{"total":1151940427776,"available":1151541100544,"internally_reserved":0,"allocated":16027648,"data_stored":9466325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.965772"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585661+0000","last_change":"2024-05-11T13:28:35.223417+0000","last_active":"2024-05-11T13:33:52.585661+0000","last_peered":"2024-05-11T13:33:52.585661+0000","last_clean":"2024-05-11T13:33:52.585661+0000","last_became_active":"2024-05-11T13:28:35.222862+0000","last_became_peered":"2024-05-11T13:28:35.222862+0000","last_unstale":"2024-05-11T13:33:52.585661+0000","last_undegraded":"2024-05-11T13:33:52.585661+0000","last_fullsized":"2024-05-11T13:33:52.585661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:20.259043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586553+0000","last_change":"2024-05-11T13:28:35.222289+0000","last_active":"2024-05-11T13:33:52.586553+0000","last_peered":"2024-05-11T13:33:52.586553+0000","last_clean":"2024-05-11T13:33:52.586553+0000","last_became_active":"2024-05-11T13:28:35.221366+0000","last_became_peered":"2024-05-11T13:28:35.221366+0000","last_unstale":"2024-05-11T13:33:52.586553+0000","last_undegraded":"2024-05-11T13:33:52.586553+0000","last_fullsized":"2024-05-11T13:33:52.586553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:02.648140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586045+0000","last_change":"2024-05-11T13:28:35.221271+0000","last_active":"2024-05-11T13:33:52.586045+0000","last_peered":"2024-05-11T13:33:52.586045+0000","last_clean":"2024-05-11T13:33:52.586045+0000","last_became_active":"2024-05-11T13:28:35.220294+0000","last_became_peered":"2024-05-11T13:28:35.220294+0000","last_unstale":"2024-05-11T13:33:52.586045+0000","last_undegraded":"2024-05-11T13:33:52.586045+0000","last_fullsized":"2024-05-11T13:33:52.586045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:04.608042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517653+0000","last_change":"2024-05-11T13:28:35.197643+0000","last_active":"2024-05-11T13:33:52.517653+0000","last_peered":"2024-05-11T13:33:52.517653+0000","last_clean":"2024-05-11T13:33:52.517653+0000","last_became_active":"2024-05-11T13:28:35.196528+0000","last_became_peered":"2024-05-11T13:28:35.196528+0000","last_unstale":"2024-05-11T13:33:52.517653+0000","last_undegraded":"2024-05-11T13:33:52.517653+0000","last_fullsized":"2024-05-11T13:33:52.517653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:02:03.088456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086886+0000","last_change":"2024-05-11T13:28:35.206833+0000","last_active":"2024-05-11T13:33:52.086886+0000","last_peered":"2024-05-11T13:33:52.086886+0000","last_clean":"2024-05-11T13:33:52.086886+0000","last_became_active":"2024-05-11T13:28:35.206287+0000","last_became_peered":"2024-05-11T13:28:35.206287+0000","last_unstale":"2024-05-11T13:33:52.086886+0000","last_undegraded":"2024-05-11T13:33:52.086886+0000","last_fullsized":"2024-05-11T13:33:52.086886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:25:20.576464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109497+0000","last_change":"2024-05-11T13:28:35.218950+0000","last_active":"2024-05-11T13:33:52.109497+0000","last_peered":"2024-05-11T13:33:52.109497+0000","last_clean":"2024-05-11T13:33:52.109497+0000","last_became_active":"2024-05-11T13:28:35.218597+0000","last_became_peered":"2024-05-11T13:28:35.218597+0000","last_unstale":"2024-05-11T13:33:52.109497+0000","last_undegraded":"2024-05-11T13:33:52.109497+0000","last_fullsized":"2024-05-11T13:33:52.109497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:08:42.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131537+0000","last_change":"2024-05-11T13:29:40.541106+0000","last_active":"2024-05-11T13:33:52.131537+0000","last_peered":"2024-05-11T13:33:52.131537+0000","last_clean":"2024-05-11T13:33:52.131537+0000","last_became_active":"2024-05-11T13:29:40.540007+0000","last_became_peered":"2024-05-11T13:29:40.540007+0000","last_unstale":"2024-05-11T13:33:52.131537+0000","last_undegraded":"2024-05-11T13:33:52.131537+0000","last_fullsized":"2024-05-11T13:33:52.131537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:00.596580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084385+0000","last_change":"2024-05-11T13:28:35.205483+0000","last_active":"2024-05-11T13:33:52.084385+0000","last_peered":"2024-05-11T13:33:52.084385+0000","last_clean":"2024-05-11T13:33:52.084385+0000","last_became_active":"2024-05-11T13:28:35.205101+0000","last_became_peered":"2024-05-11T13:28:35.205101+0000","last_unstale":"2024-05-11T13:33:52.084385+0000","last_undegraded":"2024-05-11T13:33:52.084385+0000","last_fullsized":"2024-05-11T13:33:52.084385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:37.917464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954688+0000","last_change":"2024-05-11T13:28:35.217372+0000","last_active":"2024-05-11T13:33:51.954688+0000","last_peered":"2024-05-11T13:33:51.954688+0000","last_clean":"2024-05-11T13:33:51.954688+0000","last_became_active":"2024-05-11T13:28:35.217021+0000","last_became_peered":"2024-05-11T13:28:35.217021+0000","last_unstale":"2024-05-11T13:33:51.954688+0000","last_undegraded":"2024-05-11T13:33:51.954688+0000","last_fullsized":"2024-05-11T13:33:51.954688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:24.273041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.110549+0000","last_change":"2024-05-11T13:29:40.548634+0000","last_active":"2024-05-11T13:33:52.110549+0000","last_peered":"2024-05-11T13:33:52.110549+0000","last_clean":"2024-05-11T13:33:52.110549+0000","last_became_active":"2024-05-11T13:29:40.547536+0000","last_became_peered":"2024-05-11T13:29:40.547536+0000","last_unstale":"2024-05-11T13:33:52.110549+0000","last_undegraded":"2024-05-11T13:33:52.110549+0000","last_fullsized":"2024-05-11T13:33:52.110549+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:15.362317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132495+0000","last_change":"2024-05-11T13:28:35.214091+0000","last_active":"2024-05-11T13:33:52.132495+0000","last_peered":"2024-05-11T13:33:52.132495+0000","last_clean":"2024-05-11T13:33:52.132495+0000","last_became_active":"2024-05-11T13:28:35.213668+0000","last_became_peered":"2024-05-11T13:28:35.213668+0000","last_unstale":"2024-05-11T13:33:52.132495+0000","last_undegraded":"2024-05-11T13:33:52.132495+0000","last_fullsized":"2024-05-11T13:33:52.132495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:19:55.768203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517502+0000","last_change":"2024-05-11T13:28:35.211874+0000","last_active":"2024-05-11T13:33:52.517502+0000","last_peered":"2024-05-11T13:33:52.517502+0000","last_clean":"2024-05-11T13:33:52.517502+0000","last_became_active":"2024-05-11T13:28:35.211503+0000","last_became_peered":"2024-05-11T13:28:35.211503+0000","last_unstale":"2024-05-11T13:33:52.517502+0000","last_undegraded":"2024-05-11T13:33:52.517502+0000","last_fullsized":"2024-05-11T13:33:52.517502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:46.884110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513453+0000","last_change":"2024-05-11T13:28:35.209308+0000","last_active":"2024-05-11T13:33:52.513453+0000","last_peered":"2024-05-11T13:33:52.513453+0000","last_clean":"2024-05-11T13:33:52.513453+0000","last_became_active":"2024-05-11T13:28:35.207927+0000","last_became_peered":"2024-05-11T13:28:35.207927+0000","last_unstale":"2024-05-11T13:33:52.513453+0000","last_undegraded":"2024-05-11T13:33:52.513453+0000","last_fullsized":"2024-05-11T13:33:52.513453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:04.541849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584746+0000","last_change":"2024-05-11T13:28:35.208524+0000","last_active":"2024-05-11T13:33:52.584746+0000","last_peered":"2024-05-11T13:33:52.584746+0000","last_clean":"2024-05-11T13:33:52.584746+0000","last_became_active":"2024-05-11T13:28:35.207875+0000","last_became_peered":"2024-05-11T13:28:35.207875+0000","last_unstale":"2024-05-11T13:33:52.584746+0000","last_undegraded":"2024-05-11T13:33:52.584746+0000","last_fullsized":"2024-05-11T13:33:52.584746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:23:38.896916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585399+0000","last_change":"2024-05-11T13:28:35.216418+0000","last_active":"2024-05-11T13:33:52.585399+0000","last_peered":"2024-05-11T13:33:52.585399+0000","last_clean":"2024-05-11T13:33:52.585399+0000","last_became_active":"2024-05-11T13:28:35.215984+0000","last_became_peered":"2024-05-11T13:28:35.215984+0000","last_unstale":"2024-05-11T13:33:52.585399+0000","last_undegraded":"2024-05-11T13:33:52.585399+0000","last_fullsized":"2024-05-11T13:33:52.585399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:15:13.650821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952413+0000","last_change":"2024-05-11T13:28:35.217701+0000","last_active":"2024-05-11T13:33:51.952413+0000","last_peered":"2024-05-11T13:33:51.952413+0000","last_clean":"2024-05-11T13:33:51.952413+0000","last_became_active":"2024-05-11T13:28:35.217050+0000","last_became_peered":"2024-05-11T13:28:35.217050+0000","last_unstale":"2024-05-11T13:33:51.952413+0000","last_undegraded":"2024-05-11T13:33:51.952413+0000","last_fullsized":"2024-05-11T13:33:51.952413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:29:38.274540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585521+0000","last_change":"2024-05-11T13:28:35.223794+0000","last_active":"2024-05-11T13:33:52.585521+0000","last_peered":"2024-05-11T13:33:52.585521+0000","last_clean":"2024-05-11T13:33:52.585521+0000","last_became_active":"2024-05-11T13:28:35.223388+0000","last_became_peered":"2024-05-11T13:28:35.223388+0000","last_unstale":"2024-05-11T13:33:52.585521+0000","last_undegraded":"2024-05-11T13:33:52.585521+0000","last_fullsized":"2024-05-11T13:33:52.585521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:56:13.272357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948856+0000","last_change":"2024-05-11T13:28:35.216452+0000","last_active":"2024-05-11T13:33:51.948856+0000","last_peered":"2024-05-11T13:33:51.948856+0000","last_clean":"2024-05-11T13:33:51.948856+0000","last_became_active":"2024-05-11T13:28:35.216075+0000","last_became_peered":"2024-05-11T13:28:35.216075+0000","last_unstale":"2024-05-11T13:33:51.948856+0000","last_undegraded":"2024-05-11T13:33:51.948856+0000","last_fullsized":"2024-05-11T13:33:51.948856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:23.012302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954348+0000","last_change":"2024-05-11T13:28:35.206528+0000","last_active":"2024-05-11T13:33:51.954348+0000","last_peered":"2024-05-11T13:33:51.954348+0000","last_clean":"2024-05-11T13:33:51.954348+0000","last_became_active":"2024-05-11T13:28:35.206102+0000","last_became_peered":"2024-05-11T13:28:35.206102+0000","last_unstale":"2024-05-11T13:33:51.954348+0000","last_undegraded":"2024-05-11T13:33:51.954348+0000","last_fullsized":"2024-05-11T13:33:51.954348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.685477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952356+0000","last_change":"2024-05-11T13:30:41.923155+0000","last_active":"2024-05-11T13:33:51.952356+0000","last_peered":"2024-05-11T13:33:51.952356+0000","last_clean":"2024-05-11T13:33:51.952356+0000","last_became_active":"2024-05-11T13:30:41.922484+0000","last_became_peered":"2024-05-11T13:30:41.922484+0000","last_unstale":"2024-05-11T13:33:51.952356+0000","last_undegraded":"2024-05-11T13:33:51.952356+0000","last_fullsized":"2024-05-11T13:33:51.952356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:52.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":297,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.949961+0000","last_change":"2024-05-11T13:28:30.428076+0000","last_active":"2024-05-11T13:33:51.949961+0000","last_peered":"2024-05-11T13:33:51.949961+0000","last_clean":"2024-05-11T13:33:51.949961+0000","last_became_active":"2024-05-11T13:28:30.427519+0000","last_became_peered":"2024-05-11T13:28:30.427519+0000","last_unstale":"2024-05-11T13:33:51.949961+0000","last_undegraded":"2024-05-11T13:33:51.949961+0000","last_fullsized":"2024-05-11T13:33:51.949961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:52.808051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085491+0000","last_change":"2024-05-11T13:28:35.207587+0000","last_active":"2024-05-11T13:33:52.085491+0000","last_peered":"2024-05-11T13:33:52.085491+0000","last_clean":"2024-05-11T13:33:52.085491+0000","last_became_active":"2024-05-11T13:28:35.206692+0000","last_became_peered":"2024-05-11T13:28:35.206692+0000","last_unstale":"2024-05-11T13:33:52.085491+0000","last_undegraded":"2024-05-11T13:33:52.085491+0000","last_fullsized":"2024-05-11T13:33:52.085491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:25.171342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.516989+0000","last_change":"2024-05-11T13:28:49.682649+0000","last_active":"2024-05-11T13:33:52.516989+0000","last_peered":"2024-05-11T13:33:52.516989+0000","last_clean":"2024-05-11T13:33:52.516989+0000","last_became_active":"2024-05-11T13:28:49.682192+0000","last_became_peered":"2024-05-11T13:28:49.682192+0000","last_unstale":"2024-05-11T13:33:52.516989+0000","last_undegraded":"2024-05-11T13:33:52.516989+0000","last_fullsized":"2024-05-11T13:33:52.516989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:01.728705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085711+0000","last_change":"2024-05-11T13:28:30.707050+0000","last_active":"2024-05-11T13:33:52.085711+0000","last_peered":"2024-05-11T13:33:52.085711+0000","last_clean":"2024-05-11T13:33:52.085711+0000","last_became_active":"2024-05-11T13:28:30.706279+0000","last_became_peered":"2024-05-11T13:28:30.706279+0000","last_unstale":"2024-05-11T13:33:52.085711+0000","last_undegraded":"2024-05-11T13:33:52.085711+0000","last_fullsized":"2024-05-11T13:33:52.085711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:14.135734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586854+0000","last_change":"2024-05-11T13:28:35.219622+0000","last_active":"2024-05-11T13:33:52.586854+0000","last_peered":"2024-05-11T13:33:52.586854+0000","last_clean":"2024-05-11T13:33:52.586854+0000","last_became_active":"2024-05-11T13:28:35.218687+0000","last_became_peered":"2024-05-11T13:28:35.218687+0000","last_unstale":"2024-05-11T13:33:52.586854+0000","last_undegraded":"2024-05-11T13:33:52.586854+0000","last_fullsized":"2024-05-11T13:33:52.586854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:53.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517808+0000","last_change":"2024-05-11T13:28:49.685770+0000","last_active":"2024-05-11T13:33:52.517808+0000","last_peered":"2024-05-11T13:33:52.517808+0000","last_clean":"2024-05-11T13:33:52.517808+0000","last_became_active":"2024-05-11T13:28:49.685091+0000","last_became_peered":"2024-05-11T13:28:49.685091+0000","last_unstale":"2024-05-11T13:33:52.517808+0000","last_undegraded":"2024-05-11T13:33:52.517808+0000","last_fullsized":"2024-05-11T13:33:52.517808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:42.458310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087683+0000","last_change":"2024-05-11T13:28:30.435304+0000","last_active":"2024-05-11T13:33:52.087683+0000","last_peered":"2024-05-11T13:33:52.087683+0000","last_clean":"2024-05-11T13:33:52.087683+0000","last_became_active":"2024-05-11T13:28:30.434648+0000","last_became_peered":"2024-05-11T13:28:30.434648+0000","last_unstale":"2024-05-11T13:33:52.087683+0000","last_undegraded":"2024-05-11T13:33:52.087683+0000","last_fullsized":"2024-05-11T13:33:52.087683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:21.553376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953241+0000","last_change":"2024-05-11T13:28:35.210877+0000","last_active":"2024-05-11T13:33:51.953241+0000","last_peered":"2024-05-11T13:33:51.953241+0000","last_clean":"2024-05-11T13:33:51.953241+0000","last_became_active":"2024-05-11T13:28:35.207069+0000","last_became_peered":"2024-05-11T13:28:35.207069+0000","last_unstale":"2024-05-11T13:33:51.953241+0000","last_undegraded":"2024-05-11T13:33:51.953241+0000","last_fullsized":"2024-05-11T13:33:51.953241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:59.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517166+0000","last_change":"2024-05-11T13:28:49.689070+0000","last_active":"2024-05-11T13:33:52.517166+0000","last_peered":"2024-05-11T13:33:52.517166+0000","last_clean":"2024-05-11T13:33:52.517166+0000","last_became_active":"2024-05-11T13:28:49.688502+0000","last_became_peered":"2024-05-11T13:28:49.688502+0000","last_unstale":"2024-05-11T13:33:52.517166+0000","last_undegraded":"2024-05-11T13:33:52.517166+0000","last_fullsized":"2024-05-11T13:33:52.517166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:21.516074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584935+0000","last_change":"2024-05-11T13:29:40.564494+0000","last_active":"2024-05-11T13:33:52.584935+0000","last_peered":"2024-05-11T13:33:52.584935+0000","last_clean":"2024-05-11T13:33:52.584935+0000","last_became_active":"2024-05-11T13:29:40.555386+0000","last_became_peered":"2024-05-11T13:29:40.555386+0000","last_unstale":"2024-05-11T13:33:52.584935+0000","last_undegraded":"2024-05-11T13:33:52.584935+0000","last_fullsized":"2024-05-11T13:33:52.584935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:32:50.641372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101710+0000","last_change":"2024-05-11T13:28:35.217950+0000","last_active":"2024-05-11T13:33:52.101710+0000","last_peered":"2024-05-11T13:33:52.101710+0000","last_clean":"2024-05-11T13:33:52.101710+0000","last_became_active":"2024-05-11T13:28:35.216698+0000","last_became_peered":"2024-05-11T13:28:35.216698+0000","last_unstale":"2024-05-11T13:33:52.101710+0000","last_undegraded":"2024-05-11T13:33:52.101710+0000","last_fullsized":"2024-05-11T13:33:52.101710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:01.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953153+0000","last_change":"2024-05-11T13:30:41.923049+0000","last_active":"2024-05-11T13:33:51.953153+0000","last_peered":"2024-05-11T13:33:51.953153+0000","last_clean":"2024-05-11T13:33:51.953153+0000","last_became_active":"2024-05-11T13:30:41.922585+0000","last_became_peered":"2024-05-11T13:30:41.922585+0000","last_unstale":"2024-05-11T13:33:51.953153+0000","last_undegraded":"2024-05-11T13:33:51.953153+0000","last_fullsized":"2024-05-11T13:33:51.953153+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:34:03.716885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":297,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.494692+0000","last_change":"2024-05-11T13:29:40.555238+0000","last_active":"2024-05-11T13:33:52.494692+0000","last_peered":"2024-05-11T13:33:52.494692+0000","last_clean":"2024-05-11T13:33:52.494692+0000","last_became_active":"2024-05-11T13:29:40.554661+0000","last_became_peered":"2024-05-11T13:29:40.554661+0000","last_unstale":"2024-05-11T13:33:52.494692+0000","last_undegraded":"2024-05-11T13:33:52.494692+0000","last_fullsized":"2024-05-11T13:33:52.494692+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:22.983649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086649+0000","last_change":"2024-05-11T13:28:35.209163+0000","last_active":"2024-05-11T13:33:52.086649+0000","last_peered":"2024-05-11T13:33:52.086649+0000","last_clean":"2024-05-11T13:33:52.086649+0000","last_became_active":"2024-05-11T13:28:35.204014+0000","last_became_peered":"2024-05-11T13:28:35.204014+0000","last_unstale":"2024-05-11T13:33:52.086649+0000","last_undegraded":"2024-05-11T13:33:52.086649+0000","last_fullsized":"2024-05-11T13:33:52.086649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:17.570863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953731+0000","last_change":"2024-05-11T13:29:40.555520+0000","last_active":"2024-05-11T13:33:51.953731+0000","last_peered":"2024-05-11T13:33:51.953731+0000","last_clean":"2024-05-11T13:33:51.953731+0000","last_became_active":"2024-05-11T13:29:40.553952+0000","last_became_peered":"2024-05-11T13:29:40.553952+0000","last_unstale":"2024-05-11T13:33:51.953731+0000","last_undegraded":"2024-05-11T13:33:51.953731+0000","last_fullsized":"2024-05-11T13:33:51.953731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.035691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109404+0000","last_change":"2024-05-11T13:28:30.429829+0000","last_active":"2024-05-11T13:33:52.109404+0000","last_peered":"2024-05-11T13:33:52.109404+0000","last_clean":"2024-05-11T13:33:52.109404+0000","last_became_active":"2024-05-11T13:28:30.428818+0000","last_became_peered":"2024-05-11T13:28:30.428818+0000","last_unstale":"2024-05-11T13:33:52.109404+0000","last_undegraded":"2024-05-11T13:33:52.109404+0000","last_fullsized":"2024-05-11T13:33:52.109404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:59:25.653181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101449+0000","last_change":"2024-05-11T13:28:35.214153+0000","last_active":"2024-05-11T13:33:52.101449+0000","last_peered":"2024-05-11T13:33:52.101449+0000","last_clean":"2024-05-11T13:33:52.101449+0000","last_became_active":"2024-05-11T13:28:35.213800+0000","last_became_peered":"2024-05-11T13:28:35.213800+0000","last_unstale":"2024-05-11T13:33:52.101449+0000","last_undegraded":"2024-05-11T13:33:52.101449+0000","last_fullsized":"2024-05-11T13:33:52.101449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:12.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585398+0000","last_change":"2024-05-11T13:28:49.697320+0000","last_active":"2024-05-11T13:33:52.585398+0000","last_peered":"2024-05-11T13:33:52.585398+0000","last_clean":"2024-05-11T13:33:52.585398+0000","last_became_active":"2024-05-11T13:28:49.696751+0000","last_became_peered":"2024-05-11T13:28:49.696751+0000","last_unstale":"2024-05-11T13:33:52.585398+0000","last_undegraded":"2024-05-11T13:33:52.585398+0000","last_fullsized":"2024-05-11T13:33:52.585398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:24.617241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512264+0000","last_change":"2024-05-11T13:28:30.423943+0000","last_active":"2024-05-11T13:33:52.512264+0000","last_peered":"2024-05-11T13:33:52.512264+0000","last_clean":"2024-05-11T13:33:52.512264+0000","last_became_active":"2024-05-11T13:28:30.423454+0000","last_became_peered":"2024-05-11T13:28:30.423454+0000","last_unstale":"2024-05-11T13:33:52.512264+0000","last_undegraded":"2024-05-11T13:33:52.512264+0000","last_fullsized":"2024-05-11T13:33:52.512264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:23:10.967537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084295+0000","last_change":"2024-05-11T13:28:35.220479+0000","last_active":"2024-05-11T13:33:52.084295+0000","last_peered":"2024-05-11T13:33:52.084295+0000","last_clean":"2024-05-11T13:33:52.084295+0000","last_became_active":"2024-05-11T13:28:35.214541+0000","last_became_peered":"2024-05-11T13:28:35.214541+0000","last_unstale":"2024-05-11T13:33:52.084295+0000","last_undegraded":"2024-05-11T13:33:52.084295+0000","last_fullsized":"2024-05-11T13:33:52.084295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:07.746301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586432+0000","last_change":"2024-05-11T13:28:49.691867+0000","last_active":"2024-05-11T13:33:52.586432+0000","last_peered":"2024-05-11T13:33:52.586432+0000","last_clean":"2024-05-11T13:33:52.586432+0000","last_became_active":"2024-05-11T13:28:49.690045+0000","last_became_peered":"2024-05-11T13:28:49.690045+0000","last_unstale":"2024-05-11T13:33:52.586432+0000","last_undegraded":"2024-05-11T13:33:52.586432+0000","last_fullsized":"2024-05-11T13:33:52.586432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:16.220334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100542+0000","last_change":"2024-05-11T13:28:30.436354+0000","last_active":"2024-05-11T13:33:52.100542+0000","last_peered":"2024-05-11T13:33:52.100542+0000","last_clean":"2024-05-11T13:33:52.100542+0000","last_became_active":"2024-05-11T13:28:30.435983+0000","last_became_peered":"2024-05-11T13:28:30.435983+0000","last_unstale":"2024-05-11T13:33:52.100542+0000","last_undegraded":"2024-05-11T13:33:52.100542+0000","last_fullsized":"2024-05-11T13:33:52.100542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:42.442829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948001+0000","last_change":"2024-05-11T13:28:35.207284+0000","last_active":"2024-05-11T13:33:51.948001+0000","last_peered":"2024-05-11T13:33:51.948001+0000","last_clean":"2024-05-11T13:33:51.948001+0000","last_became_active":"2024-05-11T13:28:35.206945+0000","last_became_peered":"2024-05-11T13:28:35.206945+0000","last_unstale":"2024-05-11T13:33:51.948001+0000","last_undegraded":"2024-05-11T13:33:51.948001+0000","last_fullsized":"2024-05-11T13:33:51.948001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:18.493659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084824+0000","last_change":"2024-05-11T13:28:49.689558+0000","last_active":"2024-05-11T13:33:52.084824+0000","last_peered":"2024-05-11T13:33:52.084824+0000","last_clean":"2024-05-11T13:33:52.084824+0000","last_became_active":"2024-05-11T13:28:49.689156+0000","last_became_peered":"2024-05-11T13:28:49.689156+0000","last_unstale":"2024-05-11T13:33:52.084824+0000","last_undegraded":"2024-05-11T13:33:52.084824+0000","last_fullsized":"2024-05-11T13:33:52.084824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:21.530111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":522,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.492921+0000","last_change":"2024-05-11T13:28:30.434534+0000","last_active":"2024-05-11T13:33:52.492921+0000","last_peered":"2024-05-11T13:33:52.492921+0000","last_clean":"2024-05-11T13:33:52.492921+0000","last_became_active":"2024-05-11T13:28:30.433983+0000","last_became_peered":"2024-05-11T13:28:30.433983+0000","last_unstale":"2024-05-11T13:33:52.492921+0000","last_undegraded":"2024-05-11T13:33:52.492921+0000","last_fullsized":"2024-05-11T13:33:52.492921+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:16.056662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":468,"num_read_kb":468,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953763+0000","last_change":"2024-05-11T13:28:35.205021+0000","last_active":"2024-05-11T13:33:51.953763+0000","last_peered":"2024-05-11T13:33:51.953763+0000","last_clean":"2024-05-11T13:33:51.953763+0000","last_became_active":"2024-05-11T13:28:35.204243+0000","last_became_peered":"2024-05-11T13:28:35.204243+0000","last_unstale":"2024-05-11T13:33:51.953763+0000","last_undegraded":"2024-05-11T13:33:51.953763+0000","last_fullsized":"2024-05-11T13:33:51.953763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:12.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.947982+0000","last_change":"2024-05-11T13:28:49.677392+0000","last_active":"2024-05-11T13:33:51.947982+0000","last_peered":"2024-05-11T13:33:51.947982+0000","last_clean":"2024-05-11T13:33:51.947982+0000","last_became_active":"2024-05-11T13:28:49.674008+0000","last_became_peered":"2024-05-11T13:28:49.674008+0000","last_unstale":"2024-05-11T13:33:51.947982+0000","last_undegraded":"2024-05-11T13:33:51.947982+0000","last_fullsized":"2024-05-11T13:33:51.947982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:47:14.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132482+0000","last_change":"2024-05-11T13:28:30.429500+0000","last_active":"2024-05-11T13:33:52.132482+0000","last_peered":"2024-05-11T13:33:52.132482+0000","last_clean":"2024-05-11T13:33:52.132482+0000","last_became_active":"2024-05-11T13:28:30.429049+0000","last_became_peered":"2024-05-11T13:28:30.429049+0000","last_unstale":"2024-05-11T13:33:52.132482+0000","last_undegraded":"2024-05-11T13:33:52.132482+0000","last_fullsized":"2024-05-11T13:33:52.132482+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:00.533603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586699+0000","last_change":"2024-05-11T13:28:35.223764+0000","last_active":"2024-05-11T13:33:52.586699+0000","last_peered":"2024-05-11T13:33:52.586699+0000","last_clean":"2024-05-11T13:33:52.586699+0000","last_became_active":"2024-05-11T13:28:35.223016+0000","last_became_peered":"2024-05-11T13:28:35.223016+0000","last_unstale":"2024-05-11T13:33:52.586699+0000","last_undegraded":"2024-05-11T13:33:52.586699+0000","last_fullsized":"2024-05-11T13:33:52.586699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:42.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.518233+0000","last_change":"2024-05-11T13:28:49.683733+0000","last_active":"2024-05-11T13:33:52.518233+0000","last_peered":"2024-05-11T13:33:52.518233+0000","last_clean":"2024-05-11T13:33:52.518233+0000","last_became_active":"2024-05-11T13:28:49.681245+0000","last_became_peered":"2024-05-11T13:28:49.681245+0000","last_unstale":"2024-05-11T13:33:52.518233+0000","last_undegraded":"2024-05-11T13:33:52.518233+0000","last_fullsized":"2024-05-11T13:33:52.518233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:03.386943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101317+0000","last_change":"2024-05-11T13:28:30.707394+0000","last_active":"2024-05-11T13:33:52.101317+0000","last_peered":"2024-05-11T13:33:52.101317+0000","last_clean":"2024-05-11T13:33:52.101317+0000","last_became_active":"2024-05-11T13:28:30.706114+0000","last_became_peered":"2024-05-11T13:28:30.706114+0000","last_unstale":"2024-05-11T13:33:52.101317+0000","last_undegraded":"2024-05-11T13:33:52.101317+0000","last_fullsized":"2024-05-11T13:33:52.101317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:53:04.599800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954091+0000","last_change":"2024-05-11T13:28:35.219836+0000","last_active":"2024-05-11T13:33:51.954091+0000","last_peered":"2024-05-11T13:33:51.954091+0000","last_clean":"2024-05-11T13:33:51.954091+0000","last_became_active":"2024-05-11T13:28:35.214474+0000","last_became_peered":"2024-05-11T13:28:35.214474+0000","last_unstale":"2024-05-11T13:33:51.954091+0000","last_undegraded":"2024-05-11T13:33:51.954091+0000","last_fullsized":"2024-05-11T13:33:51.954091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:02.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948036+0000","last_change":"2024-05-11T13:28:49.694677+0000","last_active":"2024-05-11T13:33:51.948036+0000","last_peered":"2024-05-11T13:33:51.948036+0000","last_clean":"2024-05-11T13:33:51.948036+0000","last_became_active":"2024-05-11T13:28:49.694176+0000","last_became_peered":"2024-05-11T13:28:49.694176+0000","last_unstale":"2024-05-11T13:33:51.948036+0000","last_undegraded":"2024-05-11T13:33:51.948036+0000","last_fullsized":"2024-05-11T13:33:51.948036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:35.461937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'42","reported_seq":570,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.578462+0000","last_change":"2024-05-11T13:28:30.433202+0000","last_active":"2024-05-11T13:33:52.578462+0000","last_peered":"2024-05-11T13:33:52.578462+0000","last_clean":"2024-05-11T13:33:52.578462+0000","last_became_active":"2024-05-11T13:28:30.429269+0000","last_became_peered":"2024-05-11T13:28:30.429269+0000","last_unstale":"2024-05-11T13:33:52.578462+0000","last_undegraded":"2024-05-11T13:33:52.578462+0000","last_fullsized":"2024-05-11T13:33:52.578462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:27:15.620717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68479,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":480,"num_read_kb":302,"num_write":44,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087908+0000","last_change":"2024-05-11T13:28:35.210882+0000","last_active":"2024-05-11T13:33:52.087908+0000","last_peered":"2024-05-11T13:33:52.087908+0000","last_clean":"2024-05-11T13:33:52.087908+0000","last_became_active":"2024-05-11T13:28:35.210506+0000","last_became_peered":"2024-05-11T13:28:35.210506+0000","last_unstale":"2024-05-11T13:33:52.087908+0000","last_undegraded":"2024-05-11T13:33:52.087908+0000","last_fullsized":"2024-05-11T13:33:52.087908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:45.752666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.102595+0000","last_change":"2024-05-11T13:28:49.693254+0000","last_active":"2024-05-11T13:33:52.102595+0000","last_peered":"2024-05-11T13:33:52.102595+0000","last_clean":"2024-05-11T13:33:52.102595+0000","last_became_active":"2024-05-11T13:28:49.690307+0000","last_became_peered":"2024-05-11T13:28:49.690307+0000","last_unstale":"2024-05-11T13:33:52.102595+0000","last_undegraded":"2024-05-11T13:33:52.102595+0000","last_fullsized":"2024-05-11T13:33:52.102595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:41:31.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953151+0000","last_change":"2024-05-11T13:28:30.429421+0000","last_active":"2024-05-11T13:33:51.953151+0000","last_peered":"2024-05-11T13:33:51.953151+0000","last_clean":"2024-05-11T13:33:51.953151+0000","last_became_active":"2024-05-11T13:28:30.429032+0000","last_became_peered":"2024-05-11T13:28:30.429032+0000","last_unstale":"2024-05-11T13:33:51.953151+0000","last_undegraded":"2024-05-11T13:33:51.953151+0000","last_fullsized":"2024-05-11T13:33:51.953151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:06.234650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512553+0000","last_change":"2024-05-11T13:28:35.207709+0000","last_active":"2024-05-11T13:33:52.512553+0000","last_peered":"2024-05-11T13:33:52.512553+0000","last_clean":"2024-05-11T13:33:52.512553+0000","last_became_active":"2024-05-11T13:28:35.207222+0000","last_became_peered":"2024-05-11T13:28:35.207222+0000","last_unstale":"2024-05-11T13:33:52.512553+0000","last_undegraded":"2024-05-11T13:33:52.512553+0000","last_fullsized":"2024-05-11T13:33:52.512553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:20.023565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585040+0000","last_change":"2024-05-11T13:28:49.692046+0000","last_active":"2024-05-11T13:33:52.585040+0000","last_peered":"2024-05-11T13:33:52.585040+0000","last_clean":"2024-05-11T13:33:52.585040+0000","last_became_active":"2024-05-11T13:28:49.691618+0000","last_became_peered":"2024-05-11T13:28:49.691618+0000","last_unstale":"2024-05-11T13:33:52.585040+0000","last_undegraded":"2024-05-11T13:33:52.585040+0000","last_fullsized":"2024-05-11T13:33:52.585040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:05.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100851+0000","last_change":"2024-05-11T13:28:30.435459+0000","last_active":"2024-05-11T13:33:52.100851+0000","last_peered":"2024-05-11T13:33:52.100851+0000","last_clean":"2024-05-11T13:33:52.100851+0000","last_became_active":"2024-05-11T13:28:30.434997+0000","last_became_peered":"2024-05-11T13:28:30.434997+0000","last_unstale":"2024-05-11T13:33:52.100851+0000","last_undegraded":"2024-05-11T13:33:52.100851+0000","last_fullsized":"2024-05-11T13:33:52.100851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:36.280546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132009+0000","last_change":"2024-05-11T13:28:35.211214+0000","last_active":"2024-05-11T13:33:52.132009+0000","last_peered":"2024-05-11T13:33:52.132009+0000","last_clean":"2024-05-11T13:33:52.132009+0000","last_became_active":"2024-05-11T13:28:35.210777+0000","last_became_peered":"2024-05-11T13:28:35.210777+0000","last_unstale":"2024-05-11T13:33:52.132009+0000","last_undegraded":"2024-05-11T13:33:52.132009+0000","last_fullsized":"2024-05-11T13:33:52.132009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:53.238549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.950153+0000","last_change":"2024-05-11T13:28:49.682336+0000","last_active":"2024-05-11T13:33:51.950153+0000","last_peered":"2024-05-11T13:33:51.950153+0000","last_clean":"2024-05-11T13:33:51.950153+0000","last_became_active":"2024-05-11T13:28:49.679081+0000","last_became_peered":"2024-05-11T13:28:49.679081+0000","last_unstale":"2024-05-11T13:33:51.950153+0000","last_undegraded":"2024-05-11T13:33:51.950153+0000","last_fullsized":"2024-05-11T13:33:51.950153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:00:21.149262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'10","reported_seq":296,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.582967+0000","last_change":"2024-05-11T13:28:30.432165+0000","last_active":"2024-05-11T13:33:52.582967+0000","last_peered":"2024-05-11T13:33:52.582967+0000","last_clean":"2024-05-11T13:33:52.582967+0000","last_became_active":"2024-05-11T13:28:30.431181+0000","last_became_peered":"2024-05-11T13:28:30.431181+0000","last_unstale":"2024-05-11T13:33:52.582967+0000","last_undegraded":"2024-05-11T13:33:52.582967+0000","last_fullsized":"2024-05-11T13:33:52.582967+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:26:50.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":6,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131539+0000","last_change":"2024-05-11T13:28:35.205662+0000","last_active":"2024-05-11T13:33:52.131539+0000","last_peered":"2024-05-11T13:33:52.131539+0000","last_clean":"2024-05-11T13:33:52.131539+0000","last_became_active":"2024-05-11T13:28:35.205276+0000","last_became_peered":"2024-05-11T13:28:35.205276+0000","last_unstale":"2024-05-11T13:33:52.131539+0000","last_undegraded":"2024-05-11T13:33:52.131539+0000","last_fullsized":"2024-05-11T13:33:52.131539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:36.308402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514146+0000","last_change":"2024-05-11T13:28:49.689705+0000","last_active":"2024-05-11T13:33:52.514146+0000","last_peered":"2024-05-11T13:33:52.514146+0000","last_clean":"2024-05-11T13:33:52.514146+0000","last_became_active":"2024-05-11T13:28:49.689223+0000","last_became_peered":"2024-05-11T13:28:49.689223+0000","last_unstale":"2024-05-11T13:33:52.514146+0000","last_undegraded":"2024-05-11T13:33:52.514146+0000","last_fullsized":"2024-05-11T13:33:52.514146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:37:20.727773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086324+0000","last_change":"2024-05-11T13:28:30.709014+0000","last_active":"2024-05-11T13:33:52.086324+0000","last_peered":"2024-05-11T13:33:52.086324+0000","last_clean":"2024-05-11T13:33:52.086324+0000","last_became_active":"2024-05-11T13:28:30.708000+0000","last_became_peered":"2024-05-11T13:28:30.708000+0000","last_unstale":"2024-05-11T13:33:52.086324+0000","last_undegraded":"2024-05-11T13:33:52.086324+0000","last_fullsized":"2024-05-11T13:33:52.086324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:00.813415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100555+0000","last_change":"2024-05-11T13:28:35.215149+0000","last_active":"2024-05-11T13:33:52.100555+0000","last_peered":"2024-05-11T13:33:52.100555+0000","last_clean":"2024-05-11T13:33:52.100555+0000","last_became_active":"2024-05-11T13:28:35.214792+0000","last_became_peered":"2024-05-11T13:28:35.214792+0000","last_unstale":"2024-05-11T13:33:52.100555+0000","last_undegraded":"2024-05-11T13:33:52.100555+0000","last_fullsized":"2024-05-11T13:33:52.100555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100519+0000","last_change":"2024-05-11T13:28:49.686347+0000","last_active":"2024-05-11T13:33:52.100519+0000","last_peered":"2024-05-11T13:33:52.100519+0000","last_clean":"2024-05-11T13:33:52.100519+0000","last_became_active":"2024-05-11T13:28:49.685391+0000","last_became_peered":"2024-05-11T13:28:49.685391+0000","last_unstale":"2024-05-11T13:33:52.100519+0000","last_undegraded":"2024-05-11T13:33:52.100519+0000","last_fullsized":"2024-05-11T13:33:52.100519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:24:50.347627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585329+0000","last_change":"2024-05-11T13:29:40.560815+0000","last_active":"2024-05-11T13:33:52.585329+0000","last_peered":"2024-05-11T13:33:52.585329+0000","last_clean":"2024-05-11T13:33:52.585329+0000","last_became_active":"2024-05-11T13:29:40.553031+0000","last_became_peered":"2024-05-11T13:29:40.553031+0000","last_unstale":"2024-05-11T13:33:52.585329+0000","last_undegraded":"2024-05-11T13:33:52.585329+0000","last_fullsized":"2024-05-11T13:33:52.585329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:27.437654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585383+0000","last_change":"2024-05-11T13:28:35.218377+0000","last_active":"2024-05-11T13:33:52.585383+0000","last_peered":"2024-05-11T13:33:52.585383+0000","last_clean":"2024-05-11T13:33:52.585383+0000","last_became_active":"2024-05-11T13:28:35.217922+0000","last_became_peered":"2024-05-11T13:28:35.217922+0000","last_unstale":"2024-05-11T13:33:52.585383+0000","last_undegraded":"2024-05-11T13:33:52.585383+0000","last_fullsized":"2024-05-11T13:33:52.585383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:58.383381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086394+0000","last_change":"2024-05-11T13:28:49.723400+0000","last_active":"2024-05-11T13:33:52.086394+0000","last_peered":"2024-05-11T13:33:52.086394+0000","last_clean":"2024-05-11T13:33:52.086394+0000","last_became_active":"2024-05-11T13:28:49.722675+0000","last_became_peered":"2024-05-11T13:28:49.722675+0000","last_unstale":"2024-05-11T13:33:52.086394+0000","last_undegraded":"2024-05-11T13:33:52.086394+0000","last_fullsized":"2024-05-11T13:33:52.086394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:14:44.810541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.949120+0000","last_change":"2024-05-11T13:28:30.710667+0000","last_active":"2024-05-11T13:33:51.949120+0000","last_peered":"2024-05-11T13:33:51.949120+0000","last_clean":"2024-05-11T13:33:51.949120+0000","last_became_active":"2024-05-11T13:28:30.709933+0000","last_became_peered":"2024-05-11T13:28:30.709933+0000","last_unstale":"2024-05-11T13:33:51.949120+0000","last_undegraded":"2024-05-11T13:33:51.949120+0000","last_fullsized":"2024-05-11T13:33:51.949120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:34.942946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109997+0000","last_change":"2024-05-11T13:28:35.215691+0000","last_active":"2024-05-11T13:33:52.109997+0000","last_peered":"2024-05-11T13:33:52.109997+0000","last_clean":"2024-05-11T13:33:52.109997+0000","last_became_active":"2024-05-11T13:28:35.209873+0000","last_became_peered":"2024-05-11T13:28:35.209873+0000","last_unstale":"2024-05-11T13:33:52.109997+0000","last_undegraded":"2024-05-11T13:33:52.109997+0000","last_fullsized":"2024-05-11T13:33:52.109997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:42:12.935570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513453+0000","last_change":"2024-05-11T13:28:49.694322+0000","last_active":"2024-05-11T13:33:52.513453+0000","last_peered":"2024-05-11T13:33:52.513453+0000","last_clean":"2024-05-11T13:33:52.513453+0000","last_became_active":"2024-05-11T13:28:49.690900+0000","last_became_peered":"2024-05-11T13:28:49.690900+0000","last_unstale":"2024-05-11T13:33:52.513453+0000","last_undegraded":"2024-05-11T13:33:52.513453+0000","last_fullsized":"2024-05-11T13:33:52.513453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:00:07.433371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'9","reported_seq":294,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.454975+0000","last_change":"2024-05-11T13:28:30.432529+0000","last_active":"2024-05-11T13:33:52.454975+0000","last_peered":"2024-05-11T13:33:52.454975+0000","last_clean":"2024-05-11T13:33:52.454975+0000","last_became_active":"2024-05-11T13:28:30.428077+0000","last_became_peered":"2024-05-11T13:28:30.428077+0000","last_unstale":"2024-05-11T13:33:52.454975+0000","last_undegraded":"2024-05-11T13:33:52.454975+0000","last_fullsized":"2024-05-11T13:33:52.454975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:42.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":238,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109674+0000","last_change":"2024-05-11T13:28:35.219160+0000","last_active":"2024-05-11T13:33:52.109674+0000","last_peered":"2024-05-11T13:33:52.109674+0000","last_clean":"2024-05-11T13:33:52.109674+0000","last_became_active":"2024-05-11T13:28:35.216810+0000","last_became_peered":"2024-05-11T13:28:35.216810+0000","last_unstale":"2024-05-11T13:33:52.109674+0000","last_undegraded":"2024-05-11T13:33:52.109674+0000","last_fullsized":"2024-05-11T13:33:52.109674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:35:54.465030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514299+0000","last_change":"2024-05-11T13:28:49.725190+0000","last_active":"2024-05-11T13:33:52.514299+0000","last_peered":"2024-05-11T13:33:52.514299+0000","last_clean":"2024-05-11T13:33:52.514299+0000","last_became_active":"2024-05-11T13:28:49.724193+0000","last_became_peered":"2024-05-11T13:28:49.724193+0000","last_unstale":"2024-05-11T13:33:52.514299+0000","last_undegraded":"2024-05-11T13:33:52.514299+0000","last_fullsized":"2024-05-11T13:33:52.514299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:47:41.181606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101663+0000","last_change":"2024-05-11T13:28:35.211262+0000","last_active":"2024-05-11T13:33:52.101663+0000","last_peered":"2024-05-11T13:33:52.101663+0000","last_clean":"2024-05-11T13:33:52.101663+0000","last_became_active":"2024-05-11T13:28:35.210859+0000","last_became_peered":"2024-05-11T13:28:35.210859+0000","last_unstale":"2024-05-11T13:33:52.101663+0000","last_undegraded":"2024-05-11T13:33:52.101663+0000","last_fullsized":"2024-05-11T13:33:52.101663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:40:33.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":290,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512198+0000","last_change":"2024-05-11T13:28:30.707326+0000","last_active":"2024-05-11T13:33:52.512198+0000","last_peered":"2024-05-11T13:33:52.512198+0000","last_clean":"2024-05-11T13:33:52.512198+0000","last_became_active":"2024-05-11T13:28:30.705496+0000","last_became_peered":"2024-05-11T13:28:30.705496+0000","last_unstale":"2024-05-11T13:33:52.512198+0000","last_undegraded":"2024-05-11T13:33:52.512198+0000","last_fullsized":"2024-05-11T13:33:52.512198+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:48.117551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132186+0000","last_change":"2024-05-11T13:28:49.687260+0000","last_active":"2024-05-11T13:33:52.132186+0000","last_peered":"2024-05-11T13:33:52.132186+0000","last_clean":"2024-05-11T13:33:52.132186+0000","last_became_active":"2024-05-11T13:28:49.686593+0000","last_became_peered":"2024-05-11T13:28:49.686593+0000","last_unstale":"2024-05-11T13:33:52.132186+0000","last_undegraded":"2024-05-11T13:33:52.132186+0000","last_fullsized":"2024-05-11T13:33:52.132186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:48.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084761+0000","last_change":"2024-05-11T13:28:35.216643+0000","last_active":"2024-05-11T13:33:52.084761+0000","last_peered":"2024-05-11T13:33:52.084761+0000","last_clean":"2024-05-11T13:33:52.084761+0000","last_became_active":"2024-05-11T13:28:35.216300+0000","last_became_peered":"2024-05-11T13:28:35.216300+0000","last_unstale":"2024-05-11T13:33:52.084761+0000","last_undegraded":"2024-05-11T13:33:52.084761+0000","last_fullsized":"2024-05-11T13:33:52.084761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:20:01.351678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517969+0000","last_change":"2024-05-11T13:28:30.427465+0000","last_active":"2024-05-11T13:33:52.517969+0000","last_peered":"2024-05-11T13:33:52.517969+0000","last_clean":"2024-05-11T13:33:52.517969+0000","last_became_active":"2024-05-11T13:28:30.426968+0000","last_became_peered":"2024-05-11T13:28:30.426968+0000","last_unstale":"2024-05-11T13:33:52.517969+0000","last_undegraded":"2024-05-11T13:33:52.517969+0000","last_fullsized":"2024-05-11T13:33:52.517969+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:02:44.969069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131147+0000","last_change":"2024-05-11T13:26:37.738725+0000","last_active":"2024-05-11T13:33:52.131147+0000","last_peered":"2024-05-11T13:33:52.131147+0000","last_clean":"2024-05-11T13:33:52.131147+0000","last_became_active":"2024-05-11T13:26:37.724302+0000","last_became_peered":"2024-05-11T13:26:37.724302+0000","last_unstale":"2024-05-11T13:33:52.131147+0000","last_undegraded":"2024-05-11T13:33:52.131147+0000","last_fullsized":"2024-05-11T13:33:52.131147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_clean_scrub_stamp":"2024-05-11T13:21:55.918148+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:32:56.277780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101952+0000","last_change":"2024-05-11T13:28:49.723829+0000","last_active":"2024-05-11T13:33:52.101952+0000","last_peered":"2024-05-11T13:33:52.101952+0000","last_clean":"2024-05-11T13:33:52.101952+0000","last_became_active":"2024-05-11T13:28:49.723190+0000","last_became_peered":"2024-05-11T13:28:49.723190+0000","last_unstale":"2024-05-11T13:33:52.101952+0000","last_undegraded":"2024-05-11T13:33:52.101952+0000","last_fullsized":"2024-05-11T13:33:52.101952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.742374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512112+0000","last_change":"2024-05-11T13:28:35.213486+0000","last_active":"2024-05-11T13:33:52.512112+0000","last_peered":"2024-05-11T13:33:52.512112+0000","last_clean":"2024-05-11T13:33:52.512112+0000","last_became_active":"2024-05-11T13:28:35.211962+0000","last_became_peered":"2024-05-11T13:28:35.211962+0000","last_unstale":"2024-05-11T13:33:52.512112+0000","last_undegraded":"2024-05-11T13:33:52.512112+0000","last_fullsized":"2024-05-11T13:33:52.512112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:28:34.046898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131990+0000","last_change":"2024-05-11T13:28:30.431023+0000","last_active":"2024-05-11T13:33:52.131990+0000","last_peered":"2024-05-11T13:33:52.131990+0000","last_clean":"2024-05-11T13:33:52.131990+0000","last_became_active":"2024-05-11T13:28:30.430609+0000","last_became_peered":"2024-05-11T13:28:30.430609+0000","last_unstale":"2024-05-11T13:33:52.131990+0000","last_undegraded":"2024-05-11T13:33:52.131990+0000","last_fullsized":"2024-05-11T13:33:52.131990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:50.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584949+0000","last_change":"2024-05-11T13:28:49.695550+0000","last_active":"2024-05-11T13:33:52.584949+0000","last_peered":"2024-05-11T13:33:52.584949+0000","last_clean":"2024-05-11T13:33:52.584949+0000","last_became_active":"2024-05-11T13:28:49.694350+0000","last_became_peered":"2024-05-11T13:28:49.694350+0000","last_unstale":"2024-05-11T13:33:52.584949+0000","last_undegraded":"2024-05-11T13:33:52.584949+0000","last_fullsized":"2024-05-11T13:33:52.584949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:49.617673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087887+0000","last_change":"2024-05-11T13:28:30.710826+0000","last_active":"2024-05-11T13:33:52.087887+0000","last_peered":"2024-05-11T13:33:52.087887+0000","last_clean":"2024-05-11T13:33:52.087887+0000","last_became_active":"2024-05-11T13:28:30.709854+0000","last_became_peered":"2024-05-11T13:28:30.709854+0000","last_unstale":"2024-05-11T13:33:52.087887+0000","last_undegraded":"2024-05-11T13:33:52.087887+0000","last_fullsized":"2024-05-11T13:33:52.087887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:35:16.384559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513698+0000","last_change":"2024-05-11T13:28:35.205521+0000","last_active":"2024-05-11T13:33:52.513698+0000","last_peered":"2024-05-11T13:33:52.513698+0000","last_clean":"2024-05-11T13:33:52.513698+0000","last_became_active":"2024-05-11T13:28:35.205167+0000","last_became_peered":"2024-05-11T13:28:35.205167+0000","last_unstale":"2024-05-11T13:33:52.513698+0000","last_undegraded":"2024-05-11T13:33:52.513698+0000","last_fullsized":"2024-05-11T13:33:52.513698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:10:27.473991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084255+0000","last_change":"2024-05-11T13:28:49.723652+0000","last_active":"2024-05-11T13:33:52.084255+0000","last_peered":"2024-05-11T13:33:52.084255+0000","last_clean":"2024-05-11T13:33:52.084255+0000","last_became_active":"2024-05-11T13:28:49.723135+0000","last_became_peered":"2024-05-11T13:28:49.723135+0000","last_unstale":"2024-05-11T13:33:52.084255+0000","last_undegraded":"2024-05-11T13:33:52.084255+0000","last_fullsized":"2024-05-11T13:33:52.084255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:21:18.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'51","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.326127+0000","last_change":"2024-05-11T13:28:30.428197+0000","last_active":"2024-05-11T13:33:52.326127+0000","last_peered":"2024-05-11T13:33:52.326127+0000","last_clean":"2024-05-11T13:33:52.326127+0000","last_became_active":"2024-05-11T13:28:30.427753+0000","last_became_peered":"2024-05-11T13:28:30.427753+0000","last_unstale":"2024-05-11T13:33:52.326127+0000","last_undegraded":"2024-05-11T13:33:52.326127+0000","last_fullsized":"2024-05-11T13:33:52.326127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:52:54.165993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.949606+0000","last_change":"2024-05-11T13:29:40.541861+0000","last_active":"2024-05-11T13:33:51.949606+0000","last_peered":"2024-05-11T13:33:51.949606+0000","last_clean":"2024-05-11T13:33:51.949606+0000","last_became_active":"2024-05-11T13:29:40.541395+0000","last_became_peered":"2024-05-11T13:29:40.541395+0000","last_unstale":"2024-05-11T13:33:51.949606+0000","last_undegraded":"2024-05-11T13:33:51.949606+0000","last_fullsized":"2024-05-11T13:33:51.949606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:50:39.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.951951+0000","last_change":"2024-05-11T13:28:49.677549+0000","last_active":"2024-05-11T13:33:51.951951+0000","last_peered":"2024-05-11T13:33:51.951951+0000","last_clean":"2024-05-11T13:33:51.951951+0000","last_became_active":"2024-05-11T13:28:49.676625+0000","last_became_peered":"2024-05-11T13:28:49.676625+0000","last_unstale":"2024-05-11T13:33:51.951951+0000","last_undegraded":"2024-05-11T13:33:51.951951+0000","last_fullsized":"2024-05-11T13:33:51.951951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:44.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.518109+0000","last_change":"2024-05-11T13:28:30.440945+0000","last_active":"2024-05-11T13:33:52.518109+0000","last_peered":"2024-05-11T13:33:52.518109+0000","last_clean":"2024-05-11T13:33:52.518109+0000","last_became_active":"2024-05-11T13:28:30.440595+0000","last_became_peered":"2024-05-11T13:28:30.440595+0000","last_unstale":"2024-05-11T13:33:52.518109+0000","last_undegraded":"2024-05-11T13:33:52.518109+0000","last_fullsized":"2024-05-11T13:33:52.518109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:51:10.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586605+0000","last_change":"2024-05-11T13:28:35.214207+0000","last_active":"2024-05-11T13:33:52.586605+0000","last_peered":"2024-05-11T13:33:52.586605+0000","last_clean":"2024-05-11T13:33:52.586605+0000","last_became_active":"2024-05-11T13:28:35.213844+0000","last_became_peered":"2024-05-11T13:28:35.213844+0000","last_unstale":"2024-05-11T13:33:52.586605+0000","last_undegraded":"2024-05-11T13:33:52.586605+0000","last_fullsized":"2024-05-11T13:33:52.586605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:53.063329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517766+0000","last_change":"2024-05-11T13:28:49.725505+0000","last_active":"2024-05-11T13:33:52.517766+0000","last_peered":"2024-05-11T13:33:52.517766+0000","last_clean":"2024-05-11T13:33:52.517766+0000","last_became_active":"2024-05-11T13:28:49.723613+0000","last_became_peered":"2024-05-11T13:28:49.723613+0000","last_unstale":"2024-05-11T13:33:52.517766+0000","last_undegraded":"2024-05-11T13:33:52.517766+0000","last_fullsized":"2024-05-11T13:33:52.517766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:54:30.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952629+0000","last_change":"2024-05-11T13:28:30.707025+0000","last_active":"2024-05-11T13:33:51.952629+0000","last_peered":"2024-05-11T13:33:51.952629+0000","last_clean":"2024-05-11T13:33:51.952629+0000","last_became_active":"2024-05-11T13:28:30.705853+0000","last_became_peered":"2024-05-11T13:28:30.705853+0000","last_unstale":"2024-05-11T13:33:51.952629+0000","last_undegraded":"2024-05-11T13:33:51.952629+0000","last_fullsized":"2024-05-11T13:33:51.952629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:15:04.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086366+0000","last_change":"2024-05-11T13:28:35.214682+0000","last_active":"2024-05-11T13:33:52.086366+0000","last_peered":"2024-05-11T13:33:52.086366+0000","last_clean":"2024-05-11T13:33:52.086366+0000","last_became_active":"2024-05-11T13:28:35.211581+0000","last_became_peered":"2024-05-11T13:28:35.211581+0000","last_unstale":"2024-05-11T13:33:52.086366+0000","last_undegraded":"2024-05-11T13:33:52.086366+0000","last_fullsized":"2024-05-11T13:33:52.086366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:19.231675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.110551+0000","last_change":"2024-05-11T13:30:41.536065+0000","last_active":"2024-05-11T13:33:52.110551+0000","last_peered":"2024-05-11T13:33:52.110551+0000","last_clean":"2024-05-11T13:33:52.110551+0000","last_became_active":"2024-05-11T13:30:41.535466+0000","last_became_peered":"2024-05-11T13:30:41.535466+0000","last_unstale":"2024-05-11T13:33:52.110551+0000","last_undegraded":"2024-05-11T13:33:52.110551+0000","last_fullsized":"2024-05-11T13:33:52.110551+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:18.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084929+0000","last_change":"2024-05-11T13:28:30.419444+0000","last_active":"2024-05-11T13:33:52.084929+0000","last_peered":"2024-05-11T13:33:52.084929+0000","last_clean":"2024-05-11T13:33:52.084929+0000","last_became_active":"2024-05-11T13:28:30.419064+0000","last_became_peered":"2024-05-11T13:28:30.419064+0000","last_unstale":"2024-05-11T13:33:52.084929+0000","last_undegraded":"2024-05-11T13:33:52.084929+0000","last_fullsized":"2024-05-11T13:33:52.084929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:02.567403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584868+0000","last_change":"2024-05-11T13:28:35.209347+0000","last_active":"2024-05-11T13:33:52.584868+0000","last_peered":"2024-05-11T13:33:52.584868+0000","last_clean":"2024-05-11T13:33:52.584868+0000","last_became_active":"2024-05-11T13:28:35.208912+0000","last_became_peered":"2024-05-11T13:28:35.208912+0000","last_unstale":"2024-05-11T13:33:52.584868+0000","last_undegraded":"2024-05-11T13:33:52.584868+0000","last_fullsized":"2024-05-11T13:33:52.584868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:08:30.832941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517255+0000","last_change":"2024-05-11T13:28:49.677398+0000","last_active":"2024-05-11T13:33:52.517255+0000","last_peered":"2024-05-11T13:33:52.517255+0000","last_clean":"2024-05-11T13:33:52.517255+0000","last_became_active":"2024-05-11T13:28:49.676725+0000","last_became_peered":"2024-05-11T13:28:49.676725+0000","last_unstale":"2024-05-11T13:33:52.517255+0000","last_undegraded":"2024-05-11T13:33:52.517255+0000","last_fullsized":"2024-05-11T13:33:52.517255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:32.476049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952245+0000","last_change":"2024-05-11T13:28:30.428782+0000","last_active":"2024-05-11T13:33:51.952245+0000","last_peered":"2024-05-11T13:33:51.952245+0000","last_clean":"2024-05-11T13:33:51.952245+0000","last_became_active":"2024-05-11T13:28:30.427819+0000","last_became_peered":"2024-05-11T13:28:30.427819+0000","last_unstale":"2024-05-11T13:33:51.952245+0000","last_undegraded":"2024-05-11T13:33:51.952245+0000","last_fullsized":"2024-05-11T13:33:51.952245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:23:24.533765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086186+0000","last_change":"2024-05-11T13:28:35.210095+0000","last_active":"2024-05-11T13:33:52.086186+0000","last_peered":"2024-05-11T13:33:52.086186+0000","last_clean":"2024-05-11T13:33:52.086186+0000","last_became_active":"2024-05-11T13:28:35.208831+0000","last_became_peered":"2024-05-11T13:28:35.208831+0000","last_unstale":"2024-05-11T13:33:52.086186+0000","last_undegraded":"2024-05-11T13:33:52.086186+0000","last_fullsized":"2024-05-11T13:33:52.086186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:06.957821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517277+0000","last_change":"2024-05-11T13:28:49.724791+0000","last_active":"2024-05-11T13:33:52.517277+0000","last_peered":"2024-05-11T13:33:52.517277+0000","last_clean":"2024-05-11T13:33:52.517277+0000","last_became_active":"2024-05-11T13:28:49.723855+0000","last_became_peered":"2024-05-11T13:28:49.723855+0000","last_unstale":"2024-05-11T13:33:52.517277+0000","last_undegraded":"2024-05-11T13:33:52.517277+0000","last_fullsized":"2024-05-11T13:33:52.517277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:34.705331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087095+0000","last_change":"2024-05-11T13:28:30.431206+0000","last_active":"2024-05-11T13:33:52.087095+0000","last_peered":"2024-05-11T13:33:52.087095+0000","last_clean":"2024-05-11T13:33:52.087095+0000","last_became_active":"2024-05-11T13:28:30.430714+0000","last_became_peered":"2024-05-11T13:28:30.430714+0000","last_unstale":"2024-05-11T13:33:52.087095+0000","last_undegraded":"2024-05-11T13:33:52.087095+0000","last_fullsized":"2024-05-11T13:33:52.087095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:29:21.070046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131757+0000","last_change":"2024-05-11T13:28:35.222254+0000","last_active":"2024-05-11T13:33:52.131757+0000","last_peered":"2024-05-11T13:33:52.131757+0000","last_clean":"2024-05-11T13:33:52.131757+0000","last_became_active":"2024-05-11T13:28:35.219539+0000","last_became_peered":"2024-05-11T13:28:35.219539+0000","last_unstale":"2024-05-11T13:33:52.131757+0000","last_undegraded":"2024-05-11T13:33:52.131757+0000","last_fullsized":"2024-05-11T13:33:52.131757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:55.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952971+0000","last_change":"2024-05-11T13:28:49.692168+0000","last_active":"2024-05-11T13:33:51.952971+0000","last_peered":"2024-05-11T13:33:51.952971+0000","last_clean":"2024-05-11T13:33:51.952971+0000","last_became_active":"2024-05-11T13:28:49.691730+0000","last_became_peered":"2024-05-11T13:28:49.691730+0000","last_unstale":"2024-05-11T13:33:51.952971+0000","last_undegraded":"2024-05-11T13:33:51.952971+0000","last_fullsized":"2024-05-11T13:33:51.952971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:02.935554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":291,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.493866+0000","last_change":"2024-05-11T13:28:30.712115+0000","last_active":"2024-05-11T13:33:52.493866+0000","last_peered":"2024-05-11T13:33:52.493866+0000","last_clean":"2024-05-11T13:33:52.493866+0000","last_became_active":"2024-05-11T13:28:30.710613+0000","last_became_peered":"2024-05-11T13:28:30.710613+0000","last_unstale":"2024-05-11T13:33:52.493866+0000","last_undegraded":"2024-05-11T13:33:52.493866+0000","last_fullsized":"2024-05-11T13:33:52.493866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:00.521915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132863+0000","last_change":"2024-05-11T13:28:35.221340+0000","last_active":"2024-05-11T13:33:52.132863+0000","last_peered":"2024-05-11T13:33:52.132863+0000","last_clean":"2024-05-11T13:33:52.132863+0000","last_became_active":"2024-05-11T13:28:35.220702+0000","last_became_peered":"2024-05-11T13:28:35.220702+0000","last_unstale":"2024-05-11T13:33:52.132863+0000","last_undegraded":"2024-05-11T13:33:52.132863+0000","last_fullsized":"2024-05-11T13:33:52.132863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:59:37.313209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584747+0000","last_change":"2024-05-11T13:28:49.682917+0000","last_active":"2024-05-11T13:33:52.584747+0000","last_peered":"2024-05-11T13:33:52.584747+0000","last_clean":"2024-05-11T13:33:52.584747+0000","last_became_active":"2024-05-11T13:28:49.681905+0000","last_became_peered":"2024-05-11T13:28:49.681905+0000","last_unstale":"2024-05-11T13:33:52.584747+0000","last_undegraded":"2024-05-11T13:33:52.584747+0000","last_fullsized":"2024-05-11T13:33:52.584747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:09:26.191565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":96,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.321913+0000","last_change":"2024-05-11T13:28:30.437356+0000","last_active":"2024-05-11T13:33:52.321913+0000","last_peered":"2024-05-11T13:33:52.321913+0000","last_clean":"2024-05-11T13:33:52.321913+0000","last_became_active":"2024-05-11T13:28:30.437013+0000","last_became_peered":"2024-05-11T13:28:30.437013+0000","last_unstale":"2024-05-11T13:33:52.321913+0000","last_undegraded":"2024-05-11T13:33:52.321913+0000","last_fullsized":"2024-05-11T13:33:52.321913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:58.756156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132631+0000","last_change":"2024-05-11T13:28:35.206105+0000","last_active":"2024-05-11T13:33:52.132631+0000","last_peered":"2024-05-11T13:33:52.132631+0000","last_clean":"2024-05-11T13:33:52.132631+0000","last_became_active":"2024-05-11T13:28:35.205684+0000","last_became_peered":"2024-05-11T13:28:35.205684+0000","last_unstale":"2024-05-11T13:33:52.132631+0000","last_undegraded":"2024-05-11T13:33:52.132631+0000","last_fullsized":"2024-05-11T13:33:52.132631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:12.326574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100161+0000","last_change":"2024-05-11T13:28:49.689545+0000","last_active":"2024-05-11T13:33:52.100161+0000","last_peered":"2024-05-11T13:33:52.100161+0000","last_clean":"2024-05-11T13:33:52.100161+0000","last_became_active":"2024-05-11T13:28:49.688528+0000","last_became_peered":"2024-05-11T13:28:49.688528+0000","last_unstale":"2024-05-11T13:33:52.100161+0000","last_undegraded":"2024-05-11T13:33:52.100161+0000","last_fullsized":"2024-05-11T13:33:52.100161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:35.418873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514120+0000","last_change":"2024-05-11T13:29:40.560835+0000","last_active":"2024-05-11T13:33:52.514120+0000","last_peered":"2024-05-11T13:33:52.514120+0000","last_clean":"2024-05-11T13:33:52.514120+0000","last_became_active":"2024-05-11T13:29:40.554310+0000","last_became_peered":"2024-05-11T13:29:40.554310+0000","last_unstale":"2024-05-11T13:33:52.514120+0000","last_undegraded":"2024-05-11T13:33:52.514120+0000","last_fullsized":"2024-05-11T13:33:52.514120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:19:26.805261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132512+0000","last_change":"2024-05-11T13:28:35.220001+0000","last_active":"2024-05-11T13:33:52.132512+0000","last_peered":"2024-05-11T13:33:52.132512+0000","last_clean":"2024-05-11T13:33:52.132512+0000","last_became_active":"2024-05-11T13:28:35.216768+0000","last_became_peered":"2024-05-11T13:28:35.216768+0000","last_unstale":"2024-05-11T13:33:52.132512+0000","last_undegraded":"2024-05-11T13:33:52.132512+0000","last_fullsized":"2024-05-11T13:33:52.132512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:54:23.547689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584998+0000","last_change":"2024-05-11T13:28:35.213481+0000","last_active":"2024-05-11T13:33:52.584998+0000","last_peered":"2024-05-11T13:33:52.584998+0000","last_clean":"2024-05-11T13:33:52.584998+0000","last_became_active":"2024-05-11T13:28:35.212956+0000","last_became_peered":"2024-05-11T13:28:35.212956+0000","last_unstale":"2024-05-11T13:33:52.584998+0000","last_undegraded":"2024-05-11T13:33:52.584998+0000","last_fullsized":"2024-05-11T13:33:52.584998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:14.814155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952702+0000","last_change":"2024-05-11T13:29:40.554654+0000","last_active":"2024-05-11T13:33:51.952702+0000","last_peered":"2024-05-11T13:33:51.952702+0000","last_clean":"2024-05-11T13:33:51.952702+0000","last_became_active":"2024-05-11T13:29:40.551836+0000","last_became_peered":"2024-05-11T13:29:40.551836+0000","last_unstale":"2024-05-11T13:33:51.952702+0000","last_undegraded":"2024-05-11T13:33:51.952702+0000","last_fullsized":"2024-05-11T13:33:51.952702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:49:44.566051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952138+0000","last_change":"2024-05-11T13:28:35.208925+0000","last_active":"2024-05-11T13:33:51.952138+0000","last_peered":"2024-05-11T13:33:51.952138+0000","last_clean":"2024-05-11T13:33:51.952138+0000","last_became_active":"2024-05-11T13:28:35.208055+0000","last_became_peered":"2024-05-11T13:28:35.208055+0000","last_unstale":"2024-05-11T13:33:51.952138+0000","last_undegraded":"2024-05-11T13:33:51.952138+0000","last_fullsized":"2024-05-11T13:33:51.952138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:20.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585665+0000","last_change":"2024-05-11T13:28:35.213226+0000","last_active":"2024-05-11T13:33:52.585665+0000","last_peered":"2024-05-11T13:33:52.585665+0000","last_clean":"2024-05-11T13:33:52.585665+0000","last_became_active":"2024-05-11T13:28:35.212839+0000","last_became_peered":"2024-05-11T13:28:35.212839+0000","last_unstale":"2024-05-11T13:33:52.585665+0000","last_undegraded":"2024-05-11T13:33:52.585665+0000","last_fullsized":"2024-05-11T13:33:52.585665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:37:05.312505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513858+0000","last_change":"2024-05-11T13:29:40.848092+0000","last_active":"2024-05-11T13:33:52.513858+0000","last_peered":"2024-05-11T13:33:52.513858+0000","last_clean":"2024-05-11T13:33:52.513858+0000","last_became_active":"2024-05-11T13:29:40.846950+0000","last_became_peered":"2024-05-11T13:29:40.846950+0000","last_unstale":"2024-05-11T13:33:52.513858+0000","last_undegraded":"2024-05-11T13:33:52.513858+0000","last_fullsized":"2024-05-11T13:33:52.513858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:18:06.675351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585503+0000","last_change":"2024-05-11T13:28:35.217529+0000","last_active":"2024-05-11T13:33:52.585503+0000","last_peered":"2024-05-11T13:33:52.585503+0000","last_clean":"2024-05-11T13:33:52.585503+0000","last_became_active":"2024-05-11T13:28:35.217062+0000","last_became_peered":"2024-05-11T13:28:35.217062+0000","last_unstale":"2024-05-11T13:33:52.585503+0000","last_undegraded":"2024-05-11T13:33:52.585503+0000","last_fullsized":"2024-05-11T13:33:52.585503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:24:33.955794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954490+0000","last_change":"2024-05-11T13:28:35.217777+0000","last_active":"2024-05-11T13:33:51.954490+0000","last_peered":"2024-05-11T13:33:51.954490+0000","last_clean":"2024-05-11T13:33:51.954490+0000","last_became_active":"2024-05-11T13:28:35.217381+0000","last_became_peered":"2024-05-11T13:28:35.217381+0000","last_unstale":"2024-05-11T13:33:51.954490+0000","last_undegraded":"2024-05-11T13:33:51.954490+0000","last_fullsized":"2024-05-11T13:33:51.954490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:43.973192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086598+0000","last_change":"2024-05-11T13:28:35.209826+0000","last_active":"2024-05-11T13:33:52.086598+0000","last_peered":"2024-05-11T13:33:52.086598+0000","last_clean":"2024-05-11T13:33:52.086598+0000","last_became_active":"2024-05-11T13:28:35.209461+0000","last_became_peered":"2024-05-11T13:28:35.209461+0000","last_unstale":"2024-05-11T13:33:52.086598+0000","last_undegraded":"2024-05-11T13:33:52.086598+0000","last_fullsized":"2024-05-11T13:33:52.086598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:03.323045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100553+0000","last_change":"2024-05-11T13:28:35.205252+0000","last_active":"2024-05-11T13:33:52.100553+0000","last_peered":"2024-05-11T13:33:52.100553+0000","last_clean":"2024-05-11T13:33:52.100553+0000","last_became_active":"2024-05-11T13:28:35.204835+0000","last_became_peered":"2024-05-11T13:28:35.204835+0000","last_unstale":"2024-05-11T13:33:52.100553+0000","last_undegraded":"2024-05-11T13:33:52.100553+0000","last_fullsized":"2024-05-11T13:33:52.100553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:10.032439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085049+0000","last_change":"2024-05-11T13:28:35.219718+0000","last_active":"2024-05-11T13:33:52.085049+0000","last_peered":"2024-05-11T13:33:52.085049+0000","last_clean":"2024-05-11T13:33:52.085049+0000","last_became_active":"2024-05-11T13:28:35.219032+0000","last_became_peered":"2024-05-11T13:28:35.219032+0000","last_unstale":"2024-05-11T13:33:52.085049+0000","last_undegraded":"2024-05-11T13:33:52.085049+0000","last_fullsized":"2024-05-11T13:33:52.085049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:03:58.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954180+0000","last_change":"2024-05-11T13:28:35.217820+0000","last_active":"2024-05-11T13:33:51.954180+0000","last_peered":"2024-05-11T13:33:51.954180+0000","last_clean":"2024-05-11T13:33:51.954180+0000","last_became_active":"2024-05-11T13:28:35.216260+0000","last_became_peered":"2024-05-11T13:28:35.216260+0000","last_unstale":"2024-05-11T13:33:51.954180+0000","last_undegraded":"2024-05-11T13:33:51.954180+0000","last_fullsized":"2024-05-11T13:33:51.954180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:20:47.188238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513331+0000","last_change":"2024-05-11T13:28:35.209762+0000","last_active":"2024-05-11T13:33:52.513331+0000","last_peered":"2024-05-11T13:33:52.513331+0000","last_clean":"2024-05-11T13:33:52.513331+0000","last_became_active":"2024-05-11T13:28:35.207404+0000","last_became_peered":"2024-05-11T13:28:35.207404+0000","last_unstale":"2024-05-11T13:33:52.513331+0000","last_undegraded":"2024-05-11T13:33:52.513331+0000","last_fullsized":"2024-05-11T13:33:52.513331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:41:20.808009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585291+0000","last_change":"2024-05-11T13:28:35.215137+0000","last_active":"2024-05-11T13:33:52.585291+0000","last_peered":"2024-05-11T13:33:52.585291+0000","last_clean":"2024-05-11T13:33:52.585291+0000","last_became_active":"2024-05-11T13:28:35.214745+0000","last_became_peered":"2024-05-11T13:28:35.214745+0000","last_unstale":"2024-05-11T13:33:52.585291+0000","last_undegraded":"2024-05-11T13:33:52.585291+0000","last_fullsized":"2024-05-11T13:33:52.585291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:44.381501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604644,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2482,"num_read_kb":1355,"num_write":390,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1813932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678097,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":519843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37184,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707968,"statfs":{"total":95995035648,"available":95956959232,"internally_reserved":0,"allocated":1892352,"data_stored":1317343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1085440,"data_stored":532683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234087,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36768,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708384,"statfs":{"total":95995035648,"available":95957385216,"internally_reserved":0,"allocated":1462272,"data_stored":921577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430318,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1675264,"data_stored":1127957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593845,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":515205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1130496,"data_stored":595000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738526,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":526415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1138688,"data_stored":588579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149835,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1048615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953604,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":520378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790076,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28972,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716180,"statfs":{"total":95995035648,"available":95965368320,"internally_reserved":0,"allocated":1818624,"data_stored":1252730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":203606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T13:33:58.946 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.953 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi071 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:33:58.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:33:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-11T13:33:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-11T13:33:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-11T13:33:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:33:58.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:58 smithi071 ceph-mon[33293]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:33:59.388 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:33:59.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:59 smithi063 ceph-mon[29809]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:33:59.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:59 smithi063 ceph-mon[29809]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:33:59.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:59 smithi063 ceph-mon[29809]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:33:59.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:59 smithi063 ceph-mon[29809]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:33:59.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:33:59 smithi063 ceph-mon[29809]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:33:59.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:59 smithi071 ceph-mon[33293]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:33:59.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:59 smithi071 ceph-mon[33293]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:33:59.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:59 smithi071 ceph-mon[33293]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:33:59.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:59 smithi071 ceph-mon[33293]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:33:59.952 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:33:59 smithi071 ceph-mon[33293]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:34:00.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:59 smithi165 ceph-mon[33235]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:34:00.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:59 smithi165 ceph-mon[33235]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:34:00.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:59 smithi165 ceph-mon[33235]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T13:34:00.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:59 smithi165 ceph-mon[33235]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:34:00.065 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:33:59 smithi165 ceph-mon[33235]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:34:01.094 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:34:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:34:00.812+0000 7f16ee800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:34:01.094 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:34:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:34:00.812+0000 7f16ee800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:34:01.094 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:34:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:34:00.812+0000 7f16ee800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:34:01.095 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:34:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:34:00.812+0000 7f16ee800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:34:01.095 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:34:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:34:00.812+0000 7f16ee800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T13:34:01.398 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:34:01.398 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.398 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.398 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.398 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-05-11T13:34:01.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-05-11T13:34:01.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: Updating smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: Updating smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: Updating smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T13:34:01.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-05-11T13:34:01.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:01.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 -- 172.21.15.63:0/2252188608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2810ff50 msgr2=0x7f7a28112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:01.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 --2- 172.21.15.63:0/2252188608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2810ff50 0x7f7a28112340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000b0a0 tx=0x7f7a2000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:34:01.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 -- 172.21.15.63:0/2252188608 shutdown_connections 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 --2- 172.21.15.63:0/2252188608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2810ff50 0x7f7a28112340 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 --2- 172.21.15.63:0/2252188608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a28075780 0x7f7a28075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 --2- 172.21.15.63:0/2252188608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a280760a0 0x7f7a2810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 -- 172.21.15.63:0/2252188608 >> 172.21.15.63:0/2252188608 conn(0x7f7a2807a6b0 msgr2=0x7f7a2807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 -- 172.21.15.63:0/2252188608 shutdown_connections 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.764+0000 7f7a2f600700 1 -- 172.21.15.63:0/2252188608 wait complete. 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.765+0000 7f7a2f600700 1 Processor -- start 2024-05-11T13:34:01.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.766+0000 7f7a2f600700 1 -- start start 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.766+0000 7f7a2f600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a28075780 0x7f7a281a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.766+0000 7f7a2f600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a280760a0 0x7f7a281a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.766+0000 7f7a2f600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a2810ff50 0x7f7a2819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.766+0000 7f7a2f600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281157b0 con 0x7f7a280760a0 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.766+0000 7f7a2f600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28115630 con 0x7f7a28075780 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.766+0000 7f7a2f600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28115930 con 0x7f7a2810ff50 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a280760a0 0x7f7a281a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:01.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a280760a0 0x7f7a281a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48198/0 (socket says 172.21.15.63:48198) 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 -- 172.21.15.63:0/1127444858 learned_addr learned my addr 172.21.15.63:0/1127444858 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a2cc00700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a28075780 0x7f7a281a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a2d600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a2810ff50 0x7f7a2819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 -- 172.21.15.63:0/1127444858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a2810ff50 msgr2=0x7f7a2819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a2810ff50 0x7f7a2819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 -- 172.21.15.63:0/1127444858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a28075780 msgr2=0x7f7a281a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a28075780 0x7f7a281a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a1fe00700 1 -- 172.21.15.63:0/1127444858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2819c250 con 0x7f7a280760a0 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a2d600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a2810ff50 0x7f7a2819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:34:01.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.767+0000 7f7a2cc00700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a28075780 0x7f7a281a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:01.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.768+0000 7f7a1fe00700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a280760a0 0x7f7a281a1e70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d980 tx=0x7f7a1000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:01.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.768+0000 7f7a26000700 1 -- 172.21.15.63:0/1127444858 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10017070 con 0x7f7a280760a0 2024-05-11T13:34:01.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.768+0000 7f7a26000700 1 -- 172.21.15.63:0/1127444858 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a10010460 con 0x7f7a280760a0 2024-05-11T13:34:01.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.768+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2819c4e0 con 0x7f7a280760a0 2024-05-11T13:34:01.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.768+0000 7f7a26000700 1 -- 172.21.15.63:0/1127444858 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000de60 con 0x7f7a280760a0 2024-05-11T13:34:01.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.768+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a281a8800 con 0x7f7a280760a0 2024-05-11T13:34:01.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.769+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2806a240 con 0x7f7a280760a0 2024-05-11T13:34:01.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.772+0000 7f7a26000700 1 -- 172.21.15.63:0/1127444858 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f7a100108c0 con 0x7f7a280760a0 2024-05-11T13:34:01.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.773+0000 7f7a26000700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7a0c0b2b00 0x7f7a0c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:01.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.774+0000 7f7a26000700 1 -- 172.21.15.63:0/1127444858 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7a100e2700 con 0x7f7a280760a0 2024-05-11T13:34:01.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.774+0000 7f7a2cc00700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7a0c0b2b00 0x7f7a0c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:01.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.774+0000 7f7a2cc00700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7a0c0b2b00 0x7f7a0c0b4fc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005950 tx=0x7f7a180058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:01.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:01.777+0000 7f7a26000700 1 -- 172.21.15.63:0/1127444858 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a100a9c00 con 0x7f7a280760a0 2024-05-11T13:34:02.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.013+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a28063990 con 0x7f7a0c0b2b00 2024-05-11T13:34:02.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.049+0000 7f7a26000700 1 -- 172.21.15.63:0/1127444858 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339455 (secure 0 0 0) 0x7f7a28063990 con 0x7f7a0c0b2b00 2024-05-11T13:34:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.059+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7a0c0b2b00 msgr2=0x7f7a0c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.059+0000 7f7a2f600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7a0c0b2b00 0x7f7a0c0b4fc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005950 tx=0x7f7a180058e0 comp rx=0 tx=0).stop 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.059+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a280760a0 msgr2=0x7f7a281a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.059+0000 7f7a2f600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a280760a0 0x7f7a281a1e70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d980 tx=0x7f7a1000dd40 comp rx=0 tx=0).stop 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.059+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 shutdown_connections 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.059+0000 7f7a2f600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7a0c0b2b00 0x7f7a0c0b4fc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.060+0000 7f7a2f600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a2810ff50 0x7f7a2819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.060+0000 7f7a2f600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a28075780 0x7f7a281a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.060+0000 7f7a2f600700 1 --2- 172.21.15.63:0/1127444858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a280760a0 0x7f7a281a1e70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.060+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 >> 172.21.15.63:0/1127444858 conn(0x7f7a2807a6b0 msgr2=0x7f7a281007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.060+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 shutdown_connections 2024-05-11T13:34:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:02.060+0000 7f7a2f600700 1 -- 172.21.15.63:0/1127444858 wait complete. 2024-05-11T13:34:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-11T13:34:02.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:34:02.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:34:02.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:34:02.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:02.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:34:02.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:34:02.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:34:02.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:02.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:34:02.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:34:02.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:34:02.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:34:02.973 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-05-11T13:34:01.970865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391878,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2934,"num_read_kb":1867,"num_write":655,"num_write_kb":4547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":996,"ondisk_log_size":996,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391160,"kb_used_data":16664,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124550664,"statfs":{"total":1151940427776,"available":1151539879936,"internally_reserved":0,"allocated":17063936,"data_stored":10488128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":443,"num_write":140,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.967489"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585661+0000","last_change":"2024-05-11T13:28:35.223417+0000","last_active":"2024-05-11T13:33:52.585661+0000","last_peered":"2024-05-11T13:33:52.585661+0000","last_clean":"2024-05-11T13:33:52.585661+0000","last_became_active":"2024-05-11T13:28:35.222862+0000","last_became_peered":"2024-05-11T13:28:35.222862+0000","last_unstale":"2024-05-11T13:33:52.585661+0000","last_undegraded":"2024-05-11T13:33:52.585661+0000","last_fullsized":"2024-05-11T13:33:52.585661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:20.259043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586553+0000","last_change":"2024-05-11T13:28:35.222289+0000","last_active":"2024-05-11T13:33:52.586553+0000","last_peered":"2024-05-11T13:33:52.586553+0000","last_clean":"2024-05-11T13:33:52.586553+0000","last_became_active":"2024-05-11T13:28:35.221366+0000","last_became_peered":"2024-05-11T13:28:35.221366+0000","last_unstale":"2024-05-11T13:33:52.586553+0000","last_undegraded":"2024-05-11T13:33:52.586553+0000","last_fullsized":"2024-05-11T13:33:52.586553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:02.648140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586045+0000","last_change":"2024-05-11T13:28:35.221271+0000","last_active":"2024-05-11T13:33:52.586045+0000","last_peered":"2024-05-11T13:33:52.586045+0000","last_clean":"2024-05-11T13:33:52.586045+0000","last_became_active":"2024-05-11T13:28:35.220294+0000","last_became_peered":"2024-05-11T13:28:35.220294+0000","last_unstale":"2024-05-11T13:33:52.586045+0000","last_undegraded":"2024-05-11T13:33:52.586045+0000","last_fullsized":"2024-05-11T13:33:52.586045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:04.608042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517653+0000","last_change":"2024-05-11T13:28:35.197643+0000","last_active":"2024-05-11T13:33:52.517653+0000","last_peered":"2024-05-11T13:33:52.517653+0000","last_clean":"2024-05-11T13:33:52.517653+0000","last_became_active":"2024-05-11T13:28:35.196528+0000","last_became_peered":"2024-05-11T13:28:35.196528+0000","last_unstale":"2024-05-11T13:33:52.517653+0000","last_undegraded":"2024-05-11T13:33:52.517653+0000","last_fullsized":"2024-05-11T13:33:52.517653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:02:03.088456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086886+0000","last_change":"2024-05-11T13:28:35.206833+0000","last_active":"2024-05-11T13:33:52.086886+0000","last_peered":"2024-05-11T13:33:52.086886+0000","last_clean":"2024-05-11T13:33:52.086886+0000","last_became_active":"2024-05-11T13:28:35.206287+0000","last_became_peered":"2024-05-11T13:28:35.206287+0000","last_unstale":"2024-05-11T13:33:52.086886+0000","last_undegraded":"2024-05-11T13:33:52.086886+0000","last_fullsized":"2024-05-11T13:33:52.086886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:25:20.576464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109497+0000","last_change":"2024-05-11T13:28:35.218950+0000","last_active":"2024-05-11T13:33:52.109497+0000","last_peered":"2024-05-11T13:33:52.109497+0000","last_clean":"2024-05-11T13:33:52.109497+0000","last_became_active":"2024-05-11T13:28:35.218597+0000","last_became_peered":"2024-05-11T13:28:35.218597+0000","last_unstale":"2024-05-11T13:33:52.109497+0000","last_undegraded":"2024-05-11T13:33:52.109497+0000","last_fullsized":"2024-05-11T13:33:52.109497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:08:42.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131537+0000","last_change":"2024-05-11T13:29:40.541106+0000","last_active":"2024-05-11T13:33:52.131537+0000","last_peered":"2024-05-11T13:33:52.131537+0000","last_clean":"2024-05-11T13:33:52.131537+0000","last_became_active":"2024-05-11T13:29:40.540007+0000","last_became_peered":"2024-05-11T13:29:40.540007+0000","last_unstale":"2024-05-11T13:33:52.131537+0000","last_undegraded":"2024-05-11T13:33:52.131537+0000","last_fullsized":"2024-05-11T13:33:52.131537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:00.596580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084385+0000","last_change":"2024-05-11T13:28:35.205483+0000","last_active":"2024-05-11T13:33:52.084385+0000","last_peered":"2024-05-11T13:33:52.084385+0000","last_clean":"2024-05-11T13:33:52.084385+0000","last_became_active":"2024-05-11T13:28:35.205101+0000","last_became_peered":"2024-05-11T13:28:35.205101+0000","last_unstale":"2024-05-11T13:33:52.084385+0000","last_undegraded":"2024-05-11T13:33:52.084385+0000","last_fullsized":"2024-05-11T13:33:52.084385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:37.917464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954688+0000","last_change":"2024-05-11T13:28:35.217372+0000","last_active":"2024-05-11T13:33:51.954688+0000","last_peered":"2024-05-11T13:33:51.954688+0000","last_clean":"2024-05-11T13:33:51.954688+0000","last_became_active":"2024-05-11T13:28:35.217021+0000","last_became_peered":"2024-05-11T13:28:35.217021+0000","last_unstale":"2024-05-11T13:33:51.954688+0000","last_undegraded":"2024-05-11T13:33:51.954688+0000","last_fullsized":"2024-05-11T13:33:51.954688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:24.273041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.110549+0000","last_change":"2024-05-11T13:29:40.548634+0000","last_active":"2024-05-11T13:33:52.110549+0000","last_peered":"2024-05-11T13:33:52.110549+0000","last_clean":"2024-05-11T13:33:52.110549+0000","last_became_active":"2024-05-11T13:29:40.547536+0000","last_became_peered":"2024-05-11T13:29:40.547536+0000","last_unstale":"2024-05-11T13:33:52.110549+0000","last_undegraded":"2024-05-11T13:33:52.110549+0000","last_fullsized":"2024-05-11T13:33:52.110549+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:15.362317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132495+0000","last_change":"2024-05-11T13:28:35.214091+0000","last_active":"2024-05-11T13:33:52.132495+0000","last_peered":"2024-05-11T13:33:52.132495+0000","last_clean":"2024-05-11T13:33:52.132495+0000","last_became_active":"2024-05-11T13:28:35.213668+0000","last_became_peered":"2024-05-11T13:28:35.213668+0000","last_unstale":"2024-05-11T13:33:52.132495+0000","last_undegraded":"2024-05-11T13:33:52.132495+0000","last_fullsized":"2024-05-11T13:33:52.132495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:19:55.768203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517502+0000","last_change":"2024-05-11T13:28:35.211874+0000","last_active":"2024-05-11T13:33:52.517502+0000","last_peered":"2024-05-11T13:33:52.517502+0000","last_clean":"2024-05-11T13:33:52.517502+0000","last_became_active":"2024-05-11T13:28:35.211503+0000","last_became_peered":"2024-05-11T13:28:35.211503+0000","last_unstale":"2024-05-11T13:33:52.517502+0000","last_undegraded":"2024-05-11T13:33:52.517502+0000","last_fullsized":"2024-05-11T13:33:52.517502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:46.884110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513453+0000","last_change":"2024-05-11T13:28:35.209308+0000","last_active":"2024-05-11T13:33:52.513453+0000","last_peered":"2024-05-11T13:33:52.513453+0000","last_clean":"2024-05-11T13:33:52.513453+0000","last_became_active":"2024-05-11T13:28:35.207927+0000","last_became_peered":"2024-05-11T13:28:35.207927+0000","last_unstale":"2024-05-11T13:33:52.513453+0000","last_undegraded":"2024-05-11T13:33:52.513453+0000","last_fullsized":"2024-05-11T13:33:52.513453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:04.541849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584746+0000","last_change":"2024-05-11T13:28:35.208524+0000","last_active":"2024-05-11T13:33:52.584746+0000","last_peered":"2024-05-11T13:33:52.584746+0000","last_clean":"2024-05-11T13:33:52.584746+0000","last_became_active":"2024-05-11T13:28:35.207875+0000","last_became_peered":"2024-05-11T13:28:35.207875+0000","last_unstale":"2024-05-11T13:33:52.584746+0000","last_undegraded":"2024-05-11T13:33:52.584746+0000","last_fullsized":"2024-05-11T13:33:52.584746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:23:38.896916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585399+0000","last_change":"2024-05-11T13:28:35.216418+0000","last_active":"2024-05-11T13:33:52.585399+0000","last_peered":"2024-05-11T13:33:52.585399+0000","last_clean":"2024-05-11T13:33:52.585399+0000","last_became_active":"2024-05-11T13:28:35.215984+0000","last_became_peered":"2024-05-11T13:28:35.215984+0000","last_unstale":"2024-05-11T13:33:52.585399+0000","last_undegraded":"2024-05-11T13:33:52.585399+0000","last_fullsized":"2024-05-11T13:33:52.585399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:15:13.650821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952413+0000","last_change":"2024-05-11T13:28:35.217701+0000","last_active":"2024-05-11T13:33:51.952413+0000","last_peered":"2024-05-11T13:33:51.952413+0000","last_clean":"2024-05-11T13:33:51.952413+0000","last_became_active":"2024-05-11T13:28:35.217050+0000","last_became_peered":"2024-05-11T13:28:35.217050+0000","last_unstale":"2024-05-11T13:33:51.952413+0000","last_undegraded":"2024-05-11T13:33:51.952413+0000","last_fullsized":"2024-05-11T13:33:51.952413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:29:38.274540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585521+0000","last_change":"2024-05-11T13:28:35.223794+0000","last_active":"2024-05-11T13:33:52.585521+0000","last_peered":"2024-05-11T13:33:52.585521+0000","last_clean":"2024-05-11T13:33:52.585521+0000","last_became_active":"2024-05-11T13:28:35.223388+0000","last_became_peered":"2024-05-11T13:28:35.223388+0000","last_unstale":"2024-05-11T13:33:52.585521+0000","last_undegraded":"2024-05-11T13:33:52.585521+0000","last_fullsized":"2024-05-11T13:33:52.585521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:56:13.272357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948856+0000","last_change":"2024-05-11T13:28:35.216452+0000","last_active":"2024-05-11T13:33:51.948856+0000","last_peered":"2024-05-11T13:33:51.948856+0000","last_clean":"2024-05-11T13:33:51.948856+0000","last_became_active":"2024-05-11T13:28:35.216075+0000","last_became_peered":"2024-05-11T13:28:35.216075+0000","last_unstale":"2024-05-11T13:33:51.948856+0000","last_undegraded":"2024-05-11T13:33:51.948856+0000","last_fullsized":"2024-05-11T13:33:51.948856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:23.012302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954348+0000","last_change":"2024-05-11T13:28:35.206528+0000","last_active":"2024-05-11T13:33:51.954348+0000","last_peered":"2024-05-11T13:33:51.954348+0000","last_clean":"2024-05-11T13:33:51.954348+0000","last_became_active":"2024-05-11T13:28:35.206102+0000","last_became_peered":"2024-05-11T13:28:35.206102+0000","last_unstale":"2024-05-11T13:33:51.954348+0000","last_undegraded":"2024-05-11T13:33:51.954348+0000","last_fullsized":"2024-05-11T13:33:51.954348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.685477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952356+0000","last_change":"2024-05-11T13:30:41.923155+0000","last_active":"2024-05-11T13:33:51.952356+0000","last_peered":"2024-05-11T13:33:51.952356+0000","last_clean":"2024-05-11T13:33:51.952356+0000","last_became_active":"2024-05-11T13:30:41.922484+0000","last_became_peered":"2024-05-11T13:30:41.922484+0000","last_unstale":"2024-05-11T13:33:51.952356+0000","last_undegraded":"2024-05-11T13:33:51.952356+0000","last_fullsized":"2024-05-11T13:33:51.952356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:52.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":302,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:56.462922+0000","last_change":"2024-05-11T13:28:30.428076+0000","last_active":"2024-05-11T13:33:56.462922+0000","last_peered":"2024-05-11T13:33:56.462922+0000","last_clean":"2024-05-11T13:33:56.462922+0000","last_became_active":"2024-05-11T13:28:30.427519+0000","last_became_peered":"2024-05-11T13:28:30.427519+0000","last_unstale":"2024-05-11T13:33:56.462922+0000","last_undegraded":"2024-05-11T13:33:56.462922+0000","last_fullsized":"2024-05-11T13:33:56.462922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:52.808051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085491+0000","last_change":"2024-05-11T13:28:35.207587+0000","last_active":"2024-05-11T13:33:52.085491+0000","last_peered":"2024-05-11T13:33:52.085491+0000","last_clean":"2024-05-11T13:33:52.085491+0000","last_became_active":"2024-05-11T13:28:35.206692+0000","last_became_peered":"2024-05-11T13:28:35.206692+0000","last_unstale":"2024-05-11T13:33:52.085491+0000","last_undegraded":"2024-05-11T13:33:52.085491+0000","last_fullsized":"2024-05-11T13:33:52.085491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:25.171342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.516989+0000","last_change":"2024-05-11T13:28:49.682649+0000","last_active":"2024-05-11T13:33:52.516989+0000","last_peered":"2024-05-11T13:33:52.516989+0000","last_clean":"2024-05-11T13:33:52.516989+0000","last_became_active":"2024-05-11T13:28:49.682192+0000","last_became_peered":"2024-05-11T13:28:49.682192+0000","last_unstale":"2024-05-11T13:33:52.516989+0000","last_undegraded":"2024-05-11T13:33:52.516989+0000","last_fullsized":"2024-05-11T13:33:52.516989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:01.728705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085711+0000","last_change":"2024-05-11T13:28:30.707050+0000","last_active":"2024-05-11T13:33:52.085711+0000","last_peered":"2024-05-11T13:33:52.085711+0000","last_clean":"2024-05-11T13:33:52.085711+0000","last_became_active":"2024-05-11T13:28:30.706279+0000","last_became_peered":"2024-05-11T13:28:30.706279+0000","last_unstale":"2024-05-11T13:33:52.085711+0000","last_undegraded":"2024-05-11T13:33:52.085711+0000","last_fullsized":"2024-05-11T13:33:52.085711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:14.135734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586854+0000","last_change":"2024-05-11T13:28:35.219622+0000","last_active":"2024-05-11T13:33:52.586854+0000","last_peered":"2024-05-11T13:33:52.586854+0000","last_clean":"2024-05-11T13:33:52.586854+0000","last_became_active":"2024-05-11T13:28:35.218687+0000","last_became_peered":"2024-05-11T13:28:35.218687+0000","last_unstale":"2024-05-11T13:33:52.586854+0000","last_undegraded":"2024-05-11T13:33:52.586854+0000","last_fullsized":"2024-05-11T13:33:52.586854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:53.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517808+0000","last_change":"2024-05-11T13:28:49.685770+0000","last_active":"2024-05-11T13:33:52.517808+0000","last_peered":"2024-05-11T13:33:52.517808+0000","last_clean":"2024-05-11T13:33:52.517808+0000","last_became_active":"2024-05-11T13:28:49.685091+0000","last_became_peered":"2024-05-11T13:28:49.685091+0000","last_unstale":"2024-05-11T13:33:52.517808+0000","last_undegraded":"2024-05-11T13:33:52.517808+0000","last_fullsized":"2024-05-11T13:33:52.517808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:42.458310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087683+0000","last_change":"2024-05-11T13:28:30.435304+0000","last_active":"2024-05-11T13:33:52.087683+0000","last_peered":"2024-05-11T13:33:52.087683+0000","last_clean":"2024-05-11T13:33:52.087683+0000","last_became_active":"2024-05-11T13:28:30.434648+0000","last_became_peered":"2024-05-11T13:28:30.434648+0000","last_unstale":"2024-05-11T13:33:52.087683+0000","last_undegraded":"2024-05-11T13:33:52.087683+0000","last_fullsized":"2024-05-11T13:33:52.087683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:21.553376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953241+0000","last_change":"2024-05-11T13:28:35.210877+0000","last_active":"2024-05-11T13:33:51.953241+0000","last_peered":"2024-05-11T13:33:51.953241+0000","last_clean":"2024-05-11T13:33:51.953241+0000","last_became_active":"2024-05-11T13:28:35.207069+0000","last_became_peered":"2024-05-11T13:28:35.207069+0000","last_unstale":"2024-05-11T13:33:51.953241+0000","last_undegraded":"2024-05-11T13:33:51.953241+0000","last_fullsized":"2024-05-11T13:33:51.953241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:59.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517166+0000","last_change":"2024-05-11T13:28:49.689070+0000","last_active":"2024-05-11T13:33:52.517166+0000","last_peered":"2024-05-11T13:33:52.517166+0000","last_clean":"2024-05-11T13:33:52.517166+0000","last_became_active":"2024-05-11T13:28:49.688502+0000","last_became_peered":"2024-05-11T13:28:49.688502+0000","last_unstale":"2024-05-11T13:33:52.517166+0000","last_undegraded":"2024-05-11T13:33:52.517166+0000","last_fullsized":"2024-05-11T13:33:52.517166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:21.516074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584935+0000","last_change":"2024-05-11T13:29:40.564494+0000","last_active":"2024-05-11T13:33:52.584935+0000","last_peered":"2024-05-11T13:33:52.584935+0000","last_clean":"2024-05-11T13:33:52.584935+0000","last_became_active":"2024-05-11T13:29:40.555386+0000","last_became_peered":"2024-05-11T13:29:40.555386+0000","last_unstale":"2024-05-11T13:33:52.584935+0000","last_undegraded":"2024-05-11T13:33:52.584935+0000","last_fullsized":"2024-05-11T13:33:52.584935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:32:50.641372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101710+0000","last_change":"2024-05-11T13:28:35.217950+0000","last_active":"2024-05-11T13:33:52.101710+0000","last_peered":"2024-05-11T13:33:52.101710+0000","last_clean":"2024-05-11T13:33:52.101710+0000","last_became_active":"2024-05-11T13:28:35.216698+0000","last_became_peered":"2024-05-11T13:28:35.216698+0000","last_unstale":"2024-05-11T13:33:52.101710+0000","last_undegraded":"2024-05-11T13:33:52.101710+0000","last_fullsized":"2024-05-11T13:33:52.101710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:01.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953153+0000","last_change":"2024-05-11T13:30:41.923049+0000","last_active":"2024-05-11T13:33:51.953153+0000","last_peered":"2024-05-11T13:33:51.953153+0000","last_clean":"2024-05-11T13:33:51.953153+0000","last_became_active":"2024-05-11T13:30:41.922585+0000","last_became_peered":"2024-05-11T13:30:41.922585+0000","last_unstale":"2024-05-11T13:33:51.953153+0000","last_undegraded":"2024-05-11T13:33:51.953153+0000","last_fullsized":"2024-05-11T13:33:51.953153+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:34:03.716885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":303,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:56.504298+0000","last_change":"2024-05-11T13:29:40.555238+0000","last_active":"2024-05-11T13:33:56.504298+0000","last_peered":"2024-05-11T13:33:56.504298+0000","last_clean":"2024-05-11T13:33:56.504298+0000","last_became_active":"2024-05-11T13:29:40.554661+0000","last_became_peered":"2024-05-11T13:29:40.554661+0000","last_unstale":"2024-05-11T13:33:56.504298+0000","last_undegraded":"2024-05-11T13:33:56.504298+0000","last_fullsized":"2024-05-11T13:33:56.504298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:22.983649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":243,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086649+0000","last_change":"2024-05-11T13:28:35.209163+0000","last_active":"2024-05-11T13:33:52.086649+0000","last_peered":"2024-05-11T13:33:52.086649+0000","last_clean":"2024-05-11T13:33:52.086649+0000","last_became_active":"2024-05-11T13:28:35.204014+0000","last_became_peered":"2024-05-11T13:28:35.204014+0000","last_unstale":"2024-05-11T13:33:52.086649+0000","last_undegraded":"2024-05-11T13:33:52.086649+0000","last_fullsized":"2024-05-11T13:33:52.086649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:17.570863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953731+0000","last_change":"2024-05-11T13:29:40.555520+0000","last_active":"2024-05-11T13:33:51.953731+0000","last_peered":"2024-05-11T13:33:51.953731+0000","last_clean":"2024-05-11T13:33:51.953731+0000","last_became_active":"2024-05-11T13:29:40.553952+0000","last_became_peered":"2024-05-11T13:29:40.553952+0000","last_unstale":"2024-05-11T13:33:51.953731+0000","last_undegraded":"2024-05-11T13:33:51.953731+0000","last_fullsized":"2024-05-11T13:33:51.953731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.035691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109404+0000","last_change":"2024-05-11T13:28:30.429829+0000","last_active":"2024-05-11T13:33:52.109404+0000","last_peered":"2024-05-11T13:33:52.109404+0000","last_clean":"2024-05-11T13:33:52.109404+0000","last_became_active":"2024-05-11T13:28:30.428818+0000","last_became_peered":"2024-05-11T13:28:30.428818+0000","last_unstale":"2024-05-11T13:33:52.109404+0000","last_undegraded":"2024-05-11T13:33:52.109404+0000","last_fullsized":"2024-05-11T13:33:52.109404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:59:25.653181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101449+0000","last_change":"2024-05-11T13:28:35.214153+0000","last_active":"2024-05-11T13:33:52.101449+0000","last_peered":"2024-05-11T13:33:52.101449+0000","last_clean":"2024-05-11T13:33:52.101449+0000","last_became_active":"2024-05-11T13:28:35.213800+0000","last_became_peered":"2024-05-11T13:28:35.213800+0000","last_unstale":"2024-05-11T13:33:52.101449+0000","last_undegraded":"2024-05-11T13:33:52.101449+0000","last_fullsized":"2024-05-11T13:33:52.101449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:12.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585398+0000","last_change":"2024-05-11T13:28:49.697320+0000","last_active":"2024-05-11T13:33:52.585398+0000","last_peered":"2024-05-11T13:33:52.585398+0000","last_clean":"2024-05-11T13:33:52.585398+0000","last_became_active":"2024-05-11T13:28:49.696751+0000","last_became_peered":"2024-05-11T13:28:49.696751+0000","last_unstale":"2024-05-11T13:33:52.585398+0000","last_undegraded":"2024-05-11T13:33:52.585398+0000","last_fullsized":"2024-05-11T13:33:52.585398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:24.617241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512264+0000","last_change":"2024-05-11T13:28:30.423943+0000","last_active":"2024-05-11T13:33:52.512264+0000","last_peered":"2024-05-11T13:33:52.512264+0000","last_clean":"2024-05-11T13:33:52.512264+0000","last_became_active":"2024-05-11T13:28:30.423454+0000","last_became_peered":"2024-05-11T13:28:30.423454+0000","last_unstale":"2024-05-11T13:33:52.512264+0000","last_undegraded":"2024-05-11T13:33:52.512264+0000","last_fullsized":"2024-05-11T13:33:52.512264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:23:10.967537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084295+0000","last_change":"2024-05-11T13:28:35.220479+0000","last_active":"2024-05-11T13:33:52.084295+0000","last_peered":"2024-05-11T13:33:52.084295+0000","last_clean":"2024-05-11T13:33:52.084295+0000","last_became_active":"2024-05-11T13:28:35.214541+0000","last_became_peered":"2024-05-11T13:28:35.214541+0000","last_unstale":"2024-05-11T13:33:52.084295+0000","last_undegraded":"2024-05-11T13:33:52.084295+0000","last_fullsized":"2024-05-11T13:33:52.084295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:07.746301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586432+0000","last_change":"2024-05-11T13:28:49.691867+0000","last_active":"2024-05-11T13:33:52.586432+0000","last_peered":"2024-05-11T13:33:52.586432+0000","last_clean":"2024-05-11T13:33:52.586432+0000","last_became_active":"2024-05-11T13:28:49.690045+0000","last_became_peered":"2024-05-11T13:28:49.690045+0000","last_unstale":"2024-05-11T13:33:52.586432+0000","last_undegraded":"2024-05-11T13:33:52.586432+0000","last_fullsized":"2024-05-11T13:33:52.586432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:16.220334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100542+0000","last_change":"2024-05-11T13:28:30.436354+0000","last_active":"2024-05-11T13:33:52.100542+0000","last_peered":"2024-05-11T13:33:52.100542+0000","last_clean":"2024-05-11T13:33:52.100542+0000","last_became_active":"2024-05-11T13:28:30.435983+0000","last_became_peered":"2024-05-11T13:28:30.435983+0000","last_unstale":"2024-05-11T13:33:52.100542+0000","last_undegraded":"2024-05-11T13:33:52.100542+0000","last_fullsized":"2024-05-11T13:33:52.100542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:42.442829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948001+0000","last_change":"2024-05-11T13:28:35.207284+0000","last_active":"2024-05-11T13:33:51.948001+0000","last_peered":"2024-05-11T13:33:51.948001+0000","last_clean":"2024-05-11T13:33:51.948001+0000","last_became_active":"2024-05-11T13:28:35.206945+0000","last_became_peered":"2024-05-11T13:28:35.206945+0000","last_unstale":"2024-05-11T13:33:51.948001+0000","last_undegraded":"2024-05-11T13:33:51.948001+0000","last_fullsized":"2024-05-11T13:33:51.948001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:18.493659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084824+0000","last_change":"2024-05-11T13:28:49.689558+0000","last_active":"2024-05-11T13:33:52.084824+0000","last_peered":"2024-05-11T13:33:52.084824+0000","last_clean":"2024-05-11T13:33:52.084824+0000","last_became_active":"2024-05-11T13:28:49.689156+0000","last_became_peered":"2024-05-11T13:28:49.689156+0000","last_unstale":"2024-05-11T13:33:52.084824+0000","last_undegraded":"2024-05-11T13:33:52.084824+0000","last_fullsized":"2024-05-11T13:33:52.084824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:21.530111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":532,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:57.505037+0000","last_change":"2024-05-11T13:28:30.434534+0000","last_active":"2024-05-11T13:33:57.505037+0000","last_peered":"2024-05-11T13:33:57.505037+0000","last_clean":"2024-05-11T13:33:57.505037+0000","last_became_active":"2024-05-11T13:28:30.433983+0000","last_became_peered":"2024-05-11T13:28:30.433983+0000","last_unstale":"2024-05-11T13:33:57.505037+0000","last_undegraded":"2024-05-11T13:33:57.505037+0000","last_fullsized":"2024-05-11T13:33:57.505037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:16.056662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":478,"num_read_kb":478,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953763+0000","last_change":"2024-05-11T13:28:35.205021+0000","last_active":"2024-05-11T13:33:51.953763+0000","last_peered":"2024-05-11T13:33:51.953763+0000","last_clean":"2024-05-11T13:33:51.953763+0000","last_became_active":"2024-05-11T13:28:35.204243+0000","last_became_peered":"2024-05-11T13:28:35.204243+0000","last_unstale":"2024-05-11T13:33:51.953763+0000","last_undegraded":"2024-05-11T13:33:51.953763+0000","last_fullsized":"2024-05-11T13:33:51.953763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:12.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.947982+0000","last_change":"2024-05-11T13:28:49.677392+0000","last_active":"2024-05-11T13:33:51.947982+0000","last_peered":"2024-05-11T13:33:51.947982+0000","last_clean":"2024-05-11T13:33:51.947982+0000","last_became_active":"2024-05-11T13:28:49.674008+0000","last_became_peered":"2024-05-11T13:28:49.674008+0000","last_unstale":"2024-05-11T13:33:51.947982+0000","last_undegraded":"2024-05-11T13:33:51.947982+0000","last_fullsized":"2024-05-11T13:33:51.947982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:47:14.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132482+0000","last_change":"2024-05-11T13:28:30.429500+0000","last_active":"2024-05-11T13:33:52.132482+0000","last_peered":"2024-05-11T13:33:52.132482+0000","last_clean":"2024-05-11T13:33:52.132482+0000","last_became_active":"2024-05-11T13:28:30.429049+0000","last_became_peered":"2024-05-11T13:28:30.429049+0000","last_unstale":"2024-05-11T13:33:52.132482+0000","last_undegraded":"2024-05-11T13:33:52.132482+0000","last_fullsized":"2024-05-11T13:33:52.132482+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:00.533603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586699+0000","last_change":"2024-05-11T13:28:35.223764+0000","last_active":"2024-05-11T13:33:52.586699+0000","last_peered":"2024-05-11T13:33:52.586699+0000","last_clean":"2024-05-11T13:33:52.586699+0000","last_became_active":"2024-05-11T13:28:35.223016+0000","last_became_peered":"2024-05-11T13:28:35.223016+0000","last_unstale":"2024-05-11T13:33:52.586699+0000","last_undegraded":"2024-05-11T13:33:52.586699+0000","last_fullsized":"2024-05-11T13:33:52.586699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:42.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.518233+0000","last_change":"2024-05-11T13:28:49.683733+0000","last_active":"2024-05-11T13:33:52.518233+0000","last_peered":"2024-05-11T13:33:52.518233+0000","last_clean":"2024-05-11T13:33:52.518233+0000","last_became_active":"2024-05-11T13:28:49.681245+0000","last_became_peered":"2024-05-11T13:28:49.681245+0000","last_unstale":"2024-05-11T13:33:52.518233+0000","last_undegraded":"2024-05-11T13:33:52.518233+0000","last_fullsized":"2024-05-11T13:33:52.518233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:03.386943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101317+0000","last_change":"2024-05-11T13:28:30.707394+0000","last_active":"2024-05-11T13:33:52.101317+0000","last_peered":"2024-05-11T13:33:52.101317+0000","last_clean":"2024-05-11T13:33:52.101317+0000","last_became_active":"2024-05-11T13:28:30.706114+0000","last_became_peered":"2024-05-11T13:28:30.706114+0000","last_unstale":"2024-05-11T13:33:52.101317+0000","last_undegraded":"2024-05-11T13:33:52.101317+0000","last_fullsized":"2024-05-11T13:33:52.101317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:53:04.599800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954091+0000","last_change":"2024-05-11T13:28:35.219836+0000","last_active":"2024-05-11T13:33:51.954091+0000","last_peered":"2024-05-11T13:33:51.954091+0000","last_clean":"2024-05-11T13:33:51.954091+0000","last_became_active":"2024-05-11T13:28:35.214474+0000","last_became_peered":"2024-05-11T13:28:35.214474+0000","last_unstale":"2024-05-11T13:33:51.954091+0000","last_undegraded":"2024-05-11T13:33:51.954091+0000","last_fullsized":"2024-05-11T13:33:51.954091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:02.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948036+0000","last_change":"2024-05-11T13:28:49.694677+0000","last_active":"2024-05-11T13:33:51.948036+0000","last_peered":"2024-05-11T13:33:51.948036+0000","last_clean":"2024-05-11T13:33:51.948036+0000","last_became_active":"2024-05-11T13:28:49.694176+0000","last_became_peered":"2024-05-11T13:28:49.694176+0000","last_unstale":"2024-05-11T13:33:51.948036+0000","last_undegraded":"2024-05-11T13:33:51.948036+0000","last_fullsized":"2024-05-11T13:33:51.948036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:35.461937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'42","reported_seq":581,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:57.508637+0000","last_change":"2024-05-11T13:28:30.433202+0000","last_active":"2024-05-11T13:33:57.508637+0000","last_peered":"2024-05-11T13:33:57.508637+0000","last_clean":"2024-05-11T13:33:57.508637+0000","last_became_active":"2024-05-11T13:28:30.429269+0000","last_became_peered":"2024-05-11T13:28:30.429269+0000","last_unstale":"2024-05-11T13:33:57.508637+0000","last_undegraded":"2024-05-11T13:33:57.508637+0000","last_fullsized":"2024-05-11T13:33:57.508637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:27:15.620717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68479,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":491,"num_read_kb":312,"num_write":44,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087908+0000","last_change":"2024-05-11T13:28:35.210882+0000","last_active":"2024-05-11T13:33:52.087908+0000","last_peered":"2024-05-11T13:33:52.087908+0000","last_clean":"2024-05-11T13:33:52.087908+0000","last_became_active":"2024-05-11T13:28:35.210506+0000","last_became_peered":"2024-05-11T13:28:35.210506+0000","last_unstale":"2024-05-11T13:33:52.087908+0000","last_undegraded":"2024-05-11T13:33:52.087908+0000","last_fullsized":"2024-05-11T13:33:52.087908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:45.752666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.102595+0000","last_change":"2024-05-11T13:28:49.693254+0000","last_active":"2024-05-11T13:33:52.102595+0000","last_peered":"2024-05-11T13:33:52.102595+0000","last_clean":"2024-05-11T13:33:52.102595+0000","last_became_active":"2024-05-11T13:28:49.690307+0000","last_became_peered":"2024-05-11T13:28:49.690307+0000","last_unstale":"2024-05-11T13:33:52.102595+0000","last_undegraded":"2024-05-11T13:33:52.102595+0000","last_fullsized":"2024-05-11T13:33:52.102595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:41:31.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953151+0000","last_change":"2024-05-11T13:28:30.429421+0000","last_active":"2024-05-11T13:33:51.953151+0000","last_peered":"2024-05-11T13:33:51.953151+0000","last_clean":"2024-05-11T13:33:51.953151+0000","last_became_active":"2024-05-11T13:28:30.429032+0000","last_became_peered":"2024-05-11T13:28:30.429032+0000","last_unstale":"2024-05-11T13:33:51.953151+0000","last_undegraded":"2024-05-11T13:33:51.953151+0000","last_fullsized":"2024-05-11T13:33:51.953151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:06.234650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512553+0000","last_change":"2024-05-11T13:28:35.207709+0000","last_active":"2024-05-11T13:33:52.512553+0000","last_peered":"2024-05-11T13:33:52.512553+0000","last_clean":"2024-05-11T13:33:52.512553+0000","last_became_active":"2024-05-11T13:28:35.207222+0000","last_became_peered":"2024-05-11T13:28:35.207222+0000","last_unstale":"2024-05-11T13:33:52.512553+0000","last_undegraded":"2024-05-11T13:33:52.512553+0000","last_fullsized":"2024-05-11T13:33:52.512553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:20.023565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585040+0000","last_change":"2024-05-11T13:28:49.692046+0000","last_active":"2024-05-11T13:33:52.585040+0000","last_peered":"2024-05-11T13:33:52.585040+0000","last_clean":"2024-05-11T13:33:52.585040+0000","last_became_active":"2024-05-11T13:28:49.691618+0000","last_became_peered":"2024-05-11T13:28:49.691618+0000","last_unstale":"2024-05-11T13:33:52.585040+0000","last_undegraded":"2024-05-11T13:33:52.585040+0000","last_fullsized":"2024-05-11T13:33:52.585040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:05.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100851+0000","last_change":"2024-05-11T13:28:30.435459+0000","last_active":"2024-05-11T13:33:52.100851+0000","last_peered":"2024-05-11T13:33:52.100851+0000","last_clean":"2024-05-11T13:33:52.100851+0000","last_became_active":"2024-05-11T13:28:30.434997+0000","last_became_peered":"2024-05-11T13:28:30.434997+0000","last_unstale":"2024-05-11T13:33:52.100851+0000","last_undegraded":"2024-05-11T13:33:52.100851+0000","last_fullsized":"2024-05-11T13:33:52.100851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:36.280546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132009+0000","last_change":"2024-05-11T13:28:35.211214+0000","last_active":"2024-05-11T13:33:52.132009+0000","last_peered":"2024-05-11T13:33:52.132009+0000","last_clean":"2024-05-11T13:33:52.132009+0000","last_became_active":"2024-05-11T13:28:35.210777+0000","last_became_peered":"2024-05-11T13:28:35.210777+0000","last_unstale":"2024-05-11T13:33:52.132009+0000","last_undegraded":"2024-05-11T13:33:52.132009+0000","last_fullsized":"2024-05-11T13:33:52.132009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:53.238549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.950153+0000","last_change":"2024-05-11T13:28:49.682336+0000","last_active":"2024-05-11T13:33:51.950153+0000","last_peered":"2024-05-11T13:33:51.950153+0000","last_clean":"2024-05-11T13:33:51.950153+0000","last_became_active":"2024-05-11T13:28:49.679081+0000","last_became_peered":"2024-05-11T13:28:49.679081+0000","last_unstale":"2024-05-11T13:33:51.950153+0000","last_undegraded":"2024-05-11T13:33:51.950153+0000","last_fullsized":"2024-05-11T13:33:51.950153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:00:21.149262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'10","reported_seq":302,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:56.527802+0000","last_change":"2024-05-11T13:28:30.432165+0000","last_active":"2024-05-11T13:33:56.527802+0000","last_peered":"2024-05-11T13:33:56.527802+0000","last_clean":"2024-05-11T13:33:56.527802+0000","last_became_active":"2024-05-11T13:28:30.431181+0000","last_became_peered":"2024-05-11T13:28:30.431181+0000","last_unstale":"2024-05-11T13:33:56.527802+0000","last_undegraded":"2024-05-11T13:33:56.527802+0000","last_fullsized":"2024-05-11T13:33:56.527802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:26:50.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":246,"num_read_kb":7,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131539+0000","last_change":"2024-05-11T13:28:35.205662+0000","last_active":"2024-05-11T13:33:52.131539+0000","last_peered":"2024-05-11T13:33:52.131539+0000","last_clean":"2024-05-11T13:33:52.131539+0000","last_became_active":"2024-05-11T13:28:35.205276+0000","last_became_peered":"2024-05-11T13:28:35.205276+0000","last_unstale":"2024-05-11T13:33:52.131539+0000","last_undegraded":"2024-05-11T13:33:52.131539+0000","last_fullsized":"2024-05-11T13:33:52.131539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:36.308402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514146+0000","last_change":"2024-05-11T13:28:49.689705+0000","last_active":"2024-05-11T13:33:52.514146+0000","last_peered":"2024-05-11T13:33:52.514146+0000","last_clean":"2024-05-11T13:33:52.514146+0000","last_became_active":"2024-05-11T13:28:49.689223+0000","last_became_peered":"2024-05-11T13:28:49.689223+0000","last_unstale":"2024-05-11T13:33:52.514146+0000","last_undegraded":"2024-05-11T13:33:52.514146+0000","last_fullsized":"2024-05-11T13:33:52.514146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:37:20.727773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086324+0000","last_change":"2024-05-11T13:28:30.709014+0000","last_active":"2024-05-11T13:33:52.086324+0000","last_peered":"2024-05-11T13:33:52.086324+0000","last_clean":"2024-05-11T13:33:52.086324+0000","last_became_active":"2024-05-11T13:28:30.708000+0000","last_became_peered":"2024-05-11T13:28:30.708000+0000","last_unstale":"2024-05-11T13:33:52.086324+0000","last_undegraded":"2024-05-11T13:33:52.086324+0000","last_fullsized":"2024-05-11T13:33:52.086324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:00.813415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100555+0000","last_change":"2024-05-11T13:28:35.215149+0000","last_active":"2024-05-11T13:33:52.100555+0000","last_peered":"2024-05-11T13:33:52.100555+0000","last_clean":"2024-05-11T13:33:52.100555+0000","last_became_active":"2024-05-11T13:28:35.214792+0000","last_became_peered":"2024-05-11T13:28:35.214792+0000","last_unstale":"2024-05-11T13:33:52.100555+0000","last_undegraded":"2024-05-11T13:33:52.100555+0000","last_fullsized":"2024-05-11T13:33:52.100555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100519+0000","last_change":"2024-05-11T13:28:49.686347+0000","last_active":"2024-05-11T13:33:52.100519+0000","last_peered":"2024-05-11T13:33:52.100519+0000","last_clean":"2024-05-11T13:33:52.100519+0000","last_became_active":"2024-05-11T13:28:49.685391+0000","last_became_peered":"2024-05-11T13:28:49.685391+0000","last_unstale":"2024-05-11T13:33:52.100519+0000","last_undegraded":"2024-05-11T13:33:52.100519+0000","last_fullsized":"2024-05-11T13:33:52.100519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:24:50.347627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585329+0000","last_change":"2024-05-11T13:29:40.560815+0000","last_active":"2024-05-11T13:33:52.585329+0000","last_peered":"2024-05-11T13:33:52.585329+0000","last_clean":"2024-05-11T13:33:52.585329+0000","last_became_active":"2024-05-11T13:29:40.553031+0000","last_became_peered":"2024-05-11T13:29:40.553031+0000","last_unstale":"2024-05-11T13:33:52.585329+0000","last_undegraded":"2024-05-11T13:33:52.585329+0000","last_fullsized":"2024-05-11T13:33:52.585329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:27.437654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585383+0000","last_change":"2024-05-11T13:28:35.218377+0000","last_active":"2024-05-11T13:33:52.585383+0000","last_peered":"2024-05-11T13:33:52.585383+0000","last_clean":"2024-05-11T13:33:52.585383+0000","last_became_active":"2024-05-11T13:28:35.217922+0000","last_became_peered":"2024-05-11T13:28:35.217922+0000","last_unstale":"2024-05-11T13:33:52.585383+0000","last_undegraded":"2024-05-11T13:33:52.585383+0000","last_fullsized":"2024-05-11T13:33:52.585383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:58.383381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086394+0000","last_change":"2024-05-11T13:28:49.723400+0000","last_active":"2024-05-11T13:33:52.086394+0000","last_peered":"2024-05-11T13:33:52.086394+0000","last_clean":"2024-05-11T13:33:52.086394+0000","last_became_active":"2024-05-11T13:28:49.722675+0000","last_became_peered":"2024-05-11T13:28:49.722675+0000","last_unstale":"2024-05-11T13:33:52.086394+0000","last_undegraded":"2024-05-11T13:33:52.086394+0000","last_fullsized":"2024-05-11T13:33:52.086394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:14:44.810541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.949120+0000","last_change":"2024-05-11T13:28:30.710667+0000","last_active":"2024-05-11T13:33:51.949120+0000","last_peered":"2024-05-11T13:33:51.949120+0000","last_clean":"2024-05-11T13:33:51.949120+0000","last_became_active":"2024-05-11T13:28:30.709933+0000","last_became_peered":"2024-05-11T13:28:30.709933+0000","last_unstale":"2024-05-11T13:33:51.949120+0000","last_undegraded":"2024-05-11T13:33:51.949120+0000","last_fullsized":"2024-05-11T13:33:51.949120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:34.942946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109997+0000","last_change":"2024-05-11T13:28:35.215691+0000","last_active":"2024-05-11T13:33:52.109997+0000","last_peered":"2024-05-11T13:33:52.109997+0000","last_clean":"2024-05-11T13:33:52.109997+0000","last_became_active":"2024-05-11T13:28:35.209873+0000","last_became_peered":"2024-05-11T13:28:35.209873+0000","last_unstale":"2024-05-11T13:33:52.109997+0000","last_undegraded":"2024-05-11T13:33:52.109997+0000","last_fullsized":"2024-05-11T13:33:52.109997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:42:12.935570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513453+0000","last_change":"2024-05-11T13:28:49.694322+0000","last_active":"2024-05-11T13:33:52.513453+0000","last_peered":"2024-05-11T13:33:52.513453+0000","last_clean":"2024-05-11T13:33:52.513453+0000","last_became_active":"2024-05-11T13:28:49.690900+0000","last_became_peered":"2024-05-11T13:28:49.690900+0000","last_unstale":"2024-05-11T13:33:52.513453+0000","last_undegraded":"2024-05-11T13:33:52.513453+0000","last_fullsized":"2024-05-11T13:33:52.513453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:00:07.433371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'9","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:56.464371+0000","last_change":"2024-05-11T13:28:30.432529+0000","last_active":"2024-05-11T13:33:56.464371+0000","last_peered":"2024-05-11T13:33:56.464371+0000","last_clean":"2024-05-11T13:33:56.464371+0000","last_became_active":"2024-05-11T13:28:30.428077+0000","last_became_peered":"2024-05-11T13:28:30.428077+0000","last_unstale":"2024-05-11T13:33:56.464371+0000","last_undegraded":"2024-05-11T13:33:56.464371+0000","last_fullsized":"2024-05-11T13:33:56.464371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:42.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":244,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109674+0000","last_change":"2024-05-11T13:28:35.219160+0000","last_active":"2024-05-11T13:33:52.109674+0000","last_peered":"2024-05-11T13:33:52.109674+0000","last_clean":"2024-05-11T13:33:52.109674+0000","last_became_active":"2024-05-11T13:28:35.216810+0000","last_became_peered":"2024-05-11T13:28:35.216810+0000","last_unstale":"2024-05-11T13:33:52.109674+0000","last_undegraded":"2024-05-11T13:33:52.109674+0000","last_fullsized":"2024-05-11T13:33:52.109674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:35:54.465030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514299+0000","last_change":"2024-05-11T13:28:49.725190+0000","last_active":"2024-05-11T13:33:52.514299+0000","last_peered":"2024-05-11T13:33:52.514299+0000","last_clean":"2024-05-11T13:33:52.514299+0000","last_became_active":"2024-05-11T13:28:49.724193+0000","last_became_peered":"2024-05-11T13:28:49.724193+0000","last_unstale":"2024-05-11T13:33:52.514299+0000","last_undegraded":"2024-05-11T13:33:52.514299+0000","last_fullsized":"2024-05-11T13:33:52.514299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:47:41.181606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101663+0000","last_change":"2024-05-11T13:28:35.211262+0000","last_active":"2024-05-11T13:33:52.101663+0000","last_peered":"2024-05-11T13:33:52.101663+0000","last_clean":"2024-05-11T13:33:52.101663+0000","last_became_active":"2024-05-11T13:28:35.210859+0000","last_became_peered":"2024-05-11T13:28:35.210859+0000","last_unstale":"2024-05-11T13:33:52.101663+0000","last_undegraded":"2024-05-11T13:33:52.101663+0000","last_fullsized":"2024-05-11T13:33:52.101663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:40:33.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":295,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:56.526399+0000","last_change":"2024-05-11T13:28:30.707326+0000","last_active":"2024-05-11T13:33:56.526399+0000","last_peered":"2024-05-11T13:33:56.526399+0000","last_clean":"2024-05-11T13:33:56.526399+0000","last_became_active":"2024-05-11T13:28:30.705496+0000","last_became_peered":"2024-05-11T13:28:30.705496+0000","last_unstale":"2024-05-11T13:33:56.526399+0000","last_undegraded":"2024-05-11T13:33:56.526399+0000","last_fullsized":"2024-05-11T13:33:56.526399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:48.117551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132186+0000","last_change":"2024-05-11T13:28:49.687260+0000","last_active":"2024-05-11T13:33:52.132186+0000","last_peered":"2024-05-11T13:33:52.132186+0000","last_clean":"2024-05-11T13:33:52.132186+0000","last_became_active":"2024-05-11T13:28:49.686593+0000","last_became_peered":"2024-05-11T13:28:49.686593+0000","last_unstale":"2024-05-11T13:33:52.132186+0000","last_undegraded":"2024-05-11T13:33:52.132186+0000","last_fullsized":"2024-05-11T13:33:52.132186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:48.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084761+0000","last_change":"2024-05-11T13:28:35.216643+0000","last_active":"2024-05-11T13:33:52.084761+0000","last_peered":"2024-05-11T13:33:52.084761+0000","last_clean":"2024-05-11T13:33:52.084761+0000","last_became_active":"2024-05-11T13:28:35.216300+0000","last_became_peered":"2024-05-11T13:28:35.216300+0000","last_unstale":"2024-05-11T13:33:52.084761+0000","last_undegraded":"2024-05-11T13:33:52.084761+0000","last_fullsized":"2024-05-11T13:33:52.084761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:20:01.351678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517969+0000","last_change":"2024-05-11T13:28:30.427465+0000","last_active":"2024-05-11T13:33:52.517969+0000","last_peered":"2024-05-11T13:33:52.517969+0000","last_clean":"2024-05-11T13:33:52.517969+0000","last_became_active":"2024-05-11T13:28:30.426968+0000","last_became_peered":"2024-05-11T13:28:30.426968+0000","last_unstale":"2024-05-11T13:33:52.517969+0000","last_undegraded":"2024-05-11T13:33:52.517969+0000","last_fullsized":"2024-05-11T13:33:52.517969+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:02:44.969069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'99","reported_seq":149,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:54.110709+0000","last_change":"2024-05-11T13:26:37.738725+0000","last_active":"2024-05-11T13:33:54.110709+0000","last_peered":"2024-05-11T13:33:54.110709+0000","last_clean":"2024-05-11T13:33:54.110709+0000","last_became_active":"2024-05-11T13:26:37.724302+0000","last_became_peered":"2024-05-11T13:26:37.724302+0000","last_unstale":"2024-05-11T13:33:54.110709+0000","last_undegraded":"2024-05-11T13:33:54.110709+0000","last_fullsized":"2024-05-11T13:33:54.110709+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_clean_scrub_stamp":"2024-05-11T13:21:55.918148+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:32:56.277780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101952+0000","last_change":"2024-05-11T13:28:49.723829+0000","last_active":"2024-05-11T13:33:52.101952+0000","last_peered":"2024-05-11T13:33:52.101952+0000","last_clean":"2024-05-11T13:33:52.101952+0000","last_became_active":"2024-05-11T13:28:49.723190+0000","last_became_peered":"2024-05-11T13:28:49.723190+0000","last_unstale":"2024-05-11T13:33:52.101952+0000","last_undegraded":"2024-05-11T13:33:52.101952+0000","last_fullsized":"2024-05-11T13:33:52.101952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.742374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512112+0000","last_change":"2024-05-11T13:28:35.213486+0000","last_active":"2024-05-11T13:33:52.512112+0000","last_peered":"2024-05-11T13:33:52.512112+0000","last_clean":"2024-05-11T13:33:52.512112+0000","last_became_active":"2024-05-11T13:28:35.211962+0000","last_became_peered":"2024-05-11T13:28:35.211962+0000","last_unstale":"2024-05-11T13:33:52.512112+0000","last_undegraded":"2024-05-11T13:33:52.512112+0000","last_fullsized":"2024-05-11T13:33:52.512112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:28:34.046898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131990+0000","last_change":"2024-05-11T13:28:30.431023+0000","last_active":"2024-05-11T13:33:52.131990+0000","last_peered":"2024-05-11T13:33:52.131990+0000","last_clean":"2024-05-11T13:33:52.131990+0000","last_became_active":"2024-05-11T13:28:30.430609+0000","last_became_peered":"2024-05-11T13:28:30.430609+0000","last_unstale":"2024-05-11T13:33:52.131990+0000","last_undegraded":"2024-05-11T13:33:52.131990+0000","last_fullsized":"2024-05-11T13:33:52.131990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:50.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584949+0000","last_change":"2024-05-11T13:28:49.695550+0000","last_active":"2024-05-11T13:33:52.584949+0000","last_peered":"2024-05-11T13:33:52.584949+0000","last_clean":"2024-05-11T13:33:52.584949+0000","last_became_active":"2024-05-11T13:28:49.694350+0000","last_became_peered":"2024-05-11T13:28:49.694350+0000","last_unstale":"2024-05-11T13:33:52.584949+0000","last_undegraded":"2024-05-11T13:33:52.584949+0000","last_fullsized":"2024-05-11T13:33:52.584949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:49.617673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087887+0000","last_change":"2024-05-11T13:28:30.710826+0000","last_active":"2024-05-11T13:33:52.087887+0000","last_peered":"2024-05-11T13:33:52.087887+0000","last_clean":"2024-05-11T13:33:52.087887+0000","last_became_active":"2024-05-11T13:28:30.709854+0000","last_became_peered":"2024-05-11T13:28:30.709854+0000","last_unstale":"2024-05-11T13:33:52.087887+0000","last_undegraded":"2024-05-11T13:33:52.087887+0000","last_fullsized":"2024-05-11T13:33:52.087887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:35:16.384559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513698+0000","last_change":"2024-05-11T13:28:35.205521+0000","last_active":"2024-05-11T13:33:52.513698+0000","last_peered":"2024-05-11T13:33:52.513698+0000","last_clean":"2024-05-11T13:33:52.513698+0000","last_became_active":"2024-05-11T13:28:35.205167+0000","last_became_peered":"2024-05-11T13:28:35.205167+0000","last_unstale":"2024-05-11T13:33:52.513698+0000","last_undegraded":"2024-05-11T13:33:52.513698+0000","last_fullsized":"2024-05-11T13:33:52.513698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:10:27.473991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084255+0000","last_change":"2024-05-11T13:28:49.723652+0000","last_active":"2024-05-11T13:33:52.084255+0000","last_peered":"2024-05-11T13:33:52.084255+0000","last_clean":"2024-05-11T13:33:52.084255+0000","last_became_active":"2024-05-11T13:28:49.723135+0000","last_became_peered":"2024-05-11T13:28:49.723135+0000","last_unstale":"2024-05-11T13:33:52.084255+0000","last_undegraded":"2024-05-11T13:33:52.084255+0000","last_fullsized":"2024-05-11T13:33:52.084255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:21:18.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'105","reported_seq":244,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:54.790802+0000","last_change":"2024-05-11T13:28:30.428197+0000","last_active":"2024-05-11T13:33:54.790802+0000","last_peered":"2024-05-11T13:33:54.790802+0000","last_clean":"2024-05-11T13:33:54.790802+0000","last_became_active":"2024-05-11T13:28:30.427753+0000","last_became_peered":"2024-05-11T13:28:30.427753+0000","last_unstale":"2024-05-11T13:33:54.790802+0000","last_undegraded":"2024-05-11T13:33:54.790802+0000","last_fullsized":"2024-05-11T13:33:54.790802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:52:54.165993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.949665+0000","last_change":"2024-05-11T13:29:40.541861+0000","last_active":"2024-05-11T13:33:51.949665+0000","last_peered":"2024-05-11T13:33:51.949665+0000","last_clean":"2024-05-11T13:33:51.949665+0000","last_became_active":"2024-05-11T13:29:40.541395+0000","last_became_peered":"2024-05-11T13:29:40.541395+0000","last_unstale":"2024-05-11T13:33:51.949665+0000","last_undegraded":"2024-05-11T13:33:51.949665+0000","last_fullsized":"2024-05-11T13:33:51.949665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:50:39.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.951951+0000","last_change":"2024-05-11T13:28:49.677549+0000","last_active":"2024-05-11T13:33:51.951951+0000","last_peered":"2024-05-11T13:33:51.951951+0000","last_clean":"2024-05-11T13:33:51.951951+0000","last_became_active":"2024-05-11T13:28:49.676625+0000","last_became_peered":"2024-05-11T13:28:49.676625+0000","last_unstale":"2024-05-11T13:33:51.951951+0000","last_undegraded":"2024-05-11T13:33:51.951951+0000","last_fullsized":"2024-05-11T13:33:51.951951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:44.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.518109+0000","last_change":"2024-05-11T13:28:30.440945+0000","last_active":"2024-05-11T13:33:52.518109+0000","last_peered":"2024-05-11T13:33:52.518109+0000","last_clean":"2024-05-11T13:33:52.518109+0000","last_became_active":"2024-05-11T13:28:30.440595+0000","last_became_peered":"2024-05-11T13:28:30.440595+0000","last_unstale":"2024-05-11T13:33:52.518109+0000","last_undegraded":"2024-05-11T13:33:52.518109+0000","last_fullsized":"2024-05-11T13:33:52.518109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:51:10.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586605+0000","last_change":"2024-05-11T13:28:35.214207+0000","last_active":"2024-05-11T13:33:52.586605+0000","last_peered":"2024-05-11T13:33:52.586605+0000","last_clean":"2024-05-11T13:33:52.586605+0000","last_became_active":"2024-05-11T13:28:35.213844+0000","last_became_peered":"2024-05-11T13:28:35.213844+0000","last_unstale":"2024-05-11T13:33:52.586605+0000","last_undegraded":"2024-05-11T13:33:52.586605+0000","last_fullsized":"2024-05-11T13:33:52.586605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:53.063329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517766+0000","last_change":"2024-05-11T13:28:49.725505+0000","last_active":"2024-05-11T13:33:52.517766+0000","last_peered":"2024-05-11T13:33:52.517766+0000","last_clean":"2024-05-11T13:33:52.517766+0000","last_became_active":"2024-05-11T13:28:49.723613+0000","last_became_peered":"2024-05-11T13:28:49.723613+0000","last_unstale":"2024-05-11T13:33:52.517766+0000","last_undegraded":"2024-05-11T13:33:52.517766+0000","last_fullsized":"2024-05-11T13:33:52.517766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:54:30.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952629+0000","last_change":"2024-05-11T13:28:30.707025+0000","last_active":"2024-05-11T13:33:51.952629+0000","last_peered":"2024-05-11T13:33:51.952629+0000","last_clean":"2024-05-11T13:33:51.952629+0000","last_became_active":"2024-05-11T13:28:30.705853+0000","last_became_peered":"2024-05-11T13:28:30.705853+0000","last_unstale":"2024-05-11T13:33:51.952629+0000","last_undegraded":"2024-05-11T13:33:51.952629+0000","last_fullsized":"2024-05-11T13:33:51.952629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:15:04.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086366+0000","last_change":"2024-05-11T13:28:35.214682+0000","last_active":"2024-05-11T13:33:52.086366+0000","last_peered":"2024-05-11T13:33:52.086366+0000","last_clean":"2024-05-11T13:33:52.086366+0000","last_became_active":"2024-05-11T13:28:35.211581+0000","last_became_peered":"2024-05-11T13:28:35.211581+0000","last_unstale":"2024-05-11T13:33:52.086366+0000","last_undegraded":"2024-05-11T13:33:52.086366+0000","last_fullsized":"2024-05-11T13:33:52.086366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:19.231675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.110551+0000","last_change":"2024-05-11T13:30:41.536065+0000","last_active":"2024-05-11T13:33:52.110551+0000","last_peered":"2024-05-11T13:33:52.110551+0000","last_clean":"2024-05-11T13:33:52.110551+0000","last_became_active":"2024-05-11T13:30:41.535466+0000","last_became_peered":"2024-05-11T13:30:41.535466+0000","last_unstale":"2024-05-11T13:33:52.110551+0000","last_undegraded":"2024-05-11T13:33:52.110551+0000","last_fullsized":"2024-05-11T13:33:52.110551+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:18.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084929+0000","last_change":"2024-05-11T13:28:30.419444+0000","last_active":"2024-05-11T13:33:52.084929+0000","last_peered":"2024-05-11T13:33:52.084929+0000","last_clean":"2024-05-11T13:33:52.084929+0000","last_became_active":"2024-05-11T13:28:30.419064+0000","last_became_peered":"2024-05-11T13:28:30.419064+0000","last_unstale":"2024-05-11T13:33:52.084929+0000","last_undegraded":"2024-05-11T13:33:52.084929+0000","last_fullsized":"2024-05-11T13:33:52.084929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:02.567403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584868+0000","last_change":"2024-05-11T13:28:35.209347+0000","last_active":"2024-05-11T13:33:52.584868+0000","last_peered":"2024-05-11T13:33:52.584868+0000","last_clean":"2024-05-11T13:33:52.584868+0000","last_became_active":"2024-05-11T13:28:35.208912+0000","last_became_peered":"2024-05-11T13:28:35.208912+0000","last_unstale":"2024-05-11T13:33:52.584868+0000","last_undegraded":"2024-05-11T13:33:52.584868+0000","last_fullsized":"2024-05-11T13:33:52.584868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:08:30.832941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517255+0000","last_change":"2024-05-11T13:28:49.677398+0000","last_active":"2024-05-11T13:33:52.517255+0000","last_peered":"2024-05-11T13:33:52.517255+0000","last_clean":"2024-05-11T13:33:52.517255+0000","last_became_active":"2024-05-11T13:28:49.676725+0000","last_became_peered":"2024-05-11T13:28:49.676725+0000","last_unstale":"2024-05-11T13:33:52.517255+0000","last_undegraded":"2024-05-11T13:33:52.517255+0000","last_fullsized":"2024-05-11T13:33:52.517255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:32.476049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952245+0000","last_change":"2024-05-11T13:28:30.428782+0000","last_active":"2024-05-11T13:33:51.952245+0000","last_peered":"2024-05-11T13:33:51.952245+0000","last_clean":"2024-05-11T13:33:51.952245+0000","last_became_active":"2024-05-11T13:28:30.427819+0000","last_became_peered":"2024-05-11T13:28:30.427819+0000","last_unstale":"2024-05-11T13:33:51.952245+0000","last_undegraded":"2024-05-11T13:33:51.952245+0000","last_fullsized":"2024-05-11T13:33:51.952245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:23:24.533765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086186+0000","last_change":"2024-05-11T13:28:35.210095+0000","last_active":"2024-05-11T13:33:52.086186+0000","last_peered":"2024-05-11T13:33:52.086186+0000","last_clean":"2024-05-11T13:33:52.086186+0000","last_became_active":"2024-05-11T13:28:35.208831+0000","last_became_peered":"2024-05-11T13:28:35.208831+0000","last_unstale":"2024-05-11T13:33:52.086186+0000","last_undegraded":"2024-05-11T13:33:52.086186+0000","last_fullsized":"2024-05-11T13:33:52.086186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:06.957821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517277+0000","last_change":"2024-05-11T13:28:49.724791+0000","last_active":"2024-05-11T13:33:52.517277+0000","last_peered":"2024-05-11T13:33:52.517277+0000","last_clean":"2024-05-11T13:33:52.517277+0000","last_became_active":"2024-05-11T13:28:49.723855+0000","last_became_peered":"2024-05-11T13:28:49.723855+0000","last_unstale":"2024-05-11T13:33:52.517277+0000","last_undegraded":"2024-05-11T13:33:52.517277+0000","last_fullsized":"2024-05-11T13:33:52.517277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:34.705331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087095+0000","last_change":"2024-05-11T13:28:30.431206+0000","last_active":"2024-05-11T13:33:52.087095+0000","last_peered":"2024-05-11T13:33:52.087095+0000","last_clean":"2024-05-11T13:33:52.087095+0000","last_became_active":"2024-05-11T13:28:30.430714+0000","last_became_peered":"2024-05-11T13:28:30.430714+0000","last_unstale":"2024-05-11T13:33:52.087095+0000","last_undegraded":"2024-05-11T13:33:52.087095+0000","last_fullsized":"2024-05-11T13:33:52.087095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:29:21.070046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131757+0000","last_change":"2024-05-11T13:28:35.222254+0000","last_active":"2024-05-11T13:33:52.131757+0000","last_peered":"2024-05-11T13:33:52.131757+0000","last_clean":"2024-05-11T13:33:52.131757+0000","last_became_active":"2024-05-11T13:28:35.219539+0000","last_became_peered":"2024-05-11T13:28:35.219539+0000","last_unstale":"2024-05-11T13:33:52.131757+0000","last_undegraded":"2024-05-11T13:33:52.131757+0000","last_fullsized":"2024-05-11T13:33:52.131757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:55.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952971+0000","last_change":"2024-05-11T13:28:49.692168+0000","last_active":"2024-05-11T13:33:51.952971+0000","last_peered":"2024-05-11T13:33:51.952971+0000","last_clean":"2024-05-11T13:33:51.952971+0000","last_became_active":"2024-05-11T13:28:49.691730+0000","last_became_peered":"2024-05-11T13:28:49.691730+0000","last_unstale":"2024-05-11T13:33:51.952971+0000","last_undegraded":"2024-05-11T13:33:51.952971+0000","last_fullsized":"2024-05-11T13:33:51.952971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:02.935554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":297,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:56.503545+0000","last_change":"2024-05-11T13:28:30.712115+0000","last_active":"2024-05-11T13:33:56.503545+0000","last_peered":"2024-05-11T13:33:56.503545+0000","last_clean":"2024-05-11T13:33:56.503545+0000","last_became_active":"2024-05-11T13:28:30.710613+0000","last_became_peered":"2024-05-11T13:28:30.710613+0000","last_unstale":"2024-05-11T13:33:56.503545+0000","last_undegraded":"2024-05-11T13:33:56.503545+0000","last_fullsized":"2024-05-11T13:33:56.503545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:00.521915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132863+0000","last_change":"2024-05-11T13:28:35.221340+0000","last_active":"2024-05-11T13:33:52.132863+0000","last_peered":"2024-05-11T13:33:52.132863+0000","last_clean":"2024-05-11T13:33:52.132863+0000","last_became_active":"2024-05-11T13:28:35.220702+0000","last_became_peered":"2024-05-11T13:28:35.220702+0000","last_unstale":"2024-05-11T13:33:52.132863+0000","last_undegraded":"2024-05-11T13:33:52.132863+0000","last_fullsized":"2024-05-11T13:33:52.132863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:59:37.313209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584747+0000","last_change":"2024-05-11T13:28:49.682917+0000","last_active":"2024-05-11T13:33:52.584747+0000","last_peered":"2024-05-11T13:33:52.584747+0000","last_clean":"2024-05-11T13:33:52.584747+0000","last_became_active":"2024-05-11T13:28:49.681905+0000","last_became_peered":"2024-05-11T13:28:49.681905+0000","last_unstale":"2024-05-11T13:33:52.584747+0000","last_undegraded":"2024-05-11T13:33:52.584747+0000","last_fullsized":"2024-05-11T13:33:52.584747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:09:26.191565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:54.274960+0000","last_change":"2024-05-11T13:28:30.437356+0000","last_active":"2024-05-11T13:33:54.274960+0000","last_peered":"2024-05-11T13:33:54.274960+0000","last_clean":"2024-05-11T13:33:54.274960+0000","last_became_active":"2024-05-11T13:28:30.437013+0000","last_became_peered":"2024-05-11T13:28:30.437013+0000","last_unstale":"2024-05-11T13:33:54.274960+0000","last_undegraded":"2024-05-11T13:33:54.274960+0000","last_fullsized":"2024-05-11T13:33:54.274960+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:58.756156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132631+0000","last_change":"2024-05-11T13:28:35.206105+0000","last_active":"2024-05-11T13:33:52.132631+0000","last_peered":"2024-05-11T13:33:52.132631+0000","last_clean":"2024-05-11T13:33:52.132631+0000","last_became_active":"2024-05-11T13:28:35.205684+0000","last_became_peered":"2024-05-11T13:28:35.205684+0000","last_unstale":"2024-05-11T13:33:52.132631+0000","last_undegraded":"2024-05-11T13:33:52.132631+0000","last_fullsized":"2024-05-11T13:33:52.132631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:12.326574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100161+0000","last_change":"2024-05-11T13:28:49.689545+0000","last_active":"2024-05-11T13:33:52.100161+0000","last_peered":"2024-05-11T13:33:52.100161+0000","last_clean":"2024-05-11T13:33:52.100161+0000","last_became_active":"2024-05-11T13:28:49.688528+0000","last_became_peered":"2024-05-11T13:28:49.688528+0000","last_unstale":"2024-05-11T13:33:52.100161+0000","last_undegraded":"2024-05-11T13:33:52.100161+0000","last_fullsized":"2024-05-11T13:33:52.100161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:35.418873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514120+0000","last_change":"2024-05-11T13:29:40.560835+0000","last_active":"2024-05-11T13:33:52.514120+0000","last_peered":"2024-05-11T13:33:52.514120+0000","last_clean":"2024-05-11T13:33:52.514120+0000","last_became_active":"2024-05-11T13:29:40.554310+0000","last_became_peered":"2024-05-11T13:29:40.554310+0000","last_unstale":"2024-05-11T13:33:52.514120+0000","last_undegraded":"2024-05-11T13:33:52.514120+0000","last_fullsized":"2024-05-11T13:33:52.514120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:19:26.805261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132512+0000","last_change":"2024-05-11T13:28:35.220001+0000","last_active":"2024-05-11T13:33:52.132512+0000","last_peered":"2024-05-11T13:33:52.132512+0000","last_clean":"2024-05-11T13:33:52.132512+0000","last_became_active":"2024-05-11T13:28:35.216768+0000","last_became_peered":"2024-05-11T13:28:35.216768+0000","last_unstale":"2024-05-11T13:33:52.132512+0000","last_undegraded":"2024-05-11T13:33:52.132512+0000","last_fullsized":"2024-05-11T13:33:52.132512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:54:23.547689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584998+0000","last_change":"2024-05-11T13:28:35.213481+0000","last_active":"2024-05-11T13:33:52.584998+0000","last_peered":"2024-05-11T13:33:52.584998+0000","last_clean":"2024-05-11T13:33:52.584998+0000","last_became_active":"2024-05-11T13:28:35.212956+0000","last_became_peered":"2024-05-11T13:28:35.212956+0000","last_unstale":"2024-05-11T13:33:52.584998+0000","last_undegraded":"2024-05-11T13:33:52.584998+0000","last_fullsized":"2024-05-11T13:33:52.584998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:14.814155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952702+0000","last_change":"2024-05-11T13:29:40.554654+0000","last_active":"2024-05-11T13:33:51.952702+0000","last_peered":"2024-05-11T13:33:51.952702+0000","last_clean":"2024-05-11T13:33:51.952702+0000","last_became_active":"2024-05-11T13:29:40.551836+0000","last_became_peered":"2024-05-11T13:29:40.551836+0000","last_unstale":"2024-05-11T13:33:51.952702+0000","last_undegraded":"2024-05-11T13:33:51.952702+0000","last_fullsized":"2024-05-11T13:33:51.952702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:49:44.566051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952138+0000","last_change":"2024-05-11T13:28:35.208925+0000","last_active":"2024-05-11T13:33:51.952138+0000","last_peered":"2024-05-11T13:33:51.952138+0000","last_clean":"2024-05-11T13:33:51.952138+0000","last_became_active":"2024-05-11T13:28:35.208055+0000","last_became_peered":"2024-05-11T13:28:35.208055+0000","last_unstale":"2024-05-11T13:33:51.952138+0000","last_undegraded":"2024-05-11T13:33:51.952138+0000","last_fullsized":"2024-05-11T13:33:51.952138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:20.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585665+0000","last_change":"2024-05-11T13:28:35.213226+0000","last_active":"2024-05-11T13:33:52.585665+0000","last_peered":"2024-05-11T13:33:52.585665+0000","last_clean":"2024-05-11T13:33:52.585665+0000","last_became_active":"2024-05-11T13:28:35.212839+0000","last_became_peered":"2024-05-11T13:28:35.212839+0000","last_unstale":"2024-05-11T13:33:52.585665+0000","last_undegraded":"2024-05-11T13:33:52.585665+0000","last_fullsized":"2024-05-11T13:33:52.585665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:37:05.312505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513858+0000","last_change":"2024-05-11T13:29:40.848092+0000","last_active":"2024-05-11T13:33:52.513858+0000","last_peered":"2024-05-11T13:33:52.513858+0000","last_clean":"2024-05-11T13:33:52.513858+0000","last_became_active":"2024-05-11T13:29:40.846950+0000","last_became_peered":"2024-05-11T13:29:40.846950+0000","last_unstale":"2024-05-11T13:33:52.513858+0000","last_undegraded":"2024-05-11T13:33:52.513858+0000","last_fullsized":"2024-05-11T13:33:52.513858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:18:06.675351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585503+0000","last_change":"2024-05-11T13:28:35.217529+0000","last_active":"2024-05-11T13:33:52.585503+0000","last_peered":"2024-05-11T13:33:52.585503+0000","last_clean":"2024-05-11T13:33:52.585503+0000","last_became_active":"2024-05-11T13:28:35.217062+0000","last_became_peered":"2024-05-11T13:28:35.217062+0000","last_unstale":"2024-05-11T13:33:52.585503+0000","last_undegraded":"2024-05-11T13:33:52.585503+0000","last_fullsized":"2024-05-11T13:33:52.585503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:24:33.955794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954490+0000","last_change":"2024-05-11T13:28:35.217777+0000","last_active":"2024-05-11T13:33:51.954490+0000","last_peered":"2024-05-11T13:33:51.954490+0000","last_clean":"2024-05-11T13:33:51.954490+0000","last_became_active":"2024-05-11T13:28:35.217381+0000","last_became_peered":"2024-05-11T13:28:35.217381+0000","last_unstale":"2024-05-11T13:33:51.954490+0000","last_undegraded":"2024-05-11T13:33:51.954490+0000","last_fullsized":"2024-05-11T13:33:51.954490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:43.973192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086598+0000","last_change":"2024-05-11T13:28:35.209826+0000","last_active":"2024-05-11T13:33:52.086598+0000","last_peered":"2024-05-11T13:33:52.086598+0000","last_clean":"2024-05-11T13:33:52.086598+0000","last_became_active":"2024-05-11T13:28:35.209461+0000","last_became_peered":"2024-05-11T13:28:35.209461+0000","last_unstale":"2024-05-11T13:33:52.086598+0000","last_undegraded":"2024-05-11T13:33:52.086598+0000","last_fullsized":"2024-05-11T13:33:52.086598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:03.323045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100553+0000","last_change":"2024-05-11T13:28:35.205252+0000","last_active":"2024-05-11T13:33:52.100553+0000","last_peered":"2024-05-11T13:33:52.100553+0000","last_clean":"2024-05-11T13:33:52.100553+0000","last_became_active":"2024-05-11T13:28:35.204835+0000","last_became_peered":"2024-05-11T13:28:35.204835+0000","last_unstale":"2024-05-11T13:33:52.100553+0000","last_undegraded":"2024-05-11T13:33:52.100553+0000","last_fullsized":"2024-05-11T13:33:52.100553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:10.032439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085049+0000","last_change":"2024-05-11T13:28:35.219718+0000","last_active":"2024-05-11T13:33:52.085049+0000","last_peered":"2024-05-11T13:33:52.085049+0000","last_clean":"2024-05-11T13:33:52.085049+0000","last_became_active":"2024-05-11T13:28:35.219032+0000","last_became_peered":"2024-05-11T13:28:35.219032+0000","last_unstale":"2024-05-11T13:33:52.085049+0000","last_undegraded":"2024-05-11T13:33:52.085049+0000","last_fullsized":"2024-05-11T13:33:52.085049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:03:58.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954180+0000","last_change":"2024-05-11T13:28:35.217820+0000","last_active":"2024-05-11T13:33:51.954180+0000","last_peered":"2024-05-11T13:33:51.954180+0000","last_clean":"2024-05-11T13:33:51.954180+0000","last_became_active":"2024-05-11T13:28:35.216260+0000","last_became_peered":"2024-05-11T13:28:35.216260+0000","last_unstale":"2024-05-11T13:33:51.954180+0000","last_undegraded":"2024-05-11T13:33:51.954180+0000","last_fullsized":"2024-05-11T13:33:51.954180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:20:47.188238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513331+0000","last_change":"2024-05-11T13:28:35.209762+0000","last_active":"2024-05-11T13:33:52.513331+0000","last_peered":"2024-05-11T13:33:52.513331+0000","last_clean":"2024-05-11T13:33:52.513331+0000","last_became_active":"2024-05-11T13:28:35.207404+0000","last_became_peered":"2024-05-11T13:28:35.207404+0000","last_unstale":"2024-05-11T13:33:52.513331+0000","last_undegraded":"2024-05-11T13:33:52.513331+0000","last_fullsized":"2024-05-11T13:33:52.513331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:41:20.808009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585291+0000","last_change":"2024-05-11T13:28:35.215137+0000","last_active":"2024-05-11T13:33:52.585291+0000","last_peered":"2024-05-11T13:33:52.585291+0000","last_clean":"2024-05-11T13:33:52.585291+0000","last_became_active":"2024-05-11T13:28:35.214745+0000","last_became_peered":"2024-05-11T13:28:35.214745+0000","last_unstale":"2024-05-11T13:33:52.585291+0000","last_undegraded":"2024-05-11T13:33:52.585291+0000","last_fullsized":"2024-05-11T13:33:52.585291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:44.381501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801276,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2790,"num_read_kb":1619,"num_write":518,"num_write_kb":2395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2403828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678098,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":530129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874329,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37496,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707656,"statfs":{"total":95995035648,"available":95956639744,"internally_reserved":0,"allocated":2088960,"data_stored":1513975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32620,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712532,"statfs":{"total":95995035648,"available":95961632768,"internally_reserved":0,"allocated":1478656,"data_stored":925899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234088,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1462272,"data_stored":921577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430319,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1129130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593846,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":525491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1130496,"data_stored":595000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738527,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":526415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1142784,"data_stored":594066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149836,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28888,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716264,"statfs":{"total":95995035648,"available":95965454336,"internally_reserved":0,"allocated":1794048,"data_stored":1245247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953605,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":531446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790077,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29104,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716048,"statfs":{"total":95995035648,"available":95965233152,"internally_reserved":0,"allocated":2015232,"data_stored":1449753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T13:34:02.978 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-11T13:34:02.978 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T13:34:03.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:03 smithi063 ceph-mon[29809]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-11T13:34:03.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:03 smithi063 ceph-mon[29809]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:34:03.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:03 smithi063 ceph-mon[29809]: mgrmap e24: x(active, since 10s), standbys: y, z 2024-05-11T13:34:03.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:03 smithi071 ceph-mon[33293]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-11T13:34:03.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:03 smithi071 ceph-mon[33293]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:34:03.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:03 smithi071 ceph-mon[33293]: mgrmap e24: x(active, since 10s), standbys: y, z 2024-05-11T13:34:03.499 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:03.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:03 smithi165 ceph-mon[33235]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-11T13:34:03.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:03 smithi165 ceph-mon[33235]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:34:03.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:03 smithi165 ceph-mon[33235]: mgrmap e24: x(active, since 10s), standbys: y, z 2024-05-11T13:34:05.183 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:05 smithi063 ceph-mon[29809]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 67 op/s 2024-05-11T13:34:05.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:05 smithi071 ceph-mon[33293]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 67 op/s 2024-05-11T13:34:05.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:05 smithi165 ceph-mon[33235]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 108 KiB/s wr, 67 op/s 2024-05-11T13:34:05.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 -- 172.21.15.63:0/823300392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880105770 msgr2=0x7fb880105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:05.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 --2- 172.21.15.63:0/823300392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880105770 0x7fb880105bf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009990 tx=0x7fb86c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:34:05.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 -- 172.21.15.63:0/823300392 shutdown_connections 2024-05-11T13:34:05.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 --2- 172.21.15.63:0/823300392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb880106130 0x7fb880112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:05.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 --2- 172.21.15.63:0/823300392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb88010b770 0x7fb88010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:05.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 --2- 172.21.15.63:0/823300392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880105770 0x7fb880105bf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:05.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 -- 172.21.15.63:0/823300392 >> 172.21.15.63:0/823300392 conn(0x7fb880101260 msgr2=0x7fb880103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:05.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 -- 172.21.15.63:0/823300392 shutdown_connections 2024-05-11T13:34:05.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.813+0000 7fb887200700 1 -- 172.21.15.63:0/823300392 wait complete. 2024-05-11T13:34:05.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.814+0000 7fb887200700 1 Processor -- start 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.814+0000 7fb887200700 1 -- start start 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb887200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb880105770 0x7fb88019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb887200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880106130 0x7fb88019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb887200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb88010b770 0x7fb8801a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb887200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801158b0 con 0x7fb880106130 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb887200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb880115730 con 0x7fb88010b770 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb887200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb880115a30 con 0x7fb880105770 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb87f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880106130 0x7fb88019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb87f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880106130 0x7fb88019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48220/0 (socket says 172.21.15.63:48220) 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb87f400700 1 -- 172.21.15.63:0/3540768059 learned_addr learned my addr 172.21.15.63:0/3540768059 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.815+0000 7fb87fe00700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb880105770 0x7fb88019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb885200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb88010b770 0x7fb8801a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb87f400700 1 -- 172.21.15.63:0/3540768059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb880105770 msgr2=0x7fb88019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb87f400700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb880105770 0x7fb88019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb87f400700 1 -- 172.21.15.63:0/3540768059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb88010b770 msgr2=0x7fb8801a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb87f400700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb88010b770 0x7fb8801a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb87f400700 1 -- 172.21.15.63:0/3540768059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8801a0db0 con 0x7fb880106130 2024-05-11T13:34:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb885200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb88010b770 0x7fb8801a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:34:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb87fe00700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb880105770 0x7fb88019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.816+0000 7fb87f400700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880106130 0x7fb88019c2e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b650 tx=0x7fb86c00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.817+0000 7fb87cc00700 1 -- 172.21.15.63:0/3540768059 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c027070 con 0x7fb880106130 2024-05-11T13:34:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.817+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8801a1040 con 0x7fb880106130 2024-05-11T13:34:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.817+0000 7fb87cc00700 1 -- 172.21.15.63:0/3540768059 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb86c022470 con 0x7fb880106130 2024-05-11T13:34:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.817+0000 7fb87cc00700 1 -- 172.21.15.63:0/3540768059 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c017650 con 0x7fb880106130 2024-05-11T13:34:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.817+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801a8920 con 0x7fb880106130 2024-05-11T13:34:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.818+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88010dd00 con 0x7fb880106130 2024-05-11T13:34:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.820+0000 7fb87cc00700 1 -- 172.21.15.63:0/3540768059 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7fb86c009d10 con 0x7fb880106130 2024-05-11T13:34:05.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.822+0000 7fb87cc00700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb8640b2b90 0x7fb8640b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:05.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.822+0000 7fb87cc00700 1 -- 172.21.15.63:0/3540768059 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb86c0e38f0 con 0x7fb880106130 2024-05-11T13:34:05.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.822+0000 7fb87fe00700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb8640b2b90 0x7fb8640b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.823+0000 7fb87fe00700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb8640b2b90 0x7fb8640b5050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb870009c90 tx=0x7fb870009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:05.827+0000 7fb87cc00700 1 -- 172.21.15.63:0/3540768059 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb86c013070 con 0x7fb880106130 2024-05-11T13:34:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.060+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb88019d100 con 0x7fb8640b2b90 2024-05-11T13:34:06.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.090+0000 7fb87cc00700 1 -- 172.21.15.63:0/3540768059 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339536 (secure 0 0 0) 0x7fb88019d100 con 0x7fb8640b2b90 2024-05-11T13:34:06.091 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:34:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb8640b2b90 msgr2=0x7fb8640b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb8640b2b90 0x7fb8640b5050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb870009c90 tx=0x7fb870009260 comp rx=0 tx=0).stop 2024-05-11T13:34:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880106130 msgr2=0x7fb88019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880106130 0x7fb88019c2e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b650 tx=0x7fb86c00bfc0 comp rx=0 tx=0).stop 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 shutdown_connections 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb8640b2b90 0x7fb8640b5050 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb88010b770 0x7fb8801a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb880105770 0x7fb88019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 --2- 172.21.15.63:0/3540768059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb880106130 0x7fb88019c2e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.098+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 >> 172.21.15.63:0/3540768059 conn(0x7fb880101260 msgr2=0x7fb880101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.099+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 shutdown_connections 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:06.099+0000 7fb887200700 1 -- 172.21.15.63:0/3540768059 wait complete. 2024-05-11T13:34:06.099 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-11T13:34:07.076 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-05-11T13:34:05.972948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1393707,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":3107,"num_read_kb":2005,"num_write":713,"num_write_kb":4813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1032,"ondisk_log_size":1032,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391152,"kb_used_data":16664,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124550672,"statfs":{"total":1151940427776,"available":1151539888128,"internally_reserved":0,"allocated":17063936,"data_stored":10493615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":198461,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":544,"num_read_kb":581,"num_write":198,"num_write_kb":1181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004532"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585661+0000","last_change":"2024-05-11T13:28:35.223417+0000","last_active":"2024-05-11T13:33:52.585661+0000","last_peered":"2024-05-11T13:33:52.585661+0000","last_clean":"2024-05-11T13:33:52.585661+0000","last_became_active":"2024-05-11T13:28:35.222862+0000","last_became_peered":"2024-05-11T13:28:35.222862+0000","last_unstale":"2024-05-11T13:33:52.585661+0000","last_undegraded":"2024-05-11T13:33:52.585661+0000","last_fullsized":"2024-05-11T13:33:52.585661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:20.259043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586553+0000","last_change":"2024-05-11T13:28:35.222289+0000","last_active":"2024-05-11T13:33:52.586553+0000","last_peered":"2024-05-11T13:33:52.586553+0000","last_clean":"2024-05-11T13:33:52.586553+0000","last_became_active":"2024-05-11T13:28:35.221366+0000","last_became_peered":"2024-05-11T13:28:35.221366+0000","last_unstale":"2024-05-11T13:33:52.586553+0000","last_undegraded":"2024-05-11T13:33:52.586553+0000","last_fullsized":"2024-05-11T13:33:52.586553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:02.648140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586045+0000","last_change":"2024-05-11T13:28:35.221271+0000","last_active":"2024-05-11T13:33:52.586045+0000","last_peered":"2024-05-11T13:33:52.586045+0000","last_clean":"2024-05-11T13:33:52.586045+0000","last_became_active":"2024-05-11T13:28:35.220294+0000","last_became_peered":"2024-05-11T13:28:35.220294+0000","last_unstale":"2024-05-11T13:33:52.586045+0000","last_undegraded":"2024-05-11T13:33:52.586045+0000","last_fullsized":"2024-05-11T13:33:52.586045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:04.608042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517653+0000","last_change":"2024-05-11T13:28:35.197643+0000","last_active":"2024-05-11T13:33:52.517653+0000","last_peered":"2024-05-11T13:33:52.517653+0000","last_clean":"2024-05-11T13:33:52.517653+0000","last_became_active":"2024-05-11T13:28:35.196528+0000","last_became_peered":"2024-05-11T13:28:35.196528+0000","last_unstale":"2024-05-11T13:33:52.517653+0000","last_undegraded":"2024-05-11T13:33:52.517653+0000","last_fullsized":"2024-05-11T13:33:52.517653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:02:03.088456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086886+0000","last_change":"2024-05-11T13:28:35.206833+0000","last_active":"2024-05-11T13:33:52.086886+0000","last_peered":"2024-05-11T13:33:52.086886+0000","last_clean":"2024-05-11T13:33:52.086886+0000","last_became_active":"2024-05-11T13:28:35.206287+0000","last_became_peered":"2024-05-11T13:28:35.206287+0000","last_unstale":"2024-05-11T13:33:52.086886+0000","last_undegraded":"2024-05-11T13:33:52.086886+0000","last_fullsized":"2024-05-11T13:33:52.086886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:25:20.576464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109497+0000","last_change":"2024-05-11T13:28:35.218950+0000","last_active":"2024-05-11T13:33:52.109497+0000","last_peered":"2024-05-11T13:33:52.109497+0000","last_clean":"2024-05-11T13:33:52.109497+0000","last_became_active":"2024-05-11T13:28:35.218597+0000","last_became_peered":"2024-05-11T13:28:35.218597+0000","last_unstale":"2024-05-11T13:33:52.109497+0000","last_undegraded":"2024-05-11T13:33:52.109497+0000","last_fullsized":"2024-05-11T13:33:52.109497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:08:42.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131537+0000","last_change":"2024-05-11T13:29:40.541106+0000","last_active":"2024-05-11T13:33:52.131537+0000","last_peered":"2024-05-11T13:33:52.131537+0000","last_clean":"2024-05-11T13:33:52.131537+0000","last_became_active":"2024-05-11T13:29:40.540007+0000","last_became_peered":"2024-05-11T13:29:40.540007+0000","last_unstale":"2024-05-11T13:33:52.131537+0000","last_undegraded":"2024-05-11T13:33:52.131537+0000","last_fullsized":"2024-05-11T13:33:52.131537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:00.596580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084385+0000","last_change":"2024-05-11T13:28:35.205483+0000","last_active":"2024-05-11T13:33:52.084385+0000","last_peered":"2024-05-11T13:33:52.084385+0000","last_clean":"2024-05-11T13:33:52.084385+0000","last_became_active":"2024-05-11T13:28:35.205101+0000","last_became_peered":"2024-05-11T13:28:35.205101+0000","last_unstale":"2024-05-11T13:33:52.084385+0000","last_undegraded":"2024-05-11T13:33:52.084385+0000","last_fullsized":"2024-05-11T13:33:52.084385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:37.917464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954688+0000","last_change":"2024-05-11T13:28:35.217372+0000","last_active":"2024-05-11T13:33:51.954688+0000","last_peered":"2024-05-11T13:33:51.954688+0000","last_clean":"2024-05-11T13:33:51.954688+0000","last_became_active":"2024-05-11T13:28:35.217021+0000","last_became_peered":"2024-05-11T13:28:35.217021+0000","last_unstale":"2024-05-11T13:33:51.954688+0000","last_undegraded":"2024-05-11T13:33:51.954688+0000","last_fullsized":"2024-05-11T13:33:51.954688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:24.273041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.110549+0000","last_change":"2024-05-11T13:29:40.548634+0000","last_active":"2024-05-11T13:33:52.110549+0000","last_peered":"2024-05-11T13:33:52.110549+0000","last_clean":"2024-05-11T13:33:52.110549+0000","last_became_active":"2024-05-11T13:29:40.547536+0000","last_became_peered":"2024-05-11T13:29:40.547536+0000","last_unstale":"2024-05-11T13:33:52.110549+0000","last_undegraded":"2024-05-11T13:33:52.110549+0000","last_fullsized":"2024-05-11T13:33:52.110549+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:15.362317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132495+0000","last_change":"2024-05-11T13:28:35.214091+0000","last_active":"2024-05-11T13:33:52.132495+0000","last_peered":"2024-05-11T13:33:52.132495+0000","last_clean":"2024-05-11T13:33:52.132495+0000","last_became_active":"2024-05-11T13:28:35.213668+0000","last_became_peered":"2024-05-11T13:28:35.213668+0000","last_unstale":"2024-05-11T13:33:52.132495+0000","last_undegraded":"2024-05-11T13:33:52.132495+0000","last_fullsized":"2024-05-11T13:33:52.132495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:19:55.768203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517502+0000","last_change":"2024-05-11T13:28:35.211874+0000","last_active":"2024-05-11T13:33:52.517502+0000","last_peered":"2024-05-11T13:33:52.517502+0000","last_clean":"2024-05-11T13:33:52.517502+0000","last_became_active":"2024-05-11T13:28:35.211503+0000","last_became_peered":"2024-05-11T13:28:35.211503+0000","last_unstale":"2024-05-11T13:33:52.517502+0000","last_undegraded":"2024-05-11T13:33:52.517502+0000","last_fullsized":"2024-05-11T13:33:52.517502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:46.884110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513453+0000","last_change":"2024-05-11T13:28:35.209308+0000","last_active":"2024-05-11T13:33:52.513453+0000","last_peered":"2024-05-11T13:33:52.513453+0000","last_clean":"2024-05-11T13:33:52.513453+0000","last_became_active":"2024-05-11T13:28:35.207927+0000","last_became_peered":"2024-05-11T13:28:35.207927+0000","last_unstale":"2024-05-11T13:33:52.513453+0000","last_undegraded":"2024-05-11T13:33:52.513453+0000","last_fullsized":"2024-05-11T13:33:52.513453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:04.541849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584746+0000","last_change":"2024-05-11T13:28:35.208524+0000","last_active":"2024-05-11T13:33:52.584746+0000","last_peered":"2024-05-11T13:33:52.584746+0000","last_clean":"2024-05-11T13:33:52.584746+0000","last_became_active":"2024-05-11T13:28:35.207875+0000","last_became_peered":"2024-05-11T13:28:35.207875+0000","last_unstale":"2024-05-11T13:33:52.584746+0000","last_undegraded":"2024-05-11T13:33:52.584746+0000","last_fullsized":"2024-05-11T13:33:52.584746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:23:38.896916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585399+0000","last_change":"2024-05-11T13:28:35.216418+0000","last_active":"2024-05-11T13:33:52.585399+0000","last_peered":"2024-05-11T13:33:52.585399+0000","last_clean":"2024-05-11T13:33:52.585399+0000","last_became_active":"2024-05-11T13:28:35.215984+0000","last_became_peered":"2024-05-11T13:28:35.215984+0000","last_unstale":"2024-05-11T13:33:52.585399+0000","last_undegraded":"2024-05-11T13:33:52.585399+0000","last_fullsized":"2024-05-11T13:33:52.585399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:15:13.650821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952413+0000","last_change":"2024-05-11T13:28:35.217701+0000","last_active":"2024-05-11T13:33:51.952413+0000","last_peered":"2024-05-11T13:33:51.952413+0000","last_clean":"2024-05-11T13:33:51.952413+0000","last_became_active":"2024-05-11T13:28:35.217050+0000","last_became_peered":"2024-05-11T13:28:35.217050+0000","last_unstale":"2024-05-11T13:33:51.952413+0000","last_undegraded":"2024-05-11T13:33:51.952413+0000","last_fullsized":"2024-05-11T13:33:51.952413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:29:38.274540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585521+0000","last_change":"2024-05-11T13:28:35.223794+0000","last_active":"2024-05-11T13:33:52.585521+0000","last_peered":"2024-05-11T13:33:52.585521+0000","last_clean":"2024-05-11T13:33:52.585521+0000","last_became_active":"2024-05-11T13:28:35.223388+0000","last_became_peered":"2024-05-11T13:28:35.223388+0000","last_unstale":"2024-05-11T13:33:52.585521+0000","last_undegraded":"2024-05-11T13:33:52.585521+0000","last_fullsized":"2024-05-11T13:33:52.585521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:56:13.272357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948856+0000","last_change":"2024-05-11T13:28:35.216452+0000","last_active":"2024-05-11T13:33:51.948856+0000","last_peered":"2024-05-11T13:33:51.948856+0000","last_clean":"2024-05-11T13:33:51.948856+0000","last_became_active":"2024-05-11T13:28:35.216075+0000","last_became_peered":"2024-05-11T13:28:35.216075+0000","last_unstale":"2024-05-11T13:33:51.948856+0000","last_undegraded":"2024-05-11T13:33:51.948856+0000","last_fullsized":"2024-05-11T13:33:51.948856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:16:23.012302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954348+0000","last_change":"2024-05-11T13:28:35.206528+0000","last_active":"2024-05-11T13:33:51.954348+0000","last_peered":"2024-05-11T13:33:51.954348+0000","last_clean":"2024-05-11T13:33:51.954348+0000","last_became_active":"2024-05-11T13:28:35.206102+0000","last_became_peered":"2024-05-11T13:28:35.206102+0000","last_unstale":"2024-05-11T13:33:51.954348+0000","last_undegraded":"2024-05-11T13:33:51.954348+0000","last_fullsized":"2024-05-11T13:33:51.954348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.685477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952356+0000","last_change":"2024-05-11T13:30:41.923155+0000","last_active":"2024-05-11T13:33:51.952356+0000","last_peered":"2024-05-11T13:33:51.952356+0000","last_clean":"2024-05-11T13:33:51.952356+0000","last_became_active":"2024-05-11T13:30:41.922484+0000","last_became_peered":"2024-05-11T13:30:41.922484+0000","last_unstale":"2024-05-11T13:33:51.952356+0000","last_undegraded":"2024-05-11T13:33:51.952356+0000","last_fullsized":"2024-05-11T13:33:51.952356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:52.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":307,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:01.475874+0000","last_change":"2024-05-11T13:28:30.428076+0000","last_active":"2024-05-11T13:34:01.475874+0000","last_peered":"2024-05-11T13:34:01.475874+0000","last_clean":"2024-05-11T13:34:01.475874+0000","last_became_active":"2024-05-11T13:28:30.427519+0000","last_became_peered":"2024-05-11T13:28:30.427519+0000","last_unstale":"2024-05-11T13:34:01.475874+0000","last_undegraded":"2024-05-11T13:34:01.475874+0000","last_fullsized":"2024-05-11T13:34:01.475874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:52.808051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":245,"num_read_kb":245,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085491+0000","last_change":"2024-05-11T13:28:35.207587+0000","last_active":"2024-05-11T13:33:52.085491+0000","last_peered":"2024-05-11T13:33:52.085491+0000","last_clean":"2024-05-11T13:33:52.085491+0000","last_became_active":"2024-05-11T13:28:35.206692+0000","last_became_peered":"2024-05-11T13:28:35.206692+0000","last_unstale":"2024-05-11T13:33:52.085491+0000","last_undegraded":"2024-05-11T13:33:52.085491+0000","last_fullsized":"2024-05-11T13:33:52.085491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:25.171342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.516989+0000","last_change":"2024-05-11T13:28:49.682649+0000","last_active":"2024-05-11T13:33:52.516989+0000","last_peered":"2024-05-11T13:33:52.516989+0000","last_clean":"2024-05-11T13:33:52.516989+0000","last_became_active":"2024-05-11T13:28:49.682192+0000","last_became_peered":"2024-05-11T13:28:49.682192+0000","last_unstale":"2024-05-11T13:33:52.516989+0000","last_undegraded":"2024-05-11T13:33:52.516989+0000","last_fullsized":"2024-05-11T13:33:52.516989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:01.728705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085711+0000","last_change":"2024-05-11T13:28:30.707050+0000","last_active":"2024-05-11T13:33:52.085711+0000","last_peered":"2024-05-11T13:33:52.085711+0000","last_clean":"2024-05-11T13:33:52.085711+0000","last_became_active":"2024-05-11T13:28:30.706279+0000","last_became_peered":"2024-05-11T13:28:30.706279+0000","last_unstale":"2024-05-11T13:33:52.085711+0000","last_undegraded":"2024-05-11T13:33:52.085711+0000","last_fullsized":"2024-05-11T13:33:52.085711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:14.135734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586854+0000","last_change":"2024-05-11T13:28:35.219622+0000","last_active":"2024-05-11T13:33:52.586854+0000","last_peered":"2024-05-11T13:33:52.586854+0000","last_clean":"2024-05-11T13:33:52.586854+0000","last_became_active":"2024-05-11T13:28:35.218687+0000","last_became_peered":"2024-05-11T13:28:35.218687+0000","last_unstale":"2024-05-11T13:33:52.586854+0000","last_undegraded":"2024-05-11T13:33:52.586854+0000","last_fullsized":"2024-05-11T13:33:52.586854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:53.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517808+0000","last_change":"2024-05-11T13:28:49.685770+0000","last_active":"2024-05-11T13:33:52.517808+0000","last_peered":"2024-05-11T13:33:52.517808+0000","last_clean":"2024-05-11T13:33:52.517808+0000","last_became_active":"2024-05-11T13:28:49.685091+0000","last_became_peered":"2024-05-11T13:28:49.685091+0000","last_unstale":"2024-05-11T13:33:52.517808+0000","last_undegraded":"2024-05-11T13:33:52.517808+0000","last_fullsized":"2024-05-11T13:33:52.517808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:42.458310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087683+0000","last_change":"2024-05-11T13:28:30.435304+0000","last_active":"2024-05-11T13:33:52.087683+0000","last_peered":"2024-05-11T13:33:52.087683+0000","last_clean":"2024-05-11T13:33:52.087683+0000","last_became_active":"2024-05-11T13:28:30.434648+0000","last_became_peered":"2024-05-11T13:28:30.434648+0000","last_unstale":"2024-05-11T13:33:52.087683+0000","last_undegraded":"2024-05-11T13:33:52.087683+0000","last_fullsized":"2024-05-11T13:33:52.087683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:21.553376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953241+0000","last_change":"2024-05-11T13:28:35.210877+0000","last_active":"2024-05-11T13:33:51.953241+0000","last_peered":"2024-05-11T13:33:51.953241+0000","last_clean":"2024-05-11T13:33:51.953241+0000","last_became_active":"2024-05-11T13:28:35.207069+0000","last_became_peered":"2024-05-11T13:28:35.207069+0000","last_unstale":"2024-05-11T13:33:51.953241+0000","last_undegraded":"2024-05-11T13:33:51.953241+0000","last_fullsized":"2024-05-11T13:33:51.953241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:59.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517166+0000","last_change":"2024-05-11T13:28:49.689070+0000","last_active":"2024-05-11T13:33:52.517166+0000","last_peered":"2024-05-11T13:33:52.517166+0000","last_clean":"2024-05-11T13:33:52.517166+0000","last_became_active":"2024-05-11T13:28:49.688502+0000","last_became_peered":"2024-05-11T13:28:49.688502+0000","last_unstale":"2024-05-11T13:33:52.517166+0000","last_undegraded":"2024-05-11T13:33:52.517166+0000","last_fullsized":"2024-05-11T13:33:52.517166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:21.516074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584935+0000","last_change":"2024-05-11T13:29:40.564494+0000","last_active":"2024-05-11T13:33:52.584935+0000","last_peered":"2024-05-11T13:33:52.584935+0000","last_clean":"2024-05-11T13:33:52.584935+0000","last_became_active":"2024-05-11T13:29:40.555386+0000","last_became_peered":"2024-05-11T13:29:40.555386+0000","last_unstale":"2024-05-11T13:33:52.584935+0000","last_undegraded":"2024-05-11T13:33:52.584935+0000","last_fullsized":"2024-05-11T13:33:52.584935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:32:50.641372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101710+0000","last_change":"2024-05-11T13:28:35.217950+0000","last_active":"2024-05-11T13:33:52.101710+0000","last_peered":"2024-05-11T13:33:52.101710+0000","last_clean":"2024-05-11T13:33:52.101710+0000","last_became_active":"2024-05-11T13:28:35.216698+0000","last_became_peered":"2024-05-11T13:28:35.216698+0000","last_unstale":"2024-05-11T13:33:52.101710+0000","last_undegraded":"2024-05-11T13:33:52.101710+0000","last_fullsized":"2024-05-11T13:33:52.101710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:31:01.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953153+0000","last_change":"2024-05-11T13:30:41.923049+0000","last_active":"2024-05-11T13:33:51.953153+0000","last_peered":"2024-05-11T13:33:51.953153+0000","last_clean":"2024-05-11T13:33:51.953153+0000","last_became_active":"2024-05-11T13:30:41.922585+0000","last_became_peered":"2024-05-11T13:30:41.922585+0000","last_unstale":"2024-05-11T13:33:51.953153+0000","last_undegraded":"2024-05-11T13:33:51.953153+0000","last_fullsized":"2024-05-11T13:33:51.953153+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:34:03.716885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":308,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:01.516252+0000","last_change":"2024-05-11T13:29:40.555238+0000","last_active":"2024-05-11T13:34:01.516252+0000","last_peered":"2024-05-11T13:34:01.516252+0000","last_clean":"2024-05-11T13:34:01.516252+0000","last_became_active":"2024-05-11T13:29:40.554661+0000","last_became_peered":"2024-05-11T13:29:40.554661+0000","last_unstale":"2024-05-11T13:34:01.516252+0000","last_undegraded":"2024-05-11T13:34:01.516252+0000","last_fullsized":"2024-05-11T13:34:01.516252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:22.983649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":248,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086649+0000","last_change":"2024-05-11T13:28:35.209163+0000","last_active":"2024-05-11T13:33:52.086649+0000","last_peered":"2024-05-11T13:33:52.086649+0000","last_clean":"2024-05-11T13:33:52.086649+0000","last_became_active":"2024-05-11T13:28:35.204014+0000","last_became_peered":"2024-05-11T13:28:35.204014+0000","last_unstale":"2024-05-11T13:33:52.086649+0000","last_undegraded":"2024-05-11T13:33:52.086649+0000","last_fullsized":"2024-05-11T13:33:52.086649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:17.570863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953731+0000","last_change":"2024-05-11T13:29:40.555520+0000","last_active":"2024-05-11T13:33:51.953731+0000","last_peered":"2024-05-11T13:33:51.953731+0000","last_clean":"2024-05-11T13:33:51.953731+0000","last_became_active":"2024-05-11T13:29:40.553952+0000","last_became_peered":"2024-05-11T13:29:40.553952+0000","last_unstale":"2024-05-11T13:33:51.953731+0000","last_undegraded":"2024-05-11T13:33:51.953731+0000","last_fullsized":"2024-05-11T13:33:51.953731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.035691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109404+0000","last_change":"2024-05-11T13:28:30.429829+0000","last_active":"2024-05-11T13:33:52.109404+0000","last_peered":"2024-05-11T13:33:52.109404+0000","last_clean":"2024-05-11T13:33:52.109404+0000","last_became_active":"2024-05-11T13:28:30.428818+0000","last_became_peered":"2024-05-11T13:28:30.428818+0000","last_unstale":"2024-05-11T13:33:52.109404+0000","last_undegraded":"2024-05-11T13:33:52.109404+0000","last_fullsized":"2024-05-11T13:33:52.109404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:59:25.653181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101449+0000","last_change":"2024-05-11T13:28:35.214153+0000","last_active":"2024-05-11T13:33:52.101449+0000","last_peered":"2024-05-11T13:33:52.101449+0000","last_clean":"2024-05-11T13:33:52.101449+0000","last_became_active":"2024-05-11T13:28:35.213800+0000","last_became_peered":"2024-05-11T13:28:35.213800+0000","last_unstale":"2024-05-11T13:33:52.101449+0000","last_undegraded":"2024-05-11T13:33:52.101449+0000","last_fullsized":"2024-05-11T13:33:52.101449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:12.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585398+0000","last_change":"2024-05-11T13:28:49.697320+0000","last_active":"2024-05-11T13:33:52.585398+0000","last_peered":"2024-05-11T13:33:52.585398+0000","last_clean":"2024-05-11T13:33:52.585398+0000","last_became_active":"2024-05-11T13:28:49.696751+0000","last_became_peered":"2024-05-11T13:28:49.696751+0000","last_unstale":"2024-05-11T13:33:52.585398+0000","last_undegraded":"2024-05-11T13:33:52.585398+0000","last_fullsized":"2024-05-11T13:33:52.585398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:24.617241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512264+0000","last_change":"2024-05-11T13:28:30.423943+0000","last_active":"2024-05-11T13:33:52.512264+0000","last_peered":"2024-05-11T13:33:52.512264+0000","last_clean":"2024-05-11T13:33:52.512264+0000","last_became_active":"2024-05-11T13:28:30.423454+0000","last_became_peered":"2024-05-11T13:28:30.423454+0000","last_unstale":"2024-05-11T13:33:52.512264+0000","last_undegraded":"2024-05-11T13:33:52.512264+0000","last_fullsized":"2024-05-11T13:33:52.512264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:23:10.967537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084295+0000","last_change":"2024-05-11T13:28:35.220479+0000","last_active":"2024-05-11T13:33:52.084295+0000","last_peered":"2024-05-11T13:33:52.084295+0000","last_clean":"2024-05-11T13:33:52.084295+0000","last_became_active":"2024-05-11T13:28:35.214541+0000","last_became_peered":"2024-05-11T13:28:35.214541+0000","last_unstale":"2024-05-11T13:33:52.084295+0000","last_undegraded":"2024-05-11T13:33:52.084295+0000","last_fullsized":"2024-05-11T13:33:52.084295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:07.746301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586432+0000","last_change":"2024-05-11T13:28:49.691867+0000","last_active":"2024-05-11T13:33:52.586432+0000","last_peered":"2024-05-11T13:33:52.586432+0000","last_clean":"2024-05-11T13:33:52.586432+0000","last_became_active":"2024-05-11T13:28:49.690045+0000","last_became_peered":"2024-05-11T13:28:49.690045+0000","last_unstale":"2024-05-11T13:33:52.586432+0000","last_undegraded":"2024-05-11T13:33:52.586432+0000","last_fullsized":"2024-05-11T13:33:52.586432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:16.220334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100542+0000","last_change":"2024-05-11T13:28:30.436354+0000","last_active":"2024-05-11T13:33:52.100542+0000","last_peered":"2024-05-11T13:33:52.100542+0000","last_clean":"2024-05-11T13:33:52.100542+0000","last_became_active":"2024-05-11T13:28:30.435983+0000","last_became_peered":"2024-05-11T13:28:30.435983+0000","last_unstale":"2024-05-11T13:33:52.100542+0000","last_undegraded":"2024-05-11T13:33:52.100542+0000","last_fullsized":"2024-05-11T13:33:52.100542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:42.442829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948001+0000","last_change":"2024-05-11T13:28:35.207284+0000","last_active":"2024-05-11T13:33:51.948001+0000","last_peered":"2024-05-11T13:33:51.948001+0000","last_clean":"2024-05-11T13:33:51.948001+0000","last_became_active":"2024-05-11T13:28:35.206945+0000","last_became_peered":"2024-05-11T13:28:35.206945+0000","last_unstale":"2024-05-11T13:33:51.948001+0000","last_undegraded":"2024-05-11T13:33:51.948001+0000","last_fullsized":"2024-05-11T13:33:51.948001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:18.493659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084824+0000","last_change":"2024-05-11T13:28:49.689558+0000","last_active":"2024-05-11T13:33:52.084824+0000","last_peered":"2024-05-11T13:33:52.084824+0000","last_clean":"2024-05-11T13:33:52.084824+0000","last_became_active":"2024-05-11T13:28:49.689156+0000","last_became_peered":"2024-05-11T13:28:49.689156+0000","last_unstale":"2024-05-11T13:33:52.084824+0000","last_undegraded":"2024-05-11T13:33:52.084824+0000","last_fullsized":"2024-05-11T13:33:52.084824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:21.530111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":542,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:02.516439+0000","last_change":"2024-05-11T13:28:30.434534+0000","last_active":"2024-05-11T13:34:02.516439+0000","last_peered":"2024-05-11T13:34:02.516439+0000","last_clean":"2024-05-11T13:34:02.516439+0000","last_became_active":"2024-05-11T13:28:30.433983+0000","last_became_peered":"2024-05-11T13:28:30.433983+0000","last_unstale":"2024-05-11T13:34:02.516439+0000","last_undegraded":"2024-05-11T13:34:02.516439+0000","last_fullsized":"2024-05-11T13:34:02.516439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:16.056662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":488,"num_read_kb":488,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953763+0000","last_change":"2024-05-11T13:28:35.205021+0000","last_active":"2024-05-11T13:33:51.953763+0000","last_peered":"2024-05-11T13:33:51.953763+0000","last_clean":"2024-05-11T13:33:51.953763+0000","last_became_active":"2024-05-11T13:28:35.204243+0000","last_became_peered":"2024-05-11T13:28:35.204243+0000","last_unstale":"2024-05-11T13:33:51.953763+0000","last_undegraded":"2024-05-11T13:33:51.953763+0000","last_fullsized":"2024-05-11T13:33:51.953763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:12.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.947982+0000","last_change":"2024-05-11T13:28:49.677392+0000","last_active":"2024-05-11T13:33:51.947982+0000","last_peered":"2024-05-11T13:33:51.947982+0000","last_clean":"2024-05-11T13:33:51.947982+0000","last_became_active":"2024-05-11T13:28:49.674008+0000","last_became_peered":"2024-05-11T13:28:49.674008+0000","last_unstale":"2024-05-11T13:33:51.947982+0000","last_undegraded":"2024-05-11T13:33:51.947982+0000","last_fullsized":"2024-05-11T13:33:51.947982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:47:14.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132482+0000","last_change":"2024-05-11T13:28:30.429500+0000","last_active":"2024-05-11T13:33:52.132482+0000","last_peered":"2024-05-11T13:33:52.132482+0000","last_clean":"2024-05-11T13:33:52.132482+0000","last_became_active":"2024-05-11T13:28:30.429049+0000","last_became_peered":"2024-05-11T13:28:30.429049+0000","last_unstale":"2024-05-11T13:33:52.132482+0000","last_undegraded":"2024-05-11T13:33:52.132482+0000","last_fullsized":"2024-05-11T13:33:52.132482+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:00.533603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586699+0000","last_change":"2024-05-11T13:28:35.223764+0000","last_active":"2024-05-11T13:33:52.586699+0000","last_peered":"2024-05-11T13:33:52.586699+0000","last_clean":"2024-05-11T13:33:52.586699+0000","last_became_active":"2024-05-11T13:28:35.223016+0000","last_became_peered":"2024-05-11T13:28:35.223016+0000","last_unstale":"2024-05-11T13:33:52.586699+0000","last_undegraded":"2024-05-11T13:33:52.586699+0000","last_fullsized":"2024-05-11T13:33:52.586699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:42.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.518233+0000","last_change":"2024-05-11T13:28:49.683733+0000","last_active":"2024-05-11T13:33:52.518233+0000","last_peered":"2024-05-11T13:33:52.518233+0000","last_clean":"2024-05-11T13:33:52.518233+0000","last_became_active":"2024-05-11T13:28:49.681245+0000","last_became_peered":"2024-05-11T13:28:49.681245+0000","last_unstale":"2024-05-11T13:33:52.518233+0000","last_undegraded":"2024-05-11T13:33:52.518233+0000","last_fullsized":"2024-05-11T13:33:52.518233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:03.386943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101317+0000","last_change":"2024-05-11T13:28:30.707394+0000","last_active":"2024-05-11T13:33:52.101317+0000","last_peered":"2024-05-11T13:33:52.101317+0000","last_clean":"2024-05-11T13:33:52.101317+0000","last_became_active":"2024-05-11T13:28:30.706114+0000","last_became_peered":"2024-05-11T13:28:30.706114+0000","last_unstale":"2024-05-11T13:33:52.101317+0000","last_undegraded":"2024-05-11T13:33:52.101317+0000","last_fullsized":"2024-05-11T13:33:52.101317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:53:04.599800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954091+0000","last_change":"2024-05-11T13:28:35.219836+0000","last_active":"2024-05-11T13:33:51.954091+0000","last_peered":"2024-05-11T13:33:51.954091+0000","last_clean":"2024-05-11T13:33:51.954091+0000","last_became_active":"2024-05-11T13:28:35.214474+0000","last_became_peered":"2024-05-11T13:28:35.214474+0000","last_unstale":"2024-05-11T13:33:51.954091+0000","last_undegraded":"2024-05-11T13:33:51.954091+0000","last_fullsized":"2024-05-11T13:33:51.954091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:02.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.948036+0000","last_change":"2024-05-11T13:28:49.694677+0000","last_active":"2024-05-11T13:33:51.948036+0000","last_peered":"2024-05-11T13:33:51.948036+0000","last_clean":"2024-05-11T13:33:51.948036+0000","last_became_active":"2024-05-11T13:28:49.694176+0000","last_became_peered":"2024-05-11T13:28:49.694176+0000","last_unstale":"2024-05-11T13:33:51.948036+0000","last_undegraded":"2024-05-11T13:33:51.948036+0000","last_fullsized":"2024-05-11T13:33:51.948036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:35.461937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'45","reported_seq":595,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:02.519469+0000","last_change":"2024-05-11T13:28:30.433202+0000","last_active":"2024-05-11T13:34:02.519469+0000","last_peered":"2024-05-11T13:34:02.519469+0000","last_clean":"2024-05-11T13:34:02.519469+0000","last_became_active":"2024-05-11T13:28:30.429269+0000","last_became_peered":"2024-05-11T13:28:30.429269+0000","last_unstale":"2024-05-11T13:34:02.519469+0000","last_undegraded":"2024-05-11T13:34:02.519469+0000","last_fullsized":"2024-05-11T13:34:02.519469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:27:15.620717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69917,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":502,"num_read_kb":319,"num_write":47,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087908+0000","last_change":"2024-05-11T13:28:35.210882+0000","last_active":"2024-05-11T13:33:52.087908+0000","last_peered":"2024-05-11T13:33:52.087908+0000","last_clean":"2024-05-11T13:33:52.087908+0000","last_became_active":"2024-05-11T13:28:35.210506+0000","last_became_peered":"2024-05-11T13:28:35.210506+0000","last_unstale":"2024-05-11T13:33:52.087908+0000","last_undegraded":"2024-05-11T13:33:52.087908+0000","last_fullsized":"2024-05-11T13:33:52.087908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:45.752666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.102595+0000","last_change":"2024-05-11T13:28:49.693254+0000","last_active":"2024-05-11T13:33:52.102595+0000","last_peered":"2024-05-11T13:33:52.102595+0000","last_clean":"2024-05-11T13:33:52.102595+0000","last_became_active":"2024-05-11T13:28:49.690307+0000","last_became_peered":"2024-05-11T13:28:49.690307+0000","last_unstale":"2024-05-11T13:33:52.102595+0000","last_undegraded":"2024-05-11T13:33:52.102595+0000","last_fullsized":"2024-05-11T13:33:52.102595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:41:31.299920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.953151+0000","last_change":"2024-05-11T13:28:30.429421+0000","last_active":"2024-05-11T13:33:51.953151+0000","last_peered":"2024-05-11T13:33:51.953151+0000","last_clean":"2024-05-11T13:33:51.953151+0000","last_became_active":"2024-05-11T13:28:30.429032+0000","last_became_peered":"2024-05-11T13:28:30.429032+0000","last_unstale":"2024-05-11T13:33:51.953151+0000","last_undegraded":"2024-05-11T13:33:51.953151+0000","last_fullsized":"2024-05-11T13:33:51.953151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:06.234650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512553+0000","last_change":"2024-05-11T13:28:35.207709+0000","last_active":"2024-05-11T13:33:52.512553+0000","last_peered":"2024-05-11T13:33:52.512553+0000","last_clean":"2024-05-11T13:33:52.512553+0000","last_became_active":"2024-05-11T13:28:35.207222+0000","last_became_peered":"2024-05-11T13:28:35.207222+0000","last_unstale":"2024-05-11T13:33:52.512553+0000","last_undegraded":"2024-05-11T13:33:52.512553+0000","last_fullsized":"2024-05-11T13:33:52.512553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:20.023565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585040+0000","last_change":"2024-05-11T13:28:49.692046+0000","last_active":"2024-05-11T13:33:52.585040+0000","last_peered":"2024-05-11T13:33:52.585040+0000","last_clean":"2024-05-11T13:33:52.585040+0000","last_became_active":"2024-05-11T13:28:49.691618+0000","last_became_peered":"2024-05-11T13:28:49.691618+0000","last_unstale":"2024-05-11T13:33:52.585040+0000","last_undegraded":"2024-05-11T13:33:52.585040+0000","last_fullsized":"2024-05-11T13:33:52.585040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:05.805899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100851+0000","last_change":"2024-05-11T13:28:30.435459+0000","last_active":"2024-05-11T13:33:52.100851+0000","last_peered":"2024-05-11T13:33:52.100851+0000","last_clean":"2024-05-11T13:33:52.100851+0000","last_became_active":"2024-05-11T13:28:30.434997+0000","last_became_peered":"2024-05-11T13:28:30.434997+0000","last_unstale":"2024-05-11T13:33:52.100851+0000","last_undegraded":"2024-05-11T13:33:52.100851+0000","last_fullsized":"2024-05-11T13:33:52.100851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:36.280546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132009+0000","last_change":"2024-05-11T13:28:35.211214+0000","last_active":"2024-05-11T13:33:52.132009+0000","last_peered":"2024-05-11T13:33:52.132009+0000","last_clean":"2024-05-11T13:33:52.132009+0000","last_became_active":"2024-05-11T13:28:35.210777+0000","last_became_peered":"2024-05-11T13:28:35.210777+0000","last_unstale":"2024-05-11T13:33:52.132009+0000","last_undegraded":"2024-05-11T13:33:52.132009+0000","last_fullsized":"2024-05-11T13:33:52.132009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:53.238549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.950153+0000","last_change":"2024-05-11T13:28:49.682336+0000","last_active":"2024-05-11T13:33:51.950153+0000","last_peered":"2024-05-11T13:33:51.950153+0000","last_clean":"2024-05-11T13:33:51.950153+0000","last_became_active":"2024-05-11T13:28:49.679081+0000","last_became_peered":"2024-05-11T13:28:49.679081+0000","last_unstale":"2024-05-11T13:33:51.950153+0000","last_undegraded":"2024-05-11T13:33:51.950153+0000","last_fullsized":"2024-05-11T13:33:51.950153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:00:21.149262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'10","reported_seq":307,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:01.539716+0000","last_change":"2024-05-11T13:28:30.432165+0000","last_active":"2024-05-11T13:34:01.539716+0000","last_peered":"2024-05-11T13:34:01.539716+0000","last_clean":"2024-05-11T13:34:01.539716+0000","last_became_active":"2024-05-11T13:28:30.431181+0000","last_became_peered":"2024-05-11T13:28:30.431181+0000","last_unstale":"2024-05-11T13:34:01.539716+0000","last_undegraded":"2024-05-11T13:34:01.539716+0000","last_fullsized":"2024-05-11T13:34:01.539716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:26:50.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":7,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131539+0000","last_change":"2024-05-11T13:28:35.205662+0000","last_active":"2024-05-11T13:33:52.131539+0000","last_peered":"2024-05-11T13:33:52.131539+0000","last_clean":"2024-05-11T13:33:52.131539+0000","last_became_active":"2024-05-11T13:28:35.205276+0000","last_became_peered":"2024-05-11T13:28:35.205276+0000","last_unstale":"2024-05-11T13:33:52.131539+0000","last_undegraded":"2024-05-11T13:33:52.131539+0000","last_fullsized":"2024-05-11T13:33:52.131539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:36.308402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514146+0000","last_change":"2024-05-11T13:28:49.689705+0000","last_active":"2024-05-11T13:33:52.514146+0000","last_peered":"2024-05-11T13:33:52.514146+0000","last_clean":"2024-05-11T13:33:52.514146+0000","last_became_active":"2024-05-11T13:28:49.689223+0000","last_became_peered":"2024-05-11T13:28:49.689223+0000","last_unstale":"2024-05-11T13:33:52.514146+0000","last_undegraded":"2024-05-11T13:33:52.514146+0000","last_fullsized":"2024-05-11T13:33:52.514146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:37:20.727773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086324+0000","last_change":"2024-05-11T13:28:30.709014+0000","last_active":"2024-05-11T13:33:52.086324+0000","last_peered":"2024-05-11T13:33:52.086324+0000","last_clean":"2024-05-11T13:33:52.086324+0000","last_became_active":"2024-05-11T13:28:30.708000+0000","last_became_peered":"2024-05-11T13:28:30.708000+0000","last_unstale":"2024-05-11T13:33:52.086324+0000","last_undegraded":"2024-05-11T13:33:52.086324+0000","last_fullsized":"2024-05-11T13:33:52.086324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:00.813415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100555+0000","last_change":"2024-05-11T13:28:35.215149+0000","last_active":"2024-05-11T13:33:52.100555+0000","last_peered":"2024-05-11T13:33:52.100555+0000","last_clean":"2024-05-11T13:33:52.100555+0000","last_became_active":"2024-05-11T13:28:35.214792+0000","last_became_peered":"2024-05-11T13:28:35.214792+0000","last_unstale":"2024-05-11T13:33:52.100555+0000","last_undegraded":"2024-05-11T13:33:52.100555+0000","last_fullsized":"2024-05-11T13:33:52.100555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.242819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100519+0000","last_change":"2024-05-11T13:28:49.686347+0000","last_active":"2024-05-11T13:33:52.100519+0000","last_peered":"2024-05-11T13:33:52.100519+0000","last_clean":"2024-05-11T13:33:52.100519+0000","last_became_active":"2024-05-11T13:28:49.685391+0000","last_became_peered":"2024-05-11T13:28:49.685391+0000","last_unstale":"2024-05-11T13:33:52.100519+0000","last_undegraded":"2024-05-11T13:33:52.100519+0000","last_fullsized":"2024-05-11T13:33:52.100519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:24:50.347627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585329+0000","last_change":"2024-05-11T13:29:40.560815+0000","last_active":"2024-05-11T13:33:52.585329+0000","last_peered":"2024-05-11T13:33:52.585329+0000","last_clean":"2024-05-11T13:33:52.585329+0000","last_became_active":"2024-05-11T13:29:40.553031+0000","last_became_peered":"2024-05-11T13:29:40.553031+0000","last_unstale":"2024-05-11T13:33:52.585329+0000","last_undegraded":"2024-05-11T13:33:52.585329+0000","last_fullsized":"2024-05-11T13:33:52.585329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:27.437654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585383+0000","last_change":"2024-05-11T13:28:35.218377+0000","last_active":"2024-05-11T13:33:52.585383+0000","last_peered":"2024-05-11T13:33:52.585383+0000","last_clean":"2024-05-11T13:33:52.585383+0000","last_became_active":"2024-05-11T13:28:35.217922+0000","last_became_peered":"2024-05-11T13:28:35.217922+0000","last_unstale":"2024-05-11T13:33:52.585383+0000","last_undegraded":"2024-05-11T13:33:52.585383+0000","last_fullsized":"2024-05-11T13:33:52.585383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:14:58.383381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086394+0000","last_change":"2024-05-11T13:28:49.723400+0000","last_active":"2024-05-11T13:33:52.086394+0000","last_peered":"2024-05-11T13:33:52.086394+0000","last_clean":"2024-05-11T13:33:52.086394+0000","last_became_active":"2024-05-11T13:28:49.722675+0000","last_became_peered":"2024-05-11T13:28:49.722675+0000","last_unstale":"2024-05-11T13:33:52.086394+0000","last_undegraded":"2024-05-11T13:33:52.086394+0000","last_fullsized":"2024-05-11T13:33:52.086394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:14:44.810541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.949120+0000","last_change":"2024-05-11T13:28:30.710667+0000","last_active":"2024-05-11T13:33:51.949120+0000","last_peered":"2024-05-11T13:33:51.949120+0000","last_clean":"2024-05-11T13:33:51.949120+0000","last_became_active":"2024-05-11T13:28:30.709933+0000","last_became_peered":"2024-05-11T13:28:30.709933+0000","last_unstale":"2024-05-11T13:33:51.949120+0000","last_undegraded":"2024-05-11T13:33:51.949120+0000","last_fullsized":"2024-05-11T13:33:51.949120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:34.942946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109997+0000","last_change":"2024-05-11T13:28:35.215691+0000","last_active":"2024-05-11T13:33:52.109997+0000","last_peered":"2024-05-11T13:33:52.109997+0000","last_clean":"2024-05-11T13:33:52.109997+0000","last_became_active":"2024-05-11T13:28:35.209873+0000","last_became_peered":"2024-05-11T13:28:35.209873+0000","last_unstale":"2024-05-11T13:33:52.109997+0000","last_undegraded":"2024-05-11T13:33:52.109997+0000","last_fullsized":"2024-05-11T13:33:52.109997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:42:12.935570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513453+0000","last_change":"2024-05-11T13:28:49.694322+0000","last_active":"2024-05-11T13:33:52.513453+0000","last_peered":"2024-05-11T13:33:52.513453+0000","last_clean":"2024-05-11T13:33:52.513453+0000","last_became_active":"2024-05-11T13:28:49.690900+0000","last_became_peered":"2024-05-11T13:28:49.690900+0000","last_unstale":"2024-05-11T13:33:52.513453+0000","last_undegraded":"2024-05-11T13:33:52.513453+0000","last_fullsized":"2024-05-11T13:33:52.513453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:00:07.433371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'10","reported_seq":307,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:01.478766+0000","last_change":"2024-05-11T13:28:30.432529+0000","last_active":"2024-05-11T13:34:01.478766+0000","last_peered":"2024-05-11T13:34:01.478766+0000","last_clean":"2024-05-11T13:34:01.478766+0000","last_became_active":"2024-05-11T13:28:30.428077+0000","last_became_peered":"2024-05-11T13:28:30.428077+0000","last_unstale":"2024-05-11T13:34:01.478766+0000","last_undegraded":"2024-05-11T13:34:01.478766+0000","last_fullsized":"2024-05-11T13:34:01.478766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:42.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":250,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.109674+0000","last_change":"2024-05-11T13:28:35.219160+0000","last_active":"2024-05-11T13:33:52.109674+0000","last_peered":"2024-05-11T13:33:52.109674+0000","last_clean":"2024-05-11T13:33:52.109674+0000","last_became_active":"2024-05-11T13:28:35.216810+0000","last_became_peered":"2024-05-11T13:28:35.216810+0000","last_unstale":"2024-05-11T13:33:52.109674+0000","last_undegraded":"2024-05-11T13:33:52.109674+0000","last_fullsized":"2024-05-11T13:33:52.109674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:35:54.465030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514299+0000","last_change":"2024-05-11T13:28:49.725190+0000","last_active":"2024-05-11T13:33:52.514299+0000","last_peered":"2024-05-11T13:33:52.514299+0000","last_clean":"2024-05-11T13:33:52.514299+0000","last_became_active":"2024-05-11T13:28:49.724193+0000","last_became_peered":"2024-05-11T13:28:49.724193+0000","last_unstale":"2024-05-11T13:33:52.514299+0000","last_undegraded":"2024-05-11T13:33:52.514299+0000","last_fullsized":"2024-05-11T13:33:52.514299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:47:41.181606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101663+0000","last_change":"2024-05-11T13:28:35.211262+0000","last_active":"2024-05-11T13:33:52.101663+0000","last_peered":"2024-05-11T13:33:52.101663+0000","last_clean":"2024-05-11T13:33:52.101663+0000","last_became_active":"2024-05-11T13:28:35.210859+0000","last_became_peered":"2024-05-11T13:28:35.210859+0000","last_unstale":"2024-05-11T13:33:52.101663+0000","last_undegraded":"2024-05-11T13:33:52.101663+0000","last_fullsized":"2024-05-11T13:33:52.101663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:40:33.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:01.538262+0000","last_change":"2024-05-11T13:28:30.707326+0000","last_active":"2024-05-11T13:34:01.538262+0000","last_peered":"2024-05-11T13:34:01.538262+0000","last_clean":"2024-05-11T13:34:01.538262+0000","last_became_active":"2024-05-11T13:28:30.705496+0000","last_became_peered":"2024-05-11T13:28:30.705496+0000","last_unstale":"2024-05-11T13:34:01.538262+0000","last_undegraded":"2024-05-11T13:34:01.538262+0000","last_fullsized":"2024-05-11T13:34:01.538262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:48.117551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":245,"num_read_kb":245,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132186+0000","last_change":"2024-05-11T13:28:49.687260+0000","last_active":"2024-05-11T13:33:52.132186+0000","last_peered":"2024-05-11T13:33:52.132186+0000","last_clean":"2024-05-11T13:33:52.132186+0000","last_became_active":"2024-05-11T13:28:49.686593+0000","last_became_peered":"2024-05-11T13:28:49.686593+0000","last_unstale":"2024-05-11T13:33:52.132186+0000","last_undegraded":"2024-05-11T13:33:52.132186+0000","last_fullsized":"2024-05-11T13:33:52.132186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:48.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084761+0000","last_change":"2024-05-11T13:28:35.216643+0000","last_active":"2024-05-11T13:33:52.084761+0000","last_peered":"2024-05-11T13:33:52.084761+0000","last_clean":"2024-05-11T13:33:52.084761+0000","last_became_active":"2024-05-11T13:28:35.216300+0000","last_became_peered":"2024-05-11T13:28:35.216300+0000","last_unstale":"2024-05-11T13:33:52.084761+0000","last_undegraded":"2024-05-11T13:33:52.084761+0000","last_fullsized":"2024-05-11T13:33:52.084761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:20:01.351678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517969+0000","last_change":"2024-05-11T13:28:30.427465+0000","last_active":"2024-05-11T13:33:52.517969+0000","last_peered":"2024-05-11T13:33:52.517969+0000","last_clean":"2024-05-11T13:33:52.517969+0000","last_became_active":"2024-05-11T13:28:30.426968+0000","last_became_peered":"2024-05-11T13:28:30.426968+0000","last_unstale":"2024-05-11T13:33:52.517969+0000","last_undegraded":"2024-05-11T13:33:52.517969+0000","last_fullsized":"2024-05-11T13:33:52.517969+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:02:44.969069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'99","reported_seq":149,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:54.110709+0000","last_change":"2024-05-11T13:26:37.738725+0000","last_active":"2024-05-11T13:33:54.110709+0000","last_peered":"2024-05-11T13:33:54.110709+0000","last_clean":"2024-05-11T13:33:54.110709+0000","last_became_active":"2024-05-11T13:26:37.724302+0000","last_became_peered":"2024-05-11T13:26:37.724302+0000","last_unstale":"2024-05-11T13:33:54.110709+0000","last_undegraded":"2024-05-11T13:33:54.110709+0000","last_fullsized":"2024-05-11T13:33:54.110709+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:21:55.918148+0000","last_clean_scrub_stamp":"2024-05-11T13:21:55.918148+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:32:56.277780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.101952+0000","last_change":"2024-05-11T13:28:49.723829+0000","last_active":"2024-05-11T13:33:52.101952+0000","last_peered":"2024-05-11T13:33:52.101952+0000","last_clean":"2024-05-11T13:33:52.101952+0000","last_became_active":"2024-05-11T13:28:49.723190+0000","last_became_peered":"2024-05-11T13:28:49.723190+0000","last_unstale":"2024-05-11T13:33:52.101952+0000","last_undegraded":"2024-05-11T13:33:52.101952+0000","last_fullsized":"2024-05-11T13:33:52.101952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:33:34.742374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.512112+0000","last_change":"2024-05-11T13:28:35.213486+0000","last_active":"2024-05-11T13:33:52.512112+0000","last_peered":"2024-05-11T13:33:52.512112+0000","last_clean":"2024-05-11T13:33:52.512112+0000","last_became_active":"2024-05-11T13:28:35.211962+0000","last_became_peered":"2024-05-11T13:28:35.211962+0000","last_unstale":"2024-05-11T13:33:52.512112+0000","last_undegraded":"2024-05-11T13:33:52.512112+0000","last_fullsized":"2024-05-11T13:33:52.512112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:28:34.046898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131990+0000","last_change":"2024-05-11T13:28:30.431023+0000","last_active":"2024-05-11T13:33:52.131990+0000","last_peered":"2024-05-11T13:33:52.131990+0000","last_clean":"2024-05-11T13:33:52.131990+0000","last_became_active":"2024-05-11T13:28:30.430609+0000","last_became_peered":"2024-05-11T13:28:30.430609+0000","last_unstale":"2024-05-11T13:33:52.131990+0000","last_undegraded":"2024-05-11T13:33:52.131990+0000","last_fullsized":"2024-05-11T13:33:52.131990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:50.768167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584949+0000","last_change":"2024-05-11T13:28:49.695550+0000","last_active":"2024-05-11T13:33:52.584949+0000","last_peered":"2024-05-11T13:33:52.584949+0000","last_clean":"2024-05-11T13:33:52.584949+0000","last_became_active":"2024-05-11T13:28:49.694350+0000","last_became_peered":"2024-05-11T13:28:49.694350+0000","last_unstale":"2024-05-11T13:33:52.584949+0000","last_undegraded":"2024-05-11T13:33:52.584949+0000","last_fullsized":"2024-05-11T13:33:52.584949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:05:49.617673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087887+0000","last_change":"2024-05-11T13:28:30.710826+0000","last_active":"2024-05-11T13:33:52.087887+0000","last_peered":"2024-05-11T13:33:52.087887+0000","last_clean":"2024-05-11T13:33:52.087887+0000","last_became_active":"2024-05-11T13:28:30.709854+0000","last_became_peered":"2024-05-11T13:28:30.709854+0000","last_unstale":"2024-05-11T13:33:52.087887+0000","last_undegraded":"2024-05-11T13:33:52.087887+0000","last_fullsized":"2024-05-11T13:33:52.087887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:35:16.384559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513698+0000","last_change":"2024-05-11T13:28:35.205521+0000","last_active":"2024-05-11T13:33:52.513698+0000","last_peered":"2024-05-11T13:33:52.513698+0000","last_clean":"2024-05-11T13:33:52.513698+0000","last_became_active":"2024-05-11T13:28:35.205167+0000","last_became_peered":"2024-05-11T13:28:35.205167+0000","last_unstale":"2024-05-11T13:33:52.513698+0000","last_undegraded":"2024-05-11T13:33:52.513698+0000","last_fullsized":"2024-05-11T13:33:52.513698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:10:27.473991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084255+0000","last_change":"2024-05-11T13:28:49.723652+0000","last_active":"2024-05-11T13:33:52.084255+0000","last_peered":"2024-05-11T13:33:52.084255+0000","last_clean":"2024-05-11T13:33:52.084255+0000","last_became_active":"2024-05-11T13:28:49.723135+0000","last_became_peered":"2024-05-11T13:28:49.723135+0000","last_unstale":"2024-05-11T13:33:52.084255+0000","last_undegraded":"2024-05-11T13:33:52.084255+0000","last_fullsized":"2024-05-11T13:33:52.084255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:21:18.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'129","reported_seq":301,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:00.969429+0000","last_change":"2024-05-11T13:28:30.428197+0000","last_active":"2024-05-11T13:34:00.969429+0000","last_peered":"2024-05-11T13:34:00.969429+0000","last_clean":"2024-05-11T13:34:00.969429+0000","last_became_active":"2024-05-11T13:28:30.427753+0000","last_became_peered":"2024-05-11T13:28:30.427753+0000","last_unstale":"2024-05-11T13:34:00.969429+0000","last_undegraded":"2024-05-11T13:34:00.969429+0000","last_fullsized":"2024-05-11T13:34:00.969429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:52:54.165993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":237,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.949665+0000","last_change":"2024-05-11T13:29:40.541861+0000","last_active":"2024-05-11T13:33:51.949665+0000","last_peered":"2024-05-11T13:33:51.949665+0000","last_clean":"2024-05-11T13:33:51.949665+0000","last_became_active":"2024-05-11T13:29:40.541395+0000","last_became_peered":"2024-05-11T13:29:40.541395+0000","last_unstale":"2024-05-11T13:33:51.949665+0000","last_undegraded":"2024-05-11T13:33:51.949665+0000","last_fullsized":"2024-05-11T13:33:51.949665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:50:39.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.951951+0000","last_change":"2024-05-11T13:28:49.677549+0000","last_active":"2024-05-11T13:33:51.951951+0000","last_peered":"2024-05-11T13:33:51.951951+0000","last_clean":"2024-05-11T13:33:51.951951+0000","last_became_active":"2024-05-11T13:28:49.676625+0000","last_became_peered":"2024-05-11T13:28:49.676625+0000","last_unstale":"2024-05-11T13:33:51.951951+0000","last_undegraded":"2024-05-11T13:33:51.951951+0000","last_fullsized":"2024-05-11T13:33:51.951951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:44.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.518109+0000","last_change":"2024-05-11T13:28:30.440945+0000","last_active":"2024-05-11T13:33:52.518109+0000","last_peered":"2024-05-11T13:33:52.518109+0000","last_clean":"2024-05-11T13:33:52.518109+0000","last_became_active":"2024-05-11T13:28:30.440595+0000","last_became_peered":"2024-05-11T13:28:30.440595+0000","last_unstale":"2024-05-11T13:33:52.518109+0000","last_undegraded":"2024-05-11T13:33:52.518109+0000","last_fullsized":"2024-05-11T13:33:52.518109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:51:10.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.586605+0000","last_change":"2024-05-11T13:28:35.214207+0000","last_active":"2024-05-11T13:33:52.586605+0000","last_peered":"2024-05-11T13:33:52.586605+0000","last_clean":"2024-05-11T13:33:52.586605+0000","last_became_active":"2024-05-11T13:28:35.213844+0000","last_became_peered":"2024-05-11T13:28:35.213844+0000","last_unstale":"2024-05-11T13:33:52.586605+0000","last_undegraded":"2024-05-11T13:33:52.586605+0000","last_fullsized":"2024-05-11T13:33:52.586605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:53.063329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517766+0000","last_change":"2024-05-11T13:28:49.725505+0000","last_active":"2024-05-11T13:33:52.517766+0000","last_peered":"2024-05-11T13:33:52.517766+0000","last_clean":"2024-05-11T13:33:52.517766+0000","last_became_active":"2024-05-11T13:28:49.723613+0000","last_became_peered":"2024-05-11T13:28:49.723613+0000","last_unstale":"2024-05-11T13:33:52.517766+0000","last_undegraded":"2024-05-11T13:33:52.517766+0000","last_fullsized":"2024-05-11T13:33:52.517766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:54:30.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952629+0000","last_change":"2024-05-11T13:28:30.707025+0000","last_active":"2024-05-11T13:33:51.952629+0000","last_peered":"2024-05-11T13:33:51.952629+0000","last_clean":"2024-05-11T13:33:51.952629+0000","last_became_active":"2024-05-11T13:28:30.705853+0000","last_became_peered":"2024-05-11T13:28:30.705853+0000","last_unstale":"2024-05-11T13:33:51.952629+0000","last_undegraded":"2024-05-11T13:33:51.952629+0000","last_fullsized":"2024-05-11T13:33:51.952629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:15:04.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086366+0000","last_change":"2024-05-11T13:28:35.214682+0000","last_active":"2024-05-11T13:33:52.086366+0000","last_peered":"2024-05-11T13:33:52.086366+0000","last_clean":"2024-05-11T13:33:52.086366+0000","last_became_active":"2024-05-11T13:28:35.211581+0000","last_became_peered":"2024-05-11T13:28:35.211581+0000","last_unstale":"2024-05-11T13:33:52.086366+0000","last_undegraded":"2024-05-11T13:33:52.086366+0000","last_fullsized":"2024-05-11T13:33:52.086366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:19.231675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.110551+0000","last_change":"2024-05-11T13:30:41.536065+0000","last_active":"2024-05-11T13:33:52.110551+0000","last_peered":"2024-05-11T13:33:52.110551+0000","last_clean":"2024-05-11T13:33:52.110551+0000","last_became_active":"2024-05-11T13:30:41.535466+0000","last_became_peered":"2024-05-11T13:30:41.535466+0000","last_unstale":"2024-05-11T13:33:52.110551+0000","last_undegraded":"2024-05-11T13:33:52.110551+0000","last_fullsized":"2024-05-11T13:33:52.110551+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:18.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.084929+0000","last_change":"2024-05-11T13:28:30.419444+0000","last_active":"2024-05-11T13:33:52.084929+0000","last_peered":"2024-05-11T13:33:52.084929+0000","last_clean":"2024-05-11T13:33:52.084929+0000","last_became_active":"2024-05-11T13:28:30.419064+0000","last_became_peered":"2024-05-11T13:28:30.419064+0000","last_unstale":"2024-05-11T13:33:52.084929+0000","last_undegraded":"2024-05-11T13:33:52.084929+0000","last_fullsized":"2024-05-11T13:33:52.084929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:02.567403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584868+0000","last_change":"2024-05-11T13:28:35.209347+0000","last_active":"2024-05-11T13:33:52.584868+0000","last_peered":"2024-05-11T13:33:52.584868+0000","last_clean":"2024-05-11T13:33:52.584868+0000","last_became_active":"2024-05-11T13:28:35.208912+0000","last_became_peered":"2024-05-11T13:28:35.208912+0000","last_unstale":"2024-05-11T13:33:52.584868+0000","last_undegraded":"2024-05-11T13:33:52.584868+0000","last_fullsized":"2024-05-11T13:33:52.584868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:08:30.832941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517255+0000","last_change":"2024-05-11T13:28:49.677398+0000","last_active":"2024-05-11T13:33:52.517255+0000","last_peered":"2024-05-11T13:33:52.517255+0000","last_clean":"2024-05-11T13:33:52.517255+0000","last_became_active":"2024-05-11T13:28:49.676725+0000","last_became_peered":"2024-05-11T13:28:49.676725+0000","last_unstale":"2024-05-11T13:33:52.517255+0000","last_undegraded":"2024-05-11T13:33:52.517255+0000","last_fullsized":"2024-05-11T13:33:52.517255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:32.476049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952245+0000","last_change":"2024-05-11T13:28:30.428782+0000","last_active":"2024-05-11T13:33:51.952245+0000","last_peered":"2024-05-11T13:33:51.952245+0000","last_clean":"2024-05-11T13:33:51.952245+0000","last_became_active":"2024-05-11T13:28:30.427819+0000","last_became_peered":"2024-05-11T13:28:30.427819+0000","last_unstale":"2024-05-11T13:33:51.952245+0000","last_undegraded":"2024-05-11T13:33:51.952245+0000","last_fullsized":"2024-05-11T13:33:51.952245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:23:24.533765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086186+0000","last_change":"2024-05-11T13:28:35.210095+0000","last_active":"2024-05-11T13:33:52.086186+0000","last_peered":"2024-05-11T13:33:52.086186+0000","last_clean":"2024-05-11T13:33:52.086186+0000","last_became_active":"2024-05-11T13:28:35.208831+0000","last_became_peered":"2024-05-11T13:28:35.208831+0000","last_unstale":"2024-05-11T13:33:52.086186+0000","last_undegraded":"2024-05-11T13:33:52.086186+0000","last_fullsized":"2024-05-11T13:33:52.086186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:06.957821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.517277+0000","last_change":"2024-05-11T13:28:49.724791+0000","last_active":"2024-05-11T13:33:52.517277+0000","last_peered":"2024-05-11T13:33:52.517277+0000","last_clean":"2024-05-11T13:33:52.517277+0000","last_became_active":"2024-05-11T13:28:49.723855+0000","last_became_peered":"2024-05-11T13:28:49.723855+0000","last_unstale":"2024-05-11T13:33:52.517277+0000","last_undegraded":"2024-05-11T13:33:52.517277+0000","last_fullsized":"2024-05-11T13:33:52.517277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:19:34.705331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.087095+0000","last_change":"2024-05-11T13:28:30.431206+0000","last_active":"2024-05-11T13:33:52.087095+0000","last_peered":"2024-05-11T13:33:52.087095+0000","last_clean":"2024-05-11T13:33:52.087095+0000","last_became_active":"2024-05-11T13:28:30.430714+0000","last_became_peered":"2024-05-11T13:28:30.430714+0000","last_unstale":"2024-05-11T13:33:52.087095+0000","last_undegraded":"2024-05-11T13:33:52.087095+0000","last_fullsized":"2024-05-11T13:33:52.087095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:29:21.070046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.131757+0000","last_change":"2024-05-11T13:28:35.222254+0000","last_active":"2024-05-11T13:33:52.131757+0000","last_peered":"2024-05-11T13:33:52.131757+0000","last_clean":"2024-05-11T13:33:52.131757+0000","last_became_active":"2024-05-11T13:28:35.219539+0000","last_became_peered":"2024-05-11T13:28:35.219539+0000","last_unstale":"2024-05-11T13:33:52.131757+0000","last_undegraded":"2024-05-11T13:33:52.131757+0000","last_fullsized":"2024-05-11T13:33:52.131757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:55.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952971+0000","last_change":"2024-05-11T13:28:49.692168+0000","last_active":"2024-05-11T13:33:51.952971+0000","last_peered":"2024-05-11T13:33:51.952971+0000","last_clean":"2024-05-11T13:33:51.952971+0000","last_became_active":"2024-05-11T13:28:49.691730+0000","last_became_peered":"2024-05-11T13:28:49.691730+0000","last_unstale":"2024-05-11T13:33:51.952971+0000","last_undegraded":"2024-05-11T13:33:51.952971+0000","last_fullsized":"2024-05-11T13:33:51.952971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:36:02.935554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":302,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:01.514989+0000","last_change":"2024-05-11T13:28:30.712115+0000","last_active":"2024-05-11T13:34:01.514989+0000","last_peered":"2024-05-11T13:34:01.514989+0000","last_clean":"2024-05-11T13:34:01.514989+0000","last_became_active":"2024-05-11T13:28:30.710613+0000","last_became_peered":"2024-05-11T13:28:30.710613+0000","last_unstale":"2024-05-11T13:34:01.514989+0000","last_undegraded":"2024-05-11T13:34:01.514989+0000","last_fullsized":"2024-05-11T13:34:01.514989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:00.521915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132863+0000","last_change":"2024-05-11T13:28:35.221340+0000","last_active":"2024-05-11T13:33:52.132863+0000","last_peered":"2024-05-11T13:33:52.132863+0000","last_clean":"2024-05-11T13:33:52.132863+0000","last_became_active":"2024-05-11T13:28:35.220702+0000","last_became_peered":"2024-05-11T13:28:35.220702+0000","last_unstale":"2024-05-11T13:33:52.132863+0000","last_undegraded":"2024-05-11T13:33:52.132863+0000","last_fullsized":"2024-05-11T13:33:52.132863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:59:37.313209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584747+0000","last_change":"2024-05-11T13:28:49.682917+0000","last_active":"2024-05-11T13:33:52.584747+0000","last_peered":"2024-05-11T13:33:52.584747+0000","last_clean":"2024-05-11T13:33:52.584747+0000","last_became_active":"2024-05-11T13:28:49.681905+0000","last_became_peered":"2024-05-11T13:28:49.681905+0000","last_unstale":"2024-05-11T13:33:52.584747+0000","last_undegraded":"2024-05-11T13:33:52.584747+0000","last_fullsized":"2024-05-11T13:33:52.584747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:09:26.191565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'75","reported_seq":164,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:34:00.956414+0000","last_change":"2024-05-11T13:28:30.437356+0000","last_active":"2024-05-11T13:34:00.956414+0000","last_peered":"2024-05-11T13:34:00.956414+0000","last_clean":"2024-05-11T13:34:00.956414+0000","last_became_active":"2024-05-11T13:28:30.437013+0000","last_became_peered":"2024-05-11T13:28:30.437013+0000","last_unstale":"2024-05-11T13:34:00.956414+0000","last_undegraded":"2024-05-11T13:34:00.956414+0000","last_fullsized":"2024-05-11T13:34:00.956414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:58.756156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132631+0000","last_change":"2024-05-11T13:28:35.206105+0000","last_active":"2024-05-11T13:33:52.132631+0000","last_peered":"2024-05-11T13:33:52.132631+0000","last_clean":"2024-05-11T13:33:52.132631+0000","last_became_active":"2024-05-11T13:28:35.205684+0000","last_became_peered":"2024-05-11T13:28:35.205684+0000","last_unstale":"2024-05-11T13:33:52.132631+0000","last_undegraded":"2024-05-11T13:33:52.132631+0000","last_fullsized":"2024-05-11T13:33:52.132631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:12.326574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100161+0000","last_change":"2024-05-11T13:28:49.689545+0000","last_active":"2024-05-11T13:33:52.100161+0000","last_peered":"2024-05-11T13:33:52.100161+0000","last_clean":"2024-05-11T13:33:52.100161+0000","last_became_active":"2024-05-11T13:28:49.688528+0000","last_became_peered":"2024-05-11T13:28:49.688528+0000","last_unstale":"2024-05-11T13:33:52.100161+0000","last_undegraded":"2024-05-11T13:33:52.100161+0000","last_fullsized":"2024-05-11T13:33:52.100161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:48.617520+0000","last_clean_scrub_stamp":"2024-05-11T13:28:48.617520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:49:35.418873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.514120+0000","last_change":"2024-05-11T13:29:40.560835+0000","last_active":"2024-05-11T13:33:52.514120+0000","last_peered":"2024-05-11T13:33:52.514120+0000","last_clean":"2024-05-11T13:33:52.514120+0000","last_became_active":"2024-05-11T13:29:40.554310+0000","last_became_peered":"2024-05-11T13:29:40.554310+0000","last_unstale":"2024-05-11T13:33:52.514120+0000","last_undegraded":"2024-05-11T13:33:52.514120+0000","last_fullsized":"2024-05-11T13:33:52.514120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:29.377356+0000","last_clean_scrub_stamp":"2024-05-11T13:28:29.377356+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:19:26.805261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.132512+0000","last_change":"2024-05-11T13:28:35.220001+0000","last_active":"2024-05-11T13:33:52.132512+0000","last_peered":"2024-05-11T13:33:52.132512+0000","last_clean":"2024-05-11T13:33:52.132512+0000","last_became_active":"2024-05-11T13:28:35.216768+0000","last_became_peered":"2024-05-11T13:28:35.216768+0000","last_unstale":"2024-05-11T13:33:52.132512+0000","last_undegraded":"2024-05-11T13:33:52.132512+0000","last_fullsized":"2024-05-11T13:33:52.132512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:54:23.547689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.584998+0000","last_change":"2024-05-11T13:28:35.213481+0000","last_active":"2024-05-11T13:33:52.584998+0000","last_peered":"2024-05-11T13:33:52.584998+0000","last_clean":"2024-05-11T13:33:52.584998+0000","last_became_active":"2024-05-11T13:28:35.212956+0000","last_became_peered":"2024-05-11T13:28:35.212956+0000","last_unstale":"2024-05-11T13:33:52.584998+0000","last_undegraded":"2024-05-11T13:33:52.584998+0000","last_fullsized":"2024-05-11T13:33:52.584998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:14.814155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952702+0000","last_change":"2024-05-11T13:29:40.554654+0000","last_active":"2024-05-11T13:33:51.952702+0000","last_peered":"2024-05-11T13:33:51.952702+0000","last_clean":"2024-05-11T13:33:51.952702+0000","last_became_active":"2024-05-11T13:29:40.551836+0000","last_became_peered":"2024-05-11T13:29:40.551836+0000","last_unstale":"2024-05-11T13:33:51.952702+0000","last_undegraded":"2024-05-11T13:33:51.952702+0000","last_fullsized":"2024-05-11T13:33:51.952702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:49:44.566051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.952138+0000","last_change":"2024-05-11T13:28:35.208925+0000","last_active":"2024-05-11T13:33:51.952138+0000","last_peered":"2024-05-11T13:33:51.952138+0000","last_clean":"2024-05-11T13:33:51.952138+0000","last_became_active":"2024-05-11T13:28:35.208055+0000","last_became_peered":"2024-05-11T13:28:35.208055+0000","last_unstale":"2024-05-11T13:33:51.952138+0000","last_undegraded":"2024-05-11T13:33:51.952138+0000","last_fullsized":"2024-05-11T13:33:51.952138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:20.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585665+0000","last_change":"2024-05-11T13:28:35.213226+0000","last_active":"2024-05-11T13:33:52.585665+0000","last_peered":"2024-05-11T13:33:52.585665+0000","last_clean":"2024-05-11T13:33:52.585665+0000","last_became_active":"2024-05-11T13:28:35.212839+0000","last_became_peered":"2024-05-11T13:28:35.212839+0000","last_unstale":"2024-05-11T13:33:52.585665+0000","last_undegraded":"2024-05-11T13:33:52.585665+0000","last_fullsized":"2024-05-11T13:33:52.585665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:37:05.312505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513858+0000","last_change":"2024-05-11T13:29:40.848092+0000","last_active":"2024-05-11T13:33:52.513858+0000","last_peered":"2024-05-11T13:33:52.513858+0000","last_clean":"2024-05-11T13:33:52.513858+0000","last_became_active":"2024-05-11T13:29:40.846950+0000","last_became_peered":"2024-05-11T13:29:40.846950+0000","last_unstale":"2024-05-11T13:33:52.513858+0000","last_undegraded":"2024-05-11T13:33:52.513858+0000","last_fullsized":"2024-05-11T13:33:52.513858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:18:06.675351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585503+0000","last_change":"2024-05-11T13:28:35.217529+0000","last_active":"2024-05-11T13:33:52.585503+0000","last_peered":"2024-05-11T13:33:52.585503+0000","last_clean":"2024-05-11T13:33:52.585503+0000","last_became_active":"2024-05-11T13:28:35.217062+0000","last_became_peered":"2024-05-11T13:28:35.217062+0000","last_unstale":"2024-05-11T13:33:52.585503+0000","last_undegraded":"2024-05-11T13:33:52.585503+0000","last_fullsized":"2024-05-11T13:33:52.585503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:24:33.955794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954490+0000","last_change":"2024-05-11T13:28:35.217777+0000","last_active":"2024-05-11T13:33:51.954490+0000","last_peered":"2024-05-11T13:33:51.954490+0000","last_clean":"2024-05-11T13:33:51.954490+0000","last_became_active":"2024-05-11T13:28:35.217381+0000","last_became_peered":"2024-05-11T13:28:35.217381+0000","last_unstale":"2024-05-11T13:33:51.954490+0000","last_undegraded":"2024-05-11T13:33:51.954490+0000","last_fullsized":"2024-05-11T13:33:51.954490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:43.973192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.086598+0000","last_change":"2024-05-11T13:28:35.209826+0000","last_active":"2024-05-11T13:33:52.086598+0000","last_peered":"2024-05-11T13:33:52.086598+0000","last_clean":"2024-05-11T13:33:52.086598+0000","last_became_active":"2024-05-11T13:28:35.209461+0000","last_became_peered":"2024-05-11T13:28:35.209461+0000","last_unstale":"2024-05-11T13:33:52.086598+0000","last_undegraded":"2024-05-11T13:33:52.086598+0000","last_fullsized":"2024-05-11T13:33:52.086598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:03.323045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.100553+0000","last_change":"2024-05-11T13:28:35.205252+0000","last_active":"2024-05-11T13:33:52.100553+0000","last_peered":"2024-05-11T13:33:52.100553+0000","last_clean":"2024-05-11T13:33:52.100553+0000","last_became_active":"2024-05-11T13:28:35.204835+0000","last_became_peered":"2024-05-11T13:28:35.204835+0000","last_unstale":"2024-05-11T13:33:52.100553+0000","last_undegraded":"2024-05-11T13:33:52.100553+0000","last_fullsized":"2024-05-11T13:33:52.100553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:10.032439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.085049+0000","last_change":"2024-05-11T13:28:35.219718+0000","last_active":"2024-05-11T13:33:52.085049+0000","last_peered":"2024-05-11T13:33:52.085049+0000","last_clean":"2024-05-11T13:33:52.085049+0000","last_became_active":"2024-05-11T13:28:35.219032+0000","last_became_peered":"2024-05-11T13:28:35.219032+0000","last_unstale":"2024-05-11T13:33:52.085049+0000","last_undegraded":"2024-05-11T13:33:52.085049+0000","last_fullsized":"2024-05-11T13:33:52.085049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:03:58.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:51.954180+0000","last_change":"2024-05-11T13:28:35.217820+0000","last_active":"2024-05-11T13:33:51.954180+0000","last_peered":"2024-05-11T13:33:51.954180+0000","last_clean":"2024-05-11T13:33:51.954180+0000","last_became_active":"2024-05-11T13:28:35.216260+0000","last_became_peered":"2024-05-11T13:28:35.216260+0000","last_unstale":"2024-05-11T13:33:51.954180+0000","last_undegraded":"2024-05-11T13:33:51.954180+0000","last_fullsized":"2024-05-11T13:33:51.954180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:20:47.188238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.513331+0000","last_change":"2024-05-11T13:28:35.209762+0000","last_active":"2024-05-11T13:33:52.513331+0000","last_peered":"2024-05-11T13:33:52.513331+0000","last_clean":"2024-05-11T13:33:52.513331+0000","last_became_active":"2024-05-11T13:28:35.207404+0000","last_became_peered":"2024-05-11T13:28:35.207404+0000","last_unstale":"2024-05-11T13:33:52.513331+0000","last_undegraded":"2024-05-11T13:33:52.513331+0000","last_fullsized":"2024-05-11T13:33:52.513331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:41:20.808009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-11T13:33:52.585291+0000","last_change":"2024-05-11T13:28:35.215137+0000","last_active":"2024-05-11T13:33:52.585291+0000","last_peered":"2024-05-11T13:33:52.585291+0000","last_clean":"2024-05-11T13:33:52.585291+0000","last_became_active":"2024-05-11T13:28:35.214745+0000","last_became_peered":"2024-05-11T13:28:35.214745+0000","last_unstale":"2024-05-11T13:33:52.585291+0000","last_undegraded":"2024-05-11T13:33:52.585291+0000","last_fullsized":"2024-05-11T13:33:52.585291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T13:28:34.153378+0000","last_clean_scrub_stamp":"2024-05-11T13:28:34.153378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:48:44.381501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":803105,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2963,"num_read_kb":1757,"num_write":576,"num_write_kb":2661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2409315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":431,"ondisk_log_size":431,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678099,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":530129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874330,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37496,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707656,"statfs":{"total":95995035648,"available":95956639744,"internally_reserved":0,"allocated":2088960,"data_stored":1515413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32620,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712532,"statfs":{"total":95995035648,"available":95961632768,"internally_reserved":0,"allocated":1478656,"data_stored":925899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234089,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1462272,"data_stored":921577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430320,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1129521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593847,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":525882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":596829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738528,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":526415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":595504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149837,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28888,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716264,"statfs":{"total":95995035648,"available":95965454336,"internally_reserved":0,"allocated":1794048,"data_stored":1245247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953606,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":531446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790078,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29104,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716048,"statfs":{"total":95995035648,"available":95965233152,"internally_reserved":0,"allocated":2015232,"data_stored":1449753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":73460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":401676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T13:34:07.081 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:34:07.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:07 smithi063 ceph-mon[29809]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 61 op/s 2024-05-11T13:34:07.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:07 smithi063 ceph-mon[29809]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:34:07.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:07 smithi071 ceph-mon[33293]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 61 op/s 2024-05-11T13:34:07.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:07 smithi071 ceph-mon[33293]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:34:07.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:07 smithi165 ceph-mon[33235]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 61 op/s 2024-05-11T13:34:07.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:07 smithi165 ceph-mon[33235]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T13:34:07.576 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:09.315 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:09 smithi165 ceph-mon[33235]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 61 op/s 2024-05-11T13:34:09.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:09 smithi063 ceph-mon[29809]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 61 op/s 2024-05-11T13:34:09.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:09 smithi071 ceph-mon[33293]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 61 op/s 2024-05-11T13:34:10.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.020+0000 7fcb2a400700 1 -- 172.21.15.63:0/1202575694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24106130 msgr2=0x7fcb24112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.020+0000 7fcb2a400700 1 --2- 172.21.15.63:0/1202575694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24106130 0x7fcb24112640 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00b0a0 tx=0x7fcb1c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.020+0000 7fcb2a400700 1 -- 172.21.15.63:0/1202575694 shutdown_connections 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.020+0000 7fcb2a400700 1 --2- 172.21.15.63:0/1202575694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24106130 0x7fcb24112640 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.020+0000 7fcb2a400700 1 --2- 172.21.15.63:0/1202575694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb2410b770 0x7fcb2410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.020+0000 7fcb2a400700 1 --2- 172.21.15.63:0/1202575694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb24105770 0x7fcb24105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.020+0000 7fcb2a400700 1 -- 172.21.15.63:0/1202575694 >> 172.21.15.63:0/1202575694 conn(0x7fcb24101260 msgr2=0x7fcb24103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.021+0000 7fcb2a400700 1 -- 172.21.15.63:0/1202575694 shutdown_connections 2024-05-11T13:34:10.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.021+0000 7fcb2a400700 1 -- 172.21.15.63:0/1202575694 wait complete. 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 Processor -- start 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 -- start start 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb24105770 0x7fcb241a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb24106130 0x7fcb241a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb2410b770 0x7fcb2419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24115850 con 0x7fcb2410b770 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241156d0 con 0x7fcb24105770 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.022+0000 7fcb2a400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241159d0 con 0x7fcb24106130 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb2410b770 0x7fcb2419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb2410b770 0x7fcb2419bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52028/0 (socket says 172.21.15.63:52028) 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 -- 172.21.15.63:0/601066939 learned_addr learned my addr 172.21.15.63:0/601066939 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:10.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb24105770 0x7fcb241a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:10.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb22a00700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb24106130 0x7fcb241a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:10.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 -- 172.21.15.63:0/601066939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb24106130 msgr2=0x7fcb241a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:10.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb24106130 0x7fcb241a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 -- 172.21.15.63:0/601066939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb24105770 msgr2=0x7fcb241a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:10.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb24105770 0x7fcb241a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 -- 172.21.15.63:0/601066939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2419c210 con 0x7fcb2410b770 2024-05-11T13:34:10.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.023+0000 7fcb23e00700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb2410b770 0x7fcb2419bab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00b070 tx=0x7fcb1c004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:10.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.024+0000 7fcb1be00700 1 -- 172.21.15.63:0/601066939 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c021070 con 0x7fcb2410b770 2024-05-11T13:34:10.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.024+0000 7fcb1be00700 1 -- 172.21.15.63:0/601066939 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb1c007c00 con 0x7fcb2410b770 2024-05-11T13:34:10.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.024+0000 7fcb1be00700 1 -- 172.21.15.63:0/601066939 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c004780 con 0x7fcb2410b770 2024-05-11T13:34:10.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.024+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2419c4a0 con 0x7fcb2410b770 2024-05-11T13:34:10.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.024+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb241a87c0 con 0x7fcb2410b770 2024-05-11T13:34:10.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.025+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2406a240 con 0x7fcb2410b770 2024-05-11T13:34:10.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.028+0000 7fcb1be00700 1 -- 172.21.15.63:0/601066939 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7fcb1c003f20 con 0x7fcb2410b770 2024-05-11T13:34:10.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.029+0000 7fcb1be00700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb080b2b10 0x7fcb080b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:10.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.029+0000 7fcb1be00700 1 -- 172.21.15.63:0/601066939 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fcb1c0e41c0 con 0x7fcb2410b770 2024-05-11T13:34:10.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.029+0000 7fcb23400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb080b2b10 0x7fcb080b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:10.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.030+0000 7fcb23400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb080b2b10 0x7fcb080b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005fd0 tx=0x7fcb1400af00 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:10.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.032+0000 7fcb1be00700 1 -- 172.21.15.63:0/601066939 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb1c0ab5f0 con 0x7fcb2410b770 2024-05-11T13:34:10.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.326+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb2402e570 con 0x7fcb2410b770 2024-05-11T13:34:10.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.328+0000 7fcb1be00700 1 -- 172.21.15.63:0/601066939 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fcb1c0aaef0 con 0x7fcb2410b770 2024-05-11T13:34:10.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:34:10.327 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:33:50.708187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:34:10.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb080b2b10 msgr2=0x7fcb080b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:10.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb080b2b10 0x7fcb080b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005fd0 tx=0x7fcb1400af00 comp rx=0 tx=0).stop 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb2410b770 msgr2=0x7fcb2419bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb2410b770 0x7fcb2419bab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00b070 tx=0x7fcb1c004e00 comp rx=0 tx=0).stop 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 shutdown_connections 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb080b2b10 0x7fcb080b4fd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb2410b770 0x7fcb2419bab0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb24105770 0x7fcb241a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 --2- 172.21.15.63:0/601066939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb24106130 0x7fcb241a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.331+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 >> 172.21.15.63:0/601066939 conn(0x7fcb24101260 msgr2=0x7fcb241018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.332+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 shutdown_connections 2024-05-11T13:34:10.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:10.332+0000 7fcb2a400700 1 -- 172.21.15.63:0/601066939 wait complete. 2024-05-11T13:34:10.332 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-05-11T13:34:11.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:34:11.361 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:11 smithi063 ceph-mon[29809]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-05-11T13:34:11.361 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:11 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/601066939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:11.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:11 smithi071 ceph-mon[33293]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-05-11T13:34:11.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:11 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/601066939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:11.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:11 smithi165 ceph-mon[33235]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-05-11T13:34:11.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:11 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/601066939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:11.734 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:13.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:13 smithi063 ceph-mon[29809]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-11T13:34:13.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:13 smithi071 ceph-mon[33293]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-11T13:34:13.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:13 smithi165 ceph-mon[33235]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-11T13:34:14.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.052+0000 7fee65c00700 1 -- 172.21.15.63:0/2376412745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee60106150 msgr2=0x7fee60112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.052+0000 7fee65c00700 1 --2- 172.21.15.63:0/2376412745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee60106150 0x7fee60112660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b0a0 tx=0x7fee5800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.052+0000 7fee65c00700 1 -- 172.21.15.63:0/2376412745 shutdown_connections 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.052+0000 7fee65c00700 1 --2- 172.21.15.63:0/2376412745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee60106150 0x7fee60112660 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.052+0000 7fee65c00700 1 --2- 172.21.15.63:0/2376412745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6010b790 0x7fee6010bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.052+0000 7fee65c00700 1 --2- 172.21.15.63:0/2376412745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60105790 0x7fee60105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.052+0000 7fee65c00700 1 -- 172.21.15.63:0/2376412745 >> 172.21.15.63:0/2376412745 conn(0x7fee60101280 msgr2=0x7fee601036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.053+0000 7fee65c00700 1 -- 172.21.15.63:0/2376412745 shutdown_connections 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.053+0000 7fee65c00700 1 -- 172.21.15.63:0/2376412745 wait complete. 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 Processor -- start 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 -- start start 2024-05-11T13:34:14.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee60105790 0x7fee601a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:14.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60106150 0x7fee601a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:14.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee6010b790 0x7fee6019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:14.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee60115820 con 0x7fee6010b790 2024-05-11T13:34:14.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fee601156a0 con 0x7fee60105790 2024-05-11T13:34:14.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.054+0000 7fee65c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee601159a0 con 0x7fee60106150 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee6010b790 0x7fee6019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee6010b790 0x7fee6019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52046/0 (socket says 172.21.15.63:52046) 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 -- 172.21.15.63:0/3264236363 learned_addr learned my addr 172.21.15.63:0/3264236363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5ea00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee60105790 0x7fee601a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5e000700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60106150 0x7fee601a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 -- 172.21.15.63:0/3264236363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60106150 msgr2=0x7fee601a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60106150 0x7fee601a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 -- 172.21.15.63:0/3264236363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee60105790 msgr2=0x7fee601a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee60105790 0x7fee601a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.055+0000 7fee5f400700 1 -- 172.21.15.63:0/3264236363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6019c1c0 con 0x7fee6010b790 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.056+0000 7fee5f400700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee6010b790 0x7fee6019ba60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee580062a0 tx=0x7fee58004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.056+0000 7fee57400700 1 -- 172.21.15.63:0/3264236363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58021070 con 0x7fee6010b790 2024-05-11T13:34:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.056+0000 7fee57400700 1 -- 172.21.15.63:0/3264236363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee58007c80 con 0x7fee6010b790 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.056+0000 7fee57400700 1 -- 172.21.15.63:0/3264236363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee580052a0 con 0x7fee6010b790 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.056+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6019c450 con 0x7fee6010b790 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.056+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a8770 con 0x7fee6010b790 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.058+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee60106ed0 con 0x7fee6010b790 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.060+0000 7fee57400700 1 -- 172.21.15.63:0/3264236363 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7fee58024070 con 0x7fee6010b790 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.061+0000 7fee57400700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fee440b2b90 0x7fee440b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.062+0000 7fee57400700 1 -- 172.21.15.63:0/3264236363 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fee580e3ea0 con 0x7fee6010b790 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.062+0000 7fee5ea00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fee440b2b90 0x7fee440b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.062+0000 7fee5ea00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fee440b2b90 0x7fee440b5050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee6019cba0 tx=0x7fee5000a380 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.066+0000 7fee57400700 1 -- 172.21.15.63:0/3264236363 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee580ab2d0 con 0x7fee6010b790 2024-05-11T13:34:14.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.355+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee6019cf90 con 0x7fee6010b790 2024-05-11T13:34:14.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.357+0000 7fee57400700 1 -- 172.21.15.63:0/3264236363 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fee580aabd0 con 0x7fee6010b790 2024-05-11T13:34:14.356 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:34:14.356 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:33:50.708187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:34:14.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fee440b2b90 msgr2=0x7fee440b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fee440b2b90 0x7fee440b5050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee6019cba0 tx=0x7fee5000a380 comp rx=0 tx=0).stop 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee6010b790 msgr2=0x7fee6019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee6010b790 0x7fee6019ba60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee580062a0 tx=0x7fee58004e20 comp rx=0 tx=0).stop 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 shutdown_connections 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fee440b2b90 0x7fee440b5050 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee6010b790 0x7fee6019ba60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee60105790 0x7fee601a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 --2- 172.21.15.63:0/3264236363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60106150 0x7fee601a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:14.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 >> 172.21.15.63:0/3264236363 conn(0x7fee60101280 msgr2=0x7fee601017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:14.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.360+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 shutdown_connections 2024-05-11T13:34:14.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:14.361+0000 7fee65c00700 1 -- 172.21.15.63:0/3264236363 wait complete. 2024-05-11T13:34:14.361 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-05-11T13:34:15.254 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:34:15.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:15 smithi063 ceph-mon[29809]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-11T13:34:15.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:15 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3264236363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:15.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:15 smithi071 ceph-mon[33293]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-11T13:34:15.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:15 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3264236363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:15.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:15 smithi165 ceph-mon[33235]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-11T13:34:15.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:15 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3264236363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:17.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:17 smithi063 ceph-mon[29809]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-11T13:34:17.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:17 smithi071 ceph-mon[33293]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-11T13:34:17.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:17 smithi165 ceph-mon[33235]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-11T13:34:18.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 -- 172.21.15.63:0/1792980316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c10b680 msgr2=0x7f4f8c10ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:18.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 --2- 172.21.15.63:0/1792980316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c10b680 0x7f4f8c10ba60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009990 tx=0x7f4f7c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 -- 172.21.15.63:0/1792980316 shutdown_connections 2024-05-11T13:34:18.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 --2- 172.21.15.63:0/1792980316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f8c106040 0x7f4f8c112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 --2- 172.21.15.63:0/1792980316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c10b680 0x7f4f8c10ba60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 --2- 172.21.15.63:0/1792980316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f8c105680 0x7f4f8c105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 -- 172.21.15.63:0/1792980316 >> 172.21.15.63:0/1792980316 conn(0x7f4f8c101170 msgr2=0x7f4f8c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:18.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 -- 172.21.15.63:0/1792980316 shutdown_connections 2024-05-11T13:34:18.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.055+0000 7f4f91200700 1 -- 172.21.15.63:0/1792980316 wait complete. 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.056+0000 7f4f91200700 1 Processor -- start 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.056+0000 7f4f91200700 1 -- start start 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f91200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c105680 0x7f4f8c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f91200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f8c106040 0x7f4f8c1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f91200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f8c10b680 0x7f4f8c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f91200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c115760 con 0x7f4f8c105680 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f91200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1155e0 con 0x7f4f8c106040 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f91200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1158e0 con 0x7f4f8c10b680 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f8b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c105680 0x7f4f8c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:18.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f8b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c105680 0x7f4f8c1a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52064/0 (socket says 172.21.15.63:52064) 2024-05-11T13:34:18.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.057+0000 7f4f8b400700 1 -- 172.21.15.63:0/4209814278 learned_addr learned my addr 172.21.15.63:0/4209814278 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:18.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f82a00700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f8c106040 0x7f4f8c1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:18.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f8be00700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f8c10b680 0x7f4f8c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:18.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f8b400700 1 -- 172.21.15.63:0/4209814278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f8c10b680 msgr2=0x7f4f8c19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:18.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f8b400700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f8c10b680 0x7f4f8c19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f8b400700 1 -- 172.21.15.63:0/4209814278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f8c106040 msgr2=0x7f4f8c1a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:18.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f8b400700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f8c106040 0x7f4f8c1a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f8b400700 1 -- 172.21.15.63:0/4209814278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c19c1a0 con 0x7f4f8c105680 2024-05-11T13:34:18.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f8b400700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c105680 0x7f4f8c1a1880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c000c00 tx=0x7f4f7c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:18.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.058+0000 7f4f88c00700 1 -- 172.21.15.63:0/4209814278 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c027070 con 0x7f4f8c105680 2024-05-11T13:34:18.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.059+0000 7f4f88c00700 1 -- 172.21.15.63:0/4209814278 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f7c00fc80 con 0x7f4f8c105680 2024-05-11T13:34:18.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.059+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c19c430 con 0x7f4f8c105680 2024-05-11T13:34:18.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.059+0000 7f4f88c00700 1 -- 172.21.15.63:0/4209814278 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c022d60 con 0x7f4f8c105680 2024-05-11T13:34:18.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.059+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c1a8750 con 0x7f4f8c105680 2024-05-11T13:34:18.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.060+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8c06a240 con 0x7f4f8c105680 2024-05-11T13:34:18.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.062+0000 7f4f88c00700 1 -- 172.21.15.63:0/4209814278 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7f4f7c02a070 con 0x7f4f8c105680 2024-05-11T13:34:18.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.064+0000 7f4f88c00700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f700b2b10 0x7f4f700b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:18.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.064+0000 7f4f88c00700 1 -- 172.21.15.63:0/4209814278 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f4f7c0e3dc0 con 0x7f4f8c105680 2024-05-11T13:34:18.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.064+0000 7f4f82a00700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f700b2b10 0x7f4f700b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:18.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.065+0000 7f4f82a00700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f700b2b10 0x7f4f700b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f74005ea0 tx=0x7f4f74005e30 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:18.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.067+0000 7f4f88c00700 1 -- 172.21.15.63:0/4209814278 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f7c031080 con 0x7f4f8c105680 2024-05-11T13:34:18.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.309+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f8c02e570 con 0x7f4f8c105680 2024-05-11T13:34:18.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.314+0000 7f4f88c00700 1 -- 172.21.15.63:0/4209814278 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25134 (secure 0 0 0) 0x7f4f7c08c240 con 0x7f4f8c105680 2024-05-11T13:34:18.313 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:34:18.313 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":91,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:33:51.933947+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/1742389159":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/712822368":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/2407940590":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/1742389159":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/66647058":"2024-05-12T13:33:51.933837+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/531186954":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:34:18.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f700b2b10 msgr2=0x7f4f700b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:18.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f700b2b10 0x7f4f700b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f74005ea0 tx=0x7f4f74005e30 comp rx=0 tx=0).stop 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c105680 msgr2=0x7f4f8c1a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c105680 0x7f4f8c1a1880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c000c00 tx=0x7f4f7c0036f0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 shutdown_connections 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f700b2b10 0x7f4f700b4fd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f8c10b680 0x7f4f8c19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8c105680 0x7f4f8c1a1880 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 --2- 172.21.15.63:0/4209814278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f8c106040 0x7f4f8c1a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.317+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 >> 172.21.15.63:0/4209814278 conn(0x7f4f8c101170 msgr2=0x7f4f8c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.318+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 shutdown_connections 2024-05-11T13:34:18.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:18.318+0000 7f4f91200700 1 -- 172.21.15.63:0/4209814278 wait complete. 2024-05-11T13:34:19.277 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-11T13:34:19.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:19 smithi063 ceph-mon[29809]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-11T13:34:19.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:19 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4209814278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:34:19.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:19 smithi071 ceph-mon[33293]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-11T13:34:19.453 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:19 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4209814278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:34:19.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:19 smithi165 ceph-mon[33235]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-11T13:34:19.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:19 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4209814278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:34:19.750 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:21.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:21 smithi063 ceph-mon[29809]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-11T13:34:21.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:21 smithi071 ceph-mon[33293]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-11T13:34:21.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:21 smithi165 ceph-mon[33235]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-11T13:34:23.195 INFO:teuthology.orchestra.run.smithi063.stdout:{"run_dir":"/var/run/ceph"} 2024-05-11T13:34:23.195 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-11T13:34:23.196 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 4085 2024-05-11T13:34:23.196 DEBUG:tasks.quiescer:Yielding 2024-05-11T13:34:23.196 INFO:teuthology.run_tasks:Running task exec... 2024-05-11T13:34:23.206 INFO:teuthology.task.exec:Executing custom commands... 2024-05-11T13:34:23.206 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:34:23.206 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-11T13:34:23.207 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:34:23.314 INFO:teuthology.orchestra.run.smithi063.stdout:total 0 2024-05-11T13:34:23.344 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-11T13:34:23.344 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-fuse 1.1T 384M 1.1T 1% /tmp/cephfs 2024-05-11T13:34:23.350 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-11T13:34:23.413 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:23 smithi063 ceph-mon[29809]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-11T13:34:23.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:23 smithi071 ceph-mon[33293]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-11T13:34:23.502 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-11T13:34:23.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:23 smithi165 ceph-mon[33235]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-11T13:34:23.615 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-11T13:34:23.652 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:25.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:25 smithi063 ceph-mon[29809]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-11T13:34:25.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:25 smithi071 ceph-mon[33293]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-11T13:34:25.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:25 smithi165 ceph-mon[33235]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-11T13:34:26.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-11T13:34:26.250 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-11T13:34:26.255 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-11T13:34:26.355 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-11T13:34:26.411 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:26 smithi063 ceph-mon[29809]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:34:26.466 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/: 2024-05-11T13:34:26.466 INFO:teuthology.orchestra.run.smithi063.stdout:total 2 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 .. 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 data 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data: 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:34:26.467 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 .. 2024-05-11T13:34:26.474 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-11T13:34:26.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:26 smithi165 ceph-mon[33235]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:34:26.600 INFO:teuthology.orchestra.run.smithi063.stdout:[Service] 2024-05-11T13:34:26.600 INFO:teuthology.orchestra.run.smithi063.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-11T13:34:26.601 INFO:teuthology.orchestra.run.smithi063.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-11T13:34:26.606 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-11T13:34:26.702 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:26 smithi071 ceph-mon[33293]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:34:26.820 INFO:teuthology.orchestra.run.smithi063.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-11T13:34:26.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 -- 172.21.15.63:0/3214307024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 msgr2=0x7fc1e410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:26.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/3214307024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e410fa10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009990 tx=0x7fc1d0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:34:26.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 -- 172.21.15.63:0/3214307024 shutdown_connections 2024-05-11T13:34:26.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/3214307024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e410ff50 0x7fc1e4112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:26.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/3214307024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e4075780 0x7fc1e4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:26.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/3214307024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e410fa10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:26.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 -- 172.21.15.63:0/3214307024 >> 172.21.15.63:0/3214307024 conn(0x7fc1e407a6b0 msgr2=0x7fc1e407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:26.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 -- 172.21.15.63:0/3214307024 shutdown_connections 2024-05-11T13:34:26.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.837+0000 7fc1e9e00700 1 -- 172.21.15.63:0/3214307024 wait complete. 2024-05-11T13:34:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.838+0000 7fc1e9e00700 1 Processor -- start 2024-05-11T13:34:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.838+0000 7fc1e9e00700 1 -- start start 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.838+0000 7fc1e9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e4075780 0x7fc1e41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e41a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e410ff50 0x7fc1e419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e9e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4115830 con 0x7fc1e40760a0 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e9e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41156b0 con 0x7fc1e410ff50 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e9e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41159b0 con 0x7fc1e4075780 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e41a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e41a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35884/0 (socket says 172.21.15.63:35884) 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e2000700 1 -- 172.21.15.63:0/2082299937 learned_addr learned my addr 172.21.15.63:0/2082299937 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e3400700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e410ff50 0x7fc1e419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.839+0000 7fc1e2a00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e4075780 0x7fc1e41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e2000700 1 -- 172.21.15.63:0/2082299937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e4075780 msgr2=0x7fc1e41a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e2000700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e4075780 0x7fc1e41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e2000700 1 -- 172.21.15.63:0/2082299937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e410ff50 msgr2=0x7fc1e419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e2000700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e410ff50 0x7fc1e419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e2000700 1 -- 172.21.15.63:0/2082299937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e419c410 con 0x7fc1e40760a0 2024-05-11T13:34:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e2000700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e41a1fa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0000c00 tx=0x7fc1d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e8a00700 1 -- 172.21.15.63:0/2082299937 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0027070 con 0x7fc1e40760a0 2024-05-11T13:34:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e8a00700 1 -- 172.21.15.63:0/2082299937 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1d000fe20 con 0x7fc1e40760a0 2024-05-11T13:34:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e419c6a0 con 0x7fc1e40760a0 2024-05-11T13:34:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.840+0000 7fc1e8a00700 1 -- 172.21.15.63:0/2082299937 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0022d00 con 0x7fc1e40760a0 2024-05-11T13:34:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.841+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e41a89e0 con 0x7fc1e40760a0 2024-05-11T13:34:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.842+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e4076e70 con 0x7fc1e40760a0 2024-05-11T13:34:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.845+0000 7fc1e8a00700 1 -- 172.21.15.63:0/2082299937 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7fc1d002a070 con 0x7fc1e40760a0 2024-05-11T13:34:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.847+0000 7fc1e8a00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc1c80b2b90 0x7fc1c80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.847+0000 7fc1e8a00700 1 -- 172.21.15.63:0/2082299937 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fc1d00e3d60 con 0x7fc1e40760a0 2024-05-11T13:34:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.847+0000 7fc1e2a00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc1c80b2b90 0x7fc1c80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.848+0000 7fc1e2a00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc1c80b2b90 0x7fc1c80b5050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400ba60 tx=0x7fc1d4005d50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:26.849+0000 7fc1e8a00700 1 -- 172.21.15.63:0/2082299937 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1d0013070 con 0x7fc1e40760a0 2024-05-11T13:34:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.145+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1e419cff0 con 0x7fc1e40760a0 2024-05-11T13:34:27.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.146+0000 7fc1e8a00700 1 -- 172.21.15.63:0/2082299937 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fc1d00a6020 con 0x7fc1e40760a0 2024-05-11T13:34:27.145 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:34:27.145 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:33:50.708187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:34:27.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc1c80b2b90 msgr2=0x7fc1c80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc1c80b2b90 0x7fc1c80b5050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400ba60 tx=0x7fc1d4005d50 comp rx=0 tx=0).stop 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 msgr2=0x7fc1e41a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e41a1fa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0000c00 tx=0x7fc1d0009f90 comp rx=0 tx=0).stop 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 shutdown_connections 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc1c80b2b90 0x7fc1c80b5050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e410ff50 0x7fc1e419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e4075780 0x7fc1e41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 --2- 172.21.15.63:0/2082299937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1e40760a0 0x7fc1e41a1fa0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 >> 172.21.15.63:0/2082299937 conn(0x7fc1e407a6b0 msgr2=0x7fc1e4100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 shutdown_connections 2024-05-11T13:34:27.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:27.150+0000 7fc1e9e00700 1 -- 172.21.15.63:0/2082299937 wait complete. 2024-05-11T13:34:27.150 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-05-11T13:34:27.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:27 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2082299937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:27.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:27 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2082299937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:27.702 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:27 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2082299937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:34:28.001 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-11T13:28:38.143704+0000', 'modified': '2024-05-11T13:33:50.708187+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34472, 'mds_1': 34451, 'mds_2': 34427, 'mds_3': 34403, 'mds_4': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6839/3189418422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6838', 'nonce': 3189418422}, {'type': 'v1', 'addr': '172.21.15.63:6839', 'nonce': 3189418422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14577': {'gid': 14577, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6841/3763187067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6840', 'nonce': 3763187067}, {'type': 'v1', 'addr': '172.21.15.63:6841', 'nonce': 3763187067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/2703259087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 2703259087}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 2703259087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/4024086251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 4024086251}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 4024086251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2308228126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2308228126}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2308228126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.165:6839/1121013005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1121013005}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1121013005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34403': {'gid': 34403, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 37, 'addr': '172.21.15.71:6833/1870291573', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 1870291573}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 1870291573}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.71:6835/1316489545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 1316489545}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 1316489545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34451': {'gid': 34451, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.71:6837/136473158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 136473158}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 136473158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.71:6839/1525017608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1525017608}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1525017608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34472, 'qdb_cluster': [34472, 34451, 34427, 24472, 34403]} 2024-05-11T13:34:28.001 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-11T13:34:28.001 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-11T13:34:28.489 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:28.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:28 smithi165 ceph-mon[33235]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:34:28.652 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:28 smithi063 ceph-mon[29809]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:34:28.702 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:28 smithi071 ceph-mon[33293]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:34:30.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 -- 172.21.15.63:0/1759860981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa530104fb0 msgr2=0x7fa53010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 --2- 172.21.15.63:0/1759860981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa530104fb0 0x7fa53010d4a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009990 tx=0x7fa51c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 -- 172.21.15.63:0/1759860981 shutdown_connections 2024-05-11T13:34:30.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 --2- 172.21.15.63:0/1759860981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa53010dac0 0x7fa53010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 --2- 172.21.15.63:0/1759860981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa530104690 0x7fa530104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 --2- 172.21.15.63:0/1759860981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa530104fb0 0x7fa53010d4a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 -- 172.21.15.63:0/1759860981 >> 172.21.15.63:0/1759860981 conn(0x7fa5300fdee0 msgr2=0x7fa530100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:30.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 -- 172.21.15.63:0/1759860981 shutdown_connections 2024-05-11T13:34:30.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.870+0000 7fa52fe00700 1 -- 172.21.15.63:0/1759860981 wait complete. 2024-05-11T13:34:30.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.871+0000 7fa52fe00700 1 Processor -- start 2024-05-11T13:34:30.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.872+0000 7fa52fe00700 1 -- start start 2024-05-11T13:34:30.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.872+0000 7fa52fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa530104690 0x7fa53019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.872+0000 7fa52fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa530104fb0 0x7fa53019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.872+0000 7fa52fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa53010dac0 0x7fa5301a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.873+0000 7fa52fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301135c0 con 0x7fa53010dac0 2024-05-11T13:34:30.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.873+0000 7fa52fe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa530113440 con 0x7fa530104690 2024-05-11T13:34:30.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.873+0000 7fa52fe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa530113740 con 0x7fa530104fb0 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.873+0000 7fa52f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa53010dac0 0x7fa5301a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.873+0000 7fa52ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa530104690 0x7fa53019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.873+0000 7fa52f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa53010dac0 0x7fa5301a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43318/0 (socket says 172.21.15.63:43318) 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.873+0000 7fa52f400700 1 -- 172.21.15.63:0/119747143 learned_addr learned my addr 172.21.15.63:0/119747143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa52e000700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa530104fb0 0x7fa53019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa52f400700 1 -- 172.21.15.63:0/119747143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa530104fb0 msgr2=0x7fa53019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa52f400700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa530104fb0 0x7fa53019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa52f400700 1 -- 172.21.15.63:0/119747143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa530104690 msgr2=0x7fa53019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa52f400700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa530104690 0x7fa53019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa52f400700 1 -- 172.21.15.63:0/119747143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5301a0d20 con 0x7fa53010dac0 2024-05-11T13:34:30.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa52f400700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa53010dac0 0x7fa5301a05e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa528007ee0 tx=0x7fa52800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:30.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.874+0000 7fa527400700 1 -- 172.21.15.63:0/119747143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52800dbd0 con 0x7fa53010dac0 2024-05-11T13:34:30.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.875+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301a1010 con 0x7fa53010dac0 2024-05-11T13:34:30.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.875+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301a8890 con 0x7fa53010dac0 2024-05-11T13:34:30.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.875+0000 7fa527400700 1 -- 172.21.15.63:0/119747143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa52800dd70 con 0x7fa53010dac0 2024-05-11T13:34:30.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.875+0000 7fa527400700 1 -- 172.21.15.63:0/119747143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280133f0 con 0x7fa53010dac0 2024-05-11T13:34:30.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.879+0000 7fa527400700 1 -- 172.21.15.63:0/119747143 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7fa52801a020 con 0x7fa53010dac0 2024-05-11T13:34:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 Processor -- start 2024-05-11T13:34:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 -- start start 2024-05-11T13:34:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e7f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e79300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e798d0 0x55e0e7e79d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7dcb9d0 con 0x55e0e7e7ee40 2024-05-11T13:34:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7d69710 con 0x55e0e7e798d0 2024-05-11T13:34:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.880+0000 7fa54007cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7dc7990 con 0x55e0e7e7f760 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.881+0000 7fa52e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e7f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.881+0000 7fa52e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e7f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43334/0 (socket says 172.21.15.63:43334) 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.881+0000 7fa52e000700 1 -- 172.21.15.63:0/3159308543 learned_addr learned my addr 172.21.15.63:0/3159308543 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.881+0000 7fa52ea00700 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.881+0000 7fa527400700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa5140b2b90 0x7fa5140b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa52e000700 1 -- 172.21.15.63:0/3159308543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 msgr2=0x55e0e7e79300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa52e000700 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e79300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa52e000700 1 -- 172.21.15.63:0/3159308543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e798d0 msgr2=0x55e0e7e79d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa52e000700 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e798d0 0x55e0e7e79d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa52e000700 1 -- 172.21.15.63:0/3159308543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e7d18880 con 0x55e0e7e7ee40 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa527400700 1 -- 172.21.15.63:0/119747143 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa5280e2260 con 0x7fa53010dac0 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa52f400700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa5140b2b90 0x7fa5140b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.882+0000 7fa52e000700 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e7f220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa51c000c00 tx=0x7fa51c005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391f7ad71ef686b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.883+0000 7fa51be00700 1 -- 172.21.15.63:0/3159308543 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0050c0 con 0x55e0e7e7ee40 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.883+0000 7fa52f400700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa5140b2b90 0x7fa5140b5050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5280df500 tx=0x7fa528009560 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.883+0000 7fa51be00700 1 -- 172.21.15.63:0/3159308543 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa51c0037c0 con 0x55e0e7e7ee40 2024-05-11T13:34:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.883+0000 7fa51be00700 1 -- 172.21.15.63:0/3159308543 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00ba30 con 0x55e0e7e7ee40 2024-05-11T13:34:30.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 -- 172.21.15.63:0/3159308543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 msgr2=0x55e0e7e7f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e7f220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa51c000c00 tx=0x7fa51c005ee0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 -- 172.21.15.63:0/3159308543 shutdown_connections 2024-05-11T13:34:30.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e798d0 0x55e0e7e79d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e7f220 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3159308543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e79300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 -- 172.21.15.63:0/3159308543 >> 172.21.15.63:0/3159308543 conn(0x55e0e7da7b10 msgr2=0x55e0e7e73de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 -- 172.21.15.63:0/3159308543 shutdown_connections 2024-05-11T13:34:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.884+0000 7fa54007cb80 1 -- 172.21.15.63:0/3159308543 wait complete. 2024-05-11T13:34:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.885+0000 7fa54007cb80 1 -- start start 2024-05-11T13:34:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.885+0000 7fa54007cb80 1 Processor -- start 2024-05-11T13:34:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 0x55e0e7e82db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa54007cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e86150 con 0x55e0e7e798d0 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa54007cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7d31ad0 con 0x55e0e7e7f760 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa54007cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e862d0 con 0x55e0e7e7ee40 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa54007cb80 10 client.0 _ll_register_callbacks cb 0x55e0e7d5f330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa52e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa52e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e854d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43350/0 (socket says 172.21.15.63:43350) 2024-05-11T13:34:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa52e000700 1 -- 172.21.15.63:0/505469211 learned_addr learned my addr 172.21.15.63:0/505469211 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:30.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa52ea00700 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.886+0000 7fa52e000700 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 0x55e0e7e82db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa52e000700 1 -- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7ee40 msgr2=0x55e0e7e82870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa52e000700 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa52e000700 1 -- 172.21.15.63:0/505469211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 msgr2=0x55e0e7e82db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa52e000700 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 0x55e0e7e82db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa52e000700 1 -- 172.21.15.63:0/505469211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e7d913d0 con 0x55e0e7e798d0 2024-05-11T13:34:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa52e000700 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e854d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00b5c0 tx=0x7fa51c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac948199e17445e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa518c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa518c00700 1 -- 172.21.15.63:0/505469211 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c027040 con 0x55e0e7e798d0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa518c00700 1 -- 172.21.15.63:0/505469211 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa51c005680 con 0x55e0e7e798d0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.887+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51c0037c0 con 0x55e0e7e798d0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.888+0000 7fa518c00700 1 -- 172.21.15.63:0/505469211 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa51c005980 con 0x55e0e7e798d0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.888+0000 7fa54007cb80 10 client.15441 fetch_fsmap learned FSMap version 29 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.888+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0e7e90a90 con 0x55e0e7e798d0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.888+0000 7fa518c00700 1 -- 172.21.15.63:0/505469211 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa51c0037c0 con 0x55e0e7e798d0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.889+0000 7fa518c00700 1 -- 172.21.15.63:0/505469211 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa51c02d070 con 0x55e0e7e798d0 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.889+0000 7fa54007cb80 10 client.15441 fetch_fsmap finished waiting for FSMap version 29 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.889+0000 7fa54007cb80 10 client.15441 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.889+0000 7fa54007cb80 20 client.15441 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.889+0000 7fa54007cb80 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7e91de0 0x55e0e7e93fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.889+0000 7fa54007cb80 4 client.15441 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:34:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.889+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 --> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e7e46010 con 0x55e0e7e91de0 2024-05-11T13:34:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.890+0000 7fa52ea00700 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7e91de0 0x55e0e7e93fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.891+0000 7fa52ea00700 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7e91de0 0x55e0e7e93fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70054fe6996dc747 server_cookie=3db17ecb29318be8 in_seq=0 out_seq=0 2024-05-11T13:34:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.891+0000 7fa518c00700 10 client.15441 ms_handle_connect on v2:172.21.15.165:6838/1121013005 2024-05-11T13:34:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.893+0000 7fa518c00700 1 -- 172.21.15.63:0/505469211 <== mds.4 v2:172.21.15.165:6838/1121013005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0e7e46010 con 0x55e0e7e91de0 2024-05-11T13:34:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.893+0000 7fa518c00700 10 client.15441 handle_command_reply: tid=0 2024-05-11T13:34:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.893+0000 7fa54007cb80 1 client.15441 shutdown 2024-05-11T13:34:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 msgr2=0x55e0e7e854d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e854d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00b5c0 tx=0x7fa51c009fa0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 shutdown_connections 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7e91de0 0x55e0e7e93fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 0x55e0e7e82db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e854d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 --2- 172.21.15.63:0/505469211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 >> 172.21.15.63:0/505469211 conn(0x55e0e7da7b10 msgr2=0x55e0e7e782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa518c00700 0 client.15441 ms_handle_reset on v2:172.21.15.165:6838/1121013005 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 shutdown_connections 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 1 -- 172.21.15.63:0/505469211 wait complete. 2024-05-11T13:34:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.894+0000 7fa54007cb80 20 client.15441 trim_cache size 0 max 16384 2024-05-11T13:34:30.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.921+0000 7fa54007cb80 1 Processor -- start 2024-05-11T13:34:30.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.921+0000 7fa54007cb80 1 -- start start 2024-05-11T13:34:30.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e798d0 0x55e0e7e79300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 0x55e0e7e92490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa54007cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e92310 con 0x55e0e7e7ee40 2024-05-11T13:34:30.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa54007cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e929e0 con 0x55e0e7e7f760 2024-05-11T13:34:30.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa54007cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e930b0 con 0x55e0e7e798d0 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa52ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa52ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43358/0 (socket says 172.21.15.63:43358) 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.922+0000 7fa52ea00700 1 -- 172.21.15.63:0/3277303987 learned_addr learned my addr 172.21.15.63:0/3277303987 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa52e000700 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e798d0 0x55e0e7e79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa52ea00700 1 -- 172.21.15.63:0/3277303987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e798d0 msgr2=0x55e0e7e79300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa52ea00700 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e798d0 0x55e0e7e79300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa52ea00700 1 -- 172.21.15.63:0/3277303987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 msgr2=0x55e0e7e92490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa52ea00700 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 0x55e0e7e92490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa52ea00700 1 -- 172.21.15.63:0/3277303987 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e7e70770 con 0x55e0e7e7ee40 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa52ea00700 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82de0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55e0e7e7fb60 tx=0x7fa52000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3001350a818a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa51b400700 1 -- 172.21.15.63:0/3277303987 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520004310 con 0x55e0e7e7ee40 2024-05-11T13:34:30.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa51b400700 1 -- 172.21.15.63:0/3277303987 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa520023030 con 0x55e0e7e7ee40 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.923+0000 7fa51b400700 1 -- 172.21.15.63:0/3277303987 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520010d70 con 0x55e0e7e7ee40 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.924+0000 7fa54007cb80 1 -- 172.21.15.63:0/3277303987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 msgr2=0x55e0e7e82de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.924+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82de0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55e0e7e7fb60 tx=0x7fa52000ae90 comp rx=0 tx=0).stop 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.924+0000 7fa54007cb80 1 -- 172.21.15.63:0/3277303987 shutdown_connections 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.924+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7f760 0x55e0e7e92490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.924+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e798d0 0x55e0e7e79300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.924+0000 7fa54007cb80 1 --2- 172.21.15.63:0/3277303987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e82de0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.924+0000 7fa54007cb80 1 -- 172.21.15.63:0/3277303987 >> 172.21.15.63:0/3277303987 conn(0x55e0e7da7b10 msgr2=0x55e0e7e7fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:30.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.925+0000 7fa54007cb80 1 -- 172.21.15.63:0/3277303987 shutdown_connections 2024-05-11T13:34:30.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.925+0000 7fa54007cb80 1 -- 172.21.15.63:0/3277303987 wait complete. 2024-05-11T13:34:30.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.925+0000 7fa54007cb80 1 -- start start 2024-05-11T13:34:30.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.925+0000 7fa54007cb80 1 Processor -- start 2024-05-11T13:34:30.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e84720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa54007cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e84c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa54007cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e93e50 con 0x55e0e7e798d0 2024-05-11T13:34:30.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa54007cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e93780 con 0x55e0e7e7ee40 2024-05-11T13:34:30.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa54007cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0e7e94520 con 0x55e0e7e7f760 2024-05-11T13:34:30.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa54007cb80 10 client.0 _ll_register_callbacks cb 0x55e0e7d5f330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa52e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa52e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e841e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43368/0 (socket says 172.21.15.63:43368) 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.926+0000 7fa52e000700 1 -- 172.21.15.63:0/1538765332 learned_addr learned my addr 172.21.15.63:0/1538765332 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52ea00700 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e84720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52e000700 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e84c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52e000700 1 -- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 msgr2=0x55e0e7e84c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52e000700 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e84c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52e000700 1 -- 172.21.15.63:0/1538765332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7ee40 msgr2=0x55e0e7e84720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52e000700 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e84720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52e000700 1 -- 172.21.15.63:0/1538765332 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e7e9bc40 con 0x55e0e7e798d0 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa52e000700 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e841e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa51c005f50 tx=0x7fa51c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c8349fb2d03205 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:30.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.927+0000 7fa518c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:30.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.928+0000 7fa518c00700 1 -- 172.21.15.63:0/1538765332 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0037c0 con 0x55e0e7e798d0 2024-05-11T13:34:30.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.928+0000 7fa518c00700 1 -- 172.21.15.63:0/1538765332 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fa51c02f590 con 0x55e0e7e798d0 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.928+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa520023030 con 0x55e0e7e798d0 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.928+0000 7fa518c00700 1 -- 172.21.15.63:0/1538765332 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa51c044050 con 0x55e0e7e798d0 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.928+0000 7fa54007cb80 10 client.15447 fetch_fsmap learned FSMap version 29 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.928+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0e7f06d50 con 0x55e0e7e798d0 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa518c00700 1 -- 172.21.15.63:0/1538765332 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa520023030 con 0x55e0e7e798d0 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa518c00700 1 -- 172.21.15.63:0/1538765332 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa51c02d030 con 0x55e0e7e798d0 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa54007cb80 10 client.15447 fetch_fsmap finished waiting for FSMap version 29 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa54007cb80 10 client.15447 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa54007cb80 20 client.15447 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa54007cb80 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7f0b8a0 0x55e0e7f0a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa54007cb80 4 client.15447 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:34:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.929+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 --> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] -- command(tid 0: {"prefix": "status"}) -- 0x55e0e7dd93b0 con 0x55e0e7f0b8a0 2024-05-11T13:34:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.930+0000 7fa52ea00700 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7f0b8a0 0x55e0e7f0a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.931+0000 7fa52ea00700 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7f0b8a0 0x55e0e7f0a5b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31899b823c5b7156 server_cookie=8f73077fd41909b0 in_seq=0 out_seq=0 2024-05-11T13:34:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.931+0000 7fa518c00700 10 client.15447 ms_handle_connect on v2:172.21.15.165:6838/1121013005 2024-05-11T13:34:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.931+0000 7fa518c00700 1 -- 172.21.15.63:0/1538765332 <== mds.4 v2:172.21.15.165:6838/1121013005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e0e7dd93b0 con 0x55e0e7f0b8a0 2024-05-11T13:34:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.931+0000 7fa518c00700 10 client.15447 handle_command_reply: tid=0 2024-05-11T13:34:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.931+0000 7fa54007cb80 1 client.15447 shutdown 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 msgr2=0x55e0e7e841e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e841e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa51c005f50 tx=0x7fa51c005fb0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 shutdown_connections 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:6838/1121013005,v1:172.21.15.165:6839/1121013005] conn(0x55e0e7f0b8a0 0x55e0e7f0a5b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0e7e7f760 0x55e0e7e84c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0e7e798d0 0x55e0e7e841e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 --2- 172.21.15.63:0/1538765332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0e7e7ee40 0x55e0e7e84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 >> 172.21.15.63:0/1538765332 conn(0x55e0e7da7b10 msgr2=0x55e0e7e7fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa518c00700 0 client.15447 ms_handle_reset on v2:172.21.15.165:6838/1121013005 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.932+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 shutdown_connections 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.933+0000 7fa54007cb80 1 -- 172.21.15.63:0/1538765332 wait complete. 2024-05-11T13:34:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.933+0000 7fa54007cb80 20 client.15447 trim_cache size 0 max 16384 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 4, 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 24472, 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 273.17606158000001, 2024-05-11T13:34:30.932 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 29, 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 90, 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 385.24359962800003 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.933+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa5140b2b90 msgr2=0x7fa5140b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.933+0000 7fa52fe00700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa5140b2b90 0x7fa5140b5050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5280df500 tx=0x7fa528009560 comp rx=0 tx=0).stop 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.933+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa53010dac0 msgr2=0x7fa5301a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.933+0000 7fa52fe00700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa53010dac0 0x7fa5301a05e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa528007ee0 tx=0x7fa52800d3b0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 shutdown_connections 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa5140b2b90 0x7fa5140b5050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa53010dac0 0x7fa5301a05e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa530104690 0x7fa53019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 --2- 172.21.15.63:0/119747143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa530104fb0 0x7fa53019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 >> 172.21.15.63:0/119747143 conn(0x7fa5300fdee0 msgr2=0x7fa5300feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 shutdown_connections 2024-05-11T13:34:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:30.934+0000 7fa52fe00700 1 -- 172.21.15.63:0/119747143 wait complete. 2024-05-11T13:34:31.314 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:31 smithi165 ceph-mon[33235]: pgmap v22: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-11T13:34:31.374 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:31 smithi063 ceph-mon[29809]: pgmap v22: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-11T13:34:31.406 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:31 smithi071 ceph-mon[33293]: pgmap v22: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-11T13:34:31.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.k status 2024-05-11T13:34:32.415 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:33.314 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:33 smithi165 ceph-mon[33235]: pgmap v23: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-05-11T13:34:33.436 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:33 smithi063 ceph-mon[29809]: pgmap v23: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-05-11T13:34:33.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:33 smithi071 ceph-mon[33293]: pgmap v23: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-05-11T13:34:34.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.869+0000 7f4839c00700 1 -- 172.21.15.63:0/3536060787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48341023c0 msgr2=0x7f4834110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.869+0000 7f4839c00700 1 --2- 172.21.15.63:0/3536060787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48341023c0 0x7f4834110390 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f482c00b0a0 tx=0x7f482c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.869+0000 7f4839c00700 1 -- 172.21.15.63:0/3536060787 shutdown_connections 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.869+0000 7f4839c00700 1 --2- 172.21.15.63:0/3536060787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48341023c0 0x7f4834110390 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.869+0000 7f4839c00700 1 --2- 172.21.15.63:0/3536060787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4834109570 0x7f4834109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.869+0000 7f4839c00700 1 --2- 172.21.15.63:0/3536060787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4834101a00 0x7f4834101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.869+0000 7f4839c00700 1 -- 172.21.15.63:0/3536060787 >> 172.21.15.63:0/3536060787 conn(0x7f48340788b0 msgr2=0x7f48340ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.870+0000 7f4839c00700 1 -- 172.21.15.63:0/3536060787 shutdown_connections 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.870+0000 7f4839c00700 1 -- 172.21.15.63:0/3536060787 wait complete. 2024-05-11T13:34:34.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.871+0000 7f4839c00700 1 Processor -- start 2024-05-11T13:34:34.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.871+0000 7f4839c00700 1 -- start start 2024-05-11T13:34:34.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4834101a00 0x7f483410a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48341023c0 0x7f483410a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4834109570 0x7f483410ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4834113530 con 0x7f4834109570 2024-05-11T13:34:34.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f48341133b0 con 0x7f48341023c0 2024-05-11T13:34:34.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48341136b0 con 0x7f4834101a00 2024-05-11T13:34:34.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4833400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48341023c0 0x7f483410a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4834109570 0x7f483410ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4834109570 0x7f483410ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43382/0 (socket says 172.21.15.63:43382) 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4839200700 1 -- 172.21.15.63:0/4285393381 learned_addr learned my addr 172.21.15.63:0/4285393381 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4833400700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48341023c0 0x7f483410a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:42330/0 (socket says 172.21.15.63:42330) 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.872+0000 7f4833e00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4834101a00 0x7f483410a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.873+0000 7f4839200700 1 -- 172.21.15.63:0/4285393381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4834101a00 msgr2=0x7f483410a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.873+0000 7f4839200700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4834101a00 0x7f483410a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.873+0000 7f4839200700 1 -- 172.21.15.63:0/4285393381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48341023c0 msgr2=0x7f483410a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.873+0000 7f4839200700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48341023c0 0x7f483410a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.873+0000 7f4839200700 1 -- 172.21.15.63:0/4285393381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483419b990 con 0x7f4834109570 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.873+0000 7f4833e00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4834101a00 0x7f483410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:34.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.873+0000 7f4839200700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4834109570 0x7f483410ae90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f482c00b830 tx=0x7f482c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:34.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.874+0000 7f4830c00700 1 -- 172.21.15.63:0/4285393381 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c021070 con 0x7f4834109570 2024-05-11T13:34:34.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.874+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483419bc20 con 0x7f4834109570 2024-05-11T13:34:34.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.874+0000 7f4830c00700 1 -- 172.21.15.63:0/4285393381 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f482c007b80 con 0x7f4834109570 2024-05-11T13:34:34.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.874+0000 7f4830c00700 1 -- 172.21.15.63:0/4285393381 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c0052a0 con 0x7f4834109570 2024-05-11T13:34:34.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.874+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483419c150 con 0x7f4834109570 2024-05-11T13:34:34.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.878+0000 7f4830c00700 1 -- 172.21.15.63:0/4285393381 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7f482c024070 con 0x7f4834109570 2024-05-11T13:34:34.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.879+0000 7f4845b49b80 1 Processor -- start 2024-05-11T13:34:34.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.879+0000 7f4845b49b80 1 -- start start 2024-05-11T13:34:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.879+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d601e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.879+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d5fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4845b49b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d54e9d0 con 0x55fe6d601aa0 2024-05-11T13:34:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4845b49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d4ec710 con 0x55fe6d5fc6c0 2024-05-11T13:34:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4845b49b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d54a990 con 0x55fe6d6023c0 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4833400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d601e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4833400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d601e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43386/0 (socket says 172.21.15.63:43386) 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4833400700 1 -- 172.21.15.63:0/2535408754 learned_addr learned my addr 172.21.15.63:0/2535408754 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4833e00700 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d5fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.880+0000 7f4833400700 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4833400700 1 -- 172.21.15.63:0/2535408754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 msgr2=0x55fe6d5fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4833400700 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d5fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4833400700 1 -- 172.21.15.63:0/2535408754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 msgr2=0x55fe6d5fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4833400700 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4833400700 1 -- 172.21.15.63:0/2535408754 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe6d49b880 con 0x55fe6d601aa0 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4830c00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f48180b2b10 0x7f48180b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4830c00700 1 -- 172.21.15.63:0/4285393381 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f482c00f020 con 0x7f4834109570 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4833400700 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d601e80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f482000bec0 tx=0x7f482000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93089e9fe9dfa80 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.881+0000 7f4833e00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f48180b2b10 0x7f48180b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.882+0000 7f4829600700 1 -- 172.21.15.63:0/2535408754 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820004030 con 0x55fe6d601aa0 2024-05-11T13:34:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.882+0000 7f4829600700 1 -- 172.21.15.63:0/2535408754 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4820004830 con 0x55fe6d601aa0 2024-05-11T13:34:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.882+0000 7f4829600700 1 -- 172.21.15.63:0/2535408754 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820005280 con 0x55fe6d601aa0 2024-05-11T13:34:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 -- 172.21.15.63:0/2535408754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 msgr2=0x55fe6d601e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d601e80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f482000bec0 tx=0x7f482000b8c0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4833e00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f48180b2b10 0x7f48180b4fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f482400bd30 tx=0x7f482400b710 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 -- 172.21.15.63:0/2535408754 shutdown_connections 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d601e80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 --2- 172.21.15.63:0/2535408754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d5fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 -- 172.21.15.63:0/2535408754 >> 172.21.15.63:0/2535408754 conn(0x55fe6d52ab10 msgr2=0x55fe6d5f6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 -- 172.21.15.63:0/2535408754 shutdown_connections 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.883+0000 7f4845b49b80 1 -- 172.21.15.63:0/2535408754 wait complete. 2024-05-11T13:34:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.884+0000 7f4845b49b80 1 -- start start 2024-05-11T13:34:34.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.884+0000 7f4845b49b80 1 Processor -- start 2024-05-11T13:34:34.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d608000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d601aa0 0x55fe6d6053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d6023c0 0x55fe6d6058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4845b49b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d608e70 con 0x55fe6d6023c0 2024-05-11T13:34:34.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4845b49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d4b4ad0 con 0x55fe6d5fc6c0 2024-05-11T13:34:34.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4845b49b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d608ff0 con 0x55fe6d601aa0 2024-05-11T13:34:34.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4845b49b80 10 client.0 _ll_register_callbacks cb 0x55fe6d4f5c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:34.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4839200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d6023c0 0x55fe6d6058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4839200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d6023c0 0x55fe6d6058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43400/0 (socket says 172.21.15.63:43400) 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.885+0000 7f4839200700 1 -- 172.21.15.63:0/3250593762 learned_addr learned my addr 172.21.15.63:0/3250593762 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4833e00700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d608000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4833400700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d601aa0 0x55fe6d6053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4839200700 1 -- 172.21.15.63:0/3250593762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d601aa0 msgr2=0x55fe6d6053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4839200700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d601aa0 0x55fe6d6053a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4839200700 1 -- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 msgr2=0x55fe6d608000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4839200700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d608000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4839200700 1 -- 172.21.15.63:0/3250593762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe6d5143d0 con 0x55fe6d6023c0 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4833e00700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d608000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.886+0000 7f4839200700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d6023c0 0x55fe6d6058e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f482c0a8220 tx=0x7f482c0aa7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264ad73d5dc71935 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.887+0000 7f481e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.887+0000 7f481e000700 1 -- 172.21.15.63:0/3250593762 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c0aafb0 con 0x55fe6d6023c0 2024-05-11T13:34:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.887+0000 7f481e000700 1 -- 172.21.15.63:0/3250593762 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f482c0a75a0 con 0x55fe6d6023c0 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.887+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4820004830 con 0x55fe6d6023c0 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.887+0000 7f481e000700 1 -- 172.21.15.63:0/3250593762 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f482c0af360 con 0x55fe6d6023c0 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.887+0000 7f4845b49b80 10 client.15459 fetch_fsmap learned FSMap version 29 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.887+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe6d613600 con 0x55fe6d6023c0 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f481e000700 1 -- 172.21.15.63:0/3250593762 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4820004830 con 0x55fe6d6023c0 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f481e000700 1 -- 172.21.15.63:0/3250593762 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f482c011070 con 0x55fe6d6023c0 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f4845b49b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f4845b49b80 10 client.15459 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f4845b49b80 20 client.15459 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d615c70 0x55fe6d6170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f4845b49b80 4 client.15459 mds_command: new command op to 34403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:34:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.888+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 --> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe6d5c9160 con 0x55fe6d615c70 2024-05-11T13:34:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.889+0000 7f4833e00700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d615c70 0x55fe6d6170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.890+0000 7f4833e00700 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d615c70 0x55fe6d6170e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=deae2133f956a9ef server_cookie=370c084a11b8ff27 in_seq=0 out_seq=0 2024-05-11T13:34:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.890+0000 7f481e000700 10 client.15459 ms_handle_connect on v2:172.21.15.71:6832/1870291573 2024-05-11T13:34:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f481e000700 1 -- 172.21.15.63:0/3250593762 <== mds.3 v2:172.21.15.71:6832/1870291573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe6d5c9160 con 0x55fe6d615c70 2024-05-11T13:34:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f481e000700 10 client.15459 handle_command_reply: tid=0 2024-05-11T13:34:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 client.15459 shutdown 2024-05-11T13:34:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d6023c0 msgr2=0x55fe6d6058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d6023c0 0x55fe6d6058e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f482c0a8220 tx=0x7f482c0aa7d0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 shutdown_connections 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d6023c0 0x55fe6d6058e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d608000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d615c70 0x55fe6d6170e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3250593762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d601aa0 0x55fe6d6053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 >> 172.21.15.63:0/3250593762 conn(0x55fe6d52ab10 msgr2=0x55fe6d5f1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.893+0000 7f481e000700 0 client.15459 ms_handle_reset on v2:172.21.15.71:6832/1870291573 2024-05-11T13:34:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.894+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 shutdown_connections 2024-05-11T13:34:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.894+0000 7f4845b49b80 1 -- 172.21.15.63:0/3250593762 wait complete. 2024-05-11T13:34:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.894+0000 7f4845b49b80 20 client.15459 trim_cache size 0 max 16384 2024-05-11T13:34:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 Processor -- start 2024-05-11T13:34:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 -- start start 2024-05-11T13:34:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d5f6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d6023c0 0x55fe6d603cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d615420 con 0x55fe6d601aa0 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d615af0 con 0x55fe6d6023c0 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.926+0000 7f4845b49b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d6161c0 con 0x55fe6d5fc6c0 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d5f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d5f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43408/0 (socket says 172.21.15.63:43408) 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 -- 172.21.15.63:0/3375710951 learned_addr learned my addr 172.21.15.63:0/3375710951 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833e00700 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4839200700 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d6023c0 0x55fe6d603cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 -- 172.21.15.63:0/3375710951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d5fc6c0 msgr2=0x55fe6d5fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 -- 172.21.15.63:0/3375710951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d6023c0 msgr2=0x55fe6d603cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d6023c0 0x55fe6d603cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.927+0000 7f4833400700 1 -- 172.21.15.63:0/3375710951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe6d607c40 con 0x55fe6d601aa0 2024-05-11T13:34:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.928+0000 7f4833400700 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d5f6970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f482000bec0 tx=0x7f4820005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861df5a819629e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.928+0000 7f4828c00700 1 -- 172.21.15.63:0/3375710951 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482000b710 con 0x55fe6d601aa0 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.928+0000 7f4828c00700 1 -- 172.21.15.63:0/3375710951 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48200120d0 con 0x55fe6d601aa0 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.928+0000 7f4828c00700 1 -- 172.21.15.63:0/3375710951 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820003c80 con 0x55fe6d601aa0 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.929+0000 7f4845b49b80 1 -- 172.21.15.63:0/3375710951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 msgr2=0x55fe6d5f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.929+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d5f6970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f482000bec0 tx=0x7f4820005c40 comp rx=0 tx=0).stop 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.929+0000 7f4845b49b80 1 -- 172.21.15.63:0/3375710951 shutdown_connections 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.929+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d6023c0 0x55fe6d603cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.929+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d5fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.929+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3375710951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d5f6970 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.929+0000 7f4845b49b80 1 -- 172.21.15.63:0/3375710951 >> 172.21.15.63:0/3375710951 conn(0x55fe6d52ab10 msgr2=0x55fe6d5f04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:34.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.930+0000 7f4845b49b80 1 -- 172.21.15.63:0/3375710951 shutdown_connections 2024-05-11T13:34:34.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.930+0000 7f4845b49b80 1 -- 172.21.15.63:0/3375710951 wait complete. 2024-05-11T13:34:34.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.930+0000 7f4845b49b80 1 -- start start 2024-05-11T13:34:34.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 1 Processor -- start 2024-05-11T13:34:34.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d620a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d61e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d61ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d616f60 con 0x55fe6d601aa0 2024-05-11T13:34:34.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d616890 con 0x55fe6d5fc6c0 2024-05-11T13:34:34.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fe6d617630 con 0x55fe6d6023c0 2024-05-11T13:34:34.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.931+0000 7f4845b49b80 10 client.0 _ll_register_callbacks cb 0x55fe6d4f5c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d61e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d61e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43410/0 (socket says 172.21.15.63:43410) 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 -- 172.21.15.63:0/3603417153 learned_addr learned my addr 172.21.15.63:0/3603417153 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833e00700 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d620a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4839200700 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d61ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 -- 172.21.15.63:0/3603417153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 msgr2=0x55fe6d61ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d61ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 -- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 msgr2=0x55fe6d620a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d620a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.932+0000 7f4833400700 1 -- 172.21.15.63:0/3603417153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe6d61f2c0 con 0x55fe6d601aa0 2024-05-11T13:34:34.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.933+0000 7f4833400700 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d61e540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f482000bfb0 tx=0x7f4820005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60d9116f12d6d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:34.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.933+0000 7f481e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.933+0000 7f481e000700 1 -- 172.21.15.63:0/3603417153 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482000b780 con 0x55fe6d601aa0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.933+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48200120d0 con 0x55fe6d601aa0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.933+0000 7f481e000700 1 -- 172.21.15.63:0/3603417153 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f4820003680 con 0x55fe6d601aa0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f481e000700 1 -- 172.21.15.63:0/3603417153 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4820033050 con 0x55fe6d601aa0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 10 client.15465 fetch_fsmap learned FSMap version 29 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe6d6810e0 con 0x55fe6d601aa0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f481e000700 1 -- 172.21.15.63:0/3603417153 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48200120d0 con 0x55fe6d601aa0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f481e000700 1 -- 172.21.15.63:0/3603417153 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4820010970 con 0x55fe6d601aa0 2024-05-11T13:34:34.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 29 2024-05-11T13:34:34.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 10 client.15465 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-11T13:34:34.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 20 client.15465 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:34.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d685480 0x55fe6d684190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:34.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 4 client.15465 mds_command: new command op to 34403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:34:34.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.934+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 --> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] -- command(tid 0: {"prefix": "status"}) -- 0x55fe6d55c3b0 con 0x55fe6d685480 2024-05-11T13:34:34.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.935+0000 7f4833e00700 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d685480 0x55fe6d684190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:34.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.937+0000 7f4833e00700 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d685480 0x55fe6d684190 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=346c8291ae2eeba1 server_cookie=f5fa6f4d415ca35a in_seq=0 out_seq=0 2024-05-11T13:34:34.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.937+0000 7f481e000700 10 client.15465 ms_handle_connect on v2:172.21.15.71:6832/1870291573 2024-05-11T13:34:34.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.937+0000 7f481e000700 1 -- 172.21.15.63:0/3603417153 <== mds.3 v2:172.21.15.71:6832/1870291573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fe6d55c3b0 con 0x55fe6d685480 2024-05-11T13:34:34.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.937+0000 7f481e000700 10 client.15465 handle_command_reply: tid=0 2024-05-11T13:34:34.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.937+0000 7f4845b49b80 1 client.15465 shutdown 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 msgr2=0x55fe6d61e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d61e540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f482000bfb0 tx=0x7f4820005380 comp rx=0 tx=0).stop 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 shutdown_connections 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:6832/1870291573,v1:172.21.15.71:6833/1870291573] conn(0x55fe6d685480 0x55fe6d684190 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fe6d6023c0 0x55fe6d61ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe6d5fc6c0 0x55fe6d620a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 --2- 172.21.15.63:0/3603417153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe6d601aa0 0x55fe6d61e540 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 >> 172.21.15.63:0/3603417153 conn(0x55fe6d52ab10 msgr2=0x55fe6d5f6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:34.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f481e000700 0 client.15465 ms_handle_reset on v2:172.21.15.71:6832/1870291573 2024-05-11T13:34:34.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 shutdown_connections 2024-05-11T13:34:34.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.938+0000 7f4845b49b80 1 -- 172.21.15.63:0/3603417153 wait complete. 2024-05-11T13:34:34.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.939+0000 7f4845b49b80 20 client.15465 trim_cache size 0 max 16384 2024-05-11T13:34:34.937 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 3, 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34403, 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 278.19807892099999, 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 29, 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 90, 2024-05-11T13:34:34.938 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 421.12274944299998 2024-05-11T13:34:34.939 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:34:34.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.940+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f48180b2b10 msgr2=0x7f48180b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.940+0000 7f4839c00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f48180b2b10 0x7f48180b4fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f482400bd30 tx=0x7f482400b710 comp rx=0 tx=0).stop 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.940+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4834109570 msgr2=0x7f483410ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.940+0000 7f4839c00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4834109570 0x7f483410ae90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f482c00b830 tx=0x7f482c004d00 comp rx=0 tx=0).stop 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 shutdown_connections 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f48180b2b10 0x7f48180b4fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4834109570 0x7f483410ae90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4834101a00 0x7f483410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 --2- 172.21.15.63:0/4285393381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48341023c0 0x7f483410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 >> 172.21.15.63:0/4285393381 conn(0x7f48340788b0 msgr2=0x7f48340ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 shutdown_connections 2024-05-11T13:34:34.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:34.941+0000 7f4839c00700 1 -- 172.21.15.63:0/4285393381 wait complete. 2024-05-11T13:34:35.314 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:35 smithi165 ceph-mon[33235]: pgmap v24: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-11T13:34:35.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:35 smithi063 ceph-mon[29809]: pgmap v24: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-11T13:34:35.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:35 smithi071 ceph-mon[33293]: pgmap v24: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-11T13:34:35.878 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.e status 2024-05-11T13:34:36.093 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:36 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:36.438 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:36.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:36 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:36.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:36 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:37.237 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:37 smithi063 ceph-mon[29809]: pgmap v25: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-05-11T13:34:37.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:37 smithi071 ceph-mon[33293]: pgmap v25: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-05-11T13:34:37.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:37 smithi165 ceph-mon[33235]: pgmap v25: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-05-11T13:34:39.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 -- 172.21.15.63:0/796004829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701034a0 msgr2=0x7f6f70103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 --2- 172.21.15.63:0/796004829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701034a0 0x7f6f70103920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0099c0 tx=0x7f6f5c009cd0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 -- 172.21.15.63:0/796004829 shutdown_connections 2024-05-11T13:34:39.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 --2- 172.21.15.63:0/796004829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f70103e60 0x7f6f70110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 --2- 172.21.15.63:0/796004829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f70109550 0x7f6f70109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 --2- 172.21.15.63:0/796004829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701034a0 0x7f6f70103920 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 -- 172.21.15.63:0/796004829 >> 172.21.15.63:0/796004829 conn(0x7f6f700fefd0 msgr2=0x7f6f701013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:39.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 -- 172.21.15.63:0/796004829 shutdown_connections 2024-05-11T13:34:39.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.028+0000 7f6f6f200700 1 -- 172.21.15.63:0/796004829 wait complete. 2024-05-11T13:34:39.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 Processor -- start 2024-05-11T13:34:39.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 -- start start 2024-05-11T13:34:39.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f701034a0 0x7f6f7019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f70103e60 0x7f6f7019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f70109550 0x7f6f701997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70113550 con 0x7f6f70103e60 2024-05-11T13:34:39.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701133d0 con 0x7f6f701034a0 2024-05-11T13:34:39.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.030+0000 7f6f6f200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701136d0 con 0x7f6f70109550 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.031+0000 7f6f6d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f70103e60 0x7f6f7019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.031+0000 7f6f6d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f70103e60 0x7f6f7019fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43436/0 (socket says 172.21.15.63:43436) 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.031+0000 7f6f6d400700 1 -- 172.21.15.63:0/865253287 learned_addr learned my addr 172.21.15.63:0/865253287 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.031+0000 7f6f6de00700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f701034a0 0x7f6f7019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.031+0000 7f6f6e800700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f70109550 0x7f6f701997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.032+0000 7f6f6d400700 1 -- 172.21.15.63:0/865253287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f70109550 msgr2=0x7f6f701997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.032+0000 7f6f6d400700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f70109550 0x7f6f701997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.032+0000 7f6f6d400700 1 -- 172.21.15.63:0/865253287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f701034a0 msgr2=0x7f6f7019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.032+0000 7f6f6d400700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f701034a0 0x7f6f7019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.032+0000 7f6f6d400700 1 -- 172.21.15.63:0/865253287 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f70199f90 con 0x7f6f70103e60 2024-05-11T13:34:39.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.032+0000 7f6f6d400700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f70103e60 0x7f6f7019fb70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:39.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.032+0000 7f6f66a00700 1 -- 172.21.15.63:0/865253287 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c027070 con 0x7f6f70103e60 2024-05-11T13:34:39.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.033+0000 7f6f66a00700 1 -- 172.21.15.63:0/865253287 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f5c00fda0 con 0x7f6f70103e60 2024-05-11T13:34:39.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.033+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7019a220 con 0x7f6f70103e60 2024-05-11T13:34:39.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.033+0000 7f6f66a00700 1 -- 172.21.15.63:0/865253287 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c022e50 con 0x7f6f70103e60 2024-05-11T13:34:39.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.033+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f701a6490 con 0x7f6f70103e60 2024-05-11T13:34:39.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.037+0000 7f6f66a00700 1 -- 172.21.15.63:0/865253287 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7f6f5c02a070 con 0x7f6f70103e60 2024-05-11T13:34:39.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f7f662b80 1 Processor -- start 2024-05-11T13:34:39.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f7f662b80 1 -- start start 2024-05-11T13:34:39.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f66a00700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6f540b2d30 0x7f6f540b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee6eb0 0x562bfdee7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 0x562bfdee14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdee1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bfde2f9d0 con 0x562bfdee6eb0 2024-05-11T13:34:39.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.038+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562bfddcd710 con 0x562bfdee77d0 2024-05-11T13:34:39.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562bfde2b990 con 0x562bfdee1ad0 2024-05-11T13:34:39.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f66a00700 1 -- 172.21.15.63:0/865253287 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6f5c0e3b20 con 0x7f6f70103e60 2024-05-11T13:34:39.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f6e800700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6f540b2d30 0x7f6f540b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f6de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee6eb0 0x562bfdee7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f6de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee6eb0 0x562bfdee7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43452/0 (socket says 172.21.15.63:43452) 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f6de00700 1 -- 172.21.15.63:0/2347298377 learned_addr learned my addr 172.21.15.63:0/2347298377 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f6e800700 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdee1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.039+0000 7f6f6d400700 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 0x562bfdee14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6de00700 1 -- 172.21.15.63:0/2347298377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 msgr2=0x562bfdee1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6de00700 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdee1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6de00700 1 -- 172.21.15.63:0/2347298377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 msgr2=0x562bfdee14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6de00700 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 0x562bfdee14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6de00700 1 -- 172.21.15.63:0/2347298377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfdd7c880 con 0x562bfdee6eb0 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6e800700 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdee1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6e800700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6f540b2d30 0x7f6f540b51f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f68012820 tx=0x7f6f68012be0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:39.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.040+0000 7f6f6de00700 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee6eb0 0x562bfdee7290 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b780 tx=0x7f6f6000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92922ce22b3b0137 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:39.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.041+0000 7f6f5b400700 1 -- 172.21.15.63:0/2347298377 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60003a80 con 0x562bfdee6eb0 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.041+0000 7f6f5b400700 1 -- 172.21.15.63:0/2347298377 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f60015070 con 0x562bfdee6eb0 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.041+0000 7f6f5b400700 1 -- 172.21.15.63:0/2347298377 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600053a0 con 0x562bfdee6eb0 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2347298377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee6eb0 msgr2=0x562bfdee7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee6eb0 0x562bfdee7290 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b780 tx=0x7f6f6000ba90 comp rx=0 tx=0).stop 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2347298377 shutdown_connections 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdee1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee6eb0 0x562bfdee7290 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2347298377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 0x562bfdee14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2347298377 >> 172.21.15.63:0/2347298377 conn(0x562bfde0bb10 msgr2=0x562bfded7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2347298377 shutdown_connections 2024-05-11T13:34:39.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.042+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2347298377 wait complete. 2024-05-11T13:34:39.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.043+0000 7f6f7f662b80 1 -- start start 2024-05-11T13:34:39.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.043+0000 7f6f7f662b80 1 Processor -- start 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdeed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 0x562bfdeea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdeeaea0 0x562bfdeeb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bfdeee280 con 0x562bfdeeaea0 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562bfdd95ad0 con 0x562bfdee77d0 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562bfdeee400 con 0x562bfdee1ad0 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f7f662b80 10 client.0 _ll_register_callbacks cb 0x562bfddd6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f6e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdeeaea0 0x562bfdeeb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f6e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdeeaea0 0x562bfdeeb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43468/0 (socket says 172.21.15.63:43468) 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f6e800700 1 -- 172.21.15.63:0/369544774 learned_addr learned my addr 172.21.15.63:0/369544774 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.044+0000 7f6f6de00700 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdeed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f6e800700 1 -- 172.21.15.63:0/369544774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 msgr2=0x562bfdeed660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f6e800700 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdeed660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f6e800700 1 -- 172.21.15.63:0/369544774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 msgr2=0x562bfdeea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f6e800700 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 0x562bfdeea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f6e800700 1 -- 172.21.15.63:0/369544774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfddf53d0 con 0x562bfdeeaea0 2024-05-11T13:34:39.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f6e800700 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdeeaea0 0x562bfdeeb320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6f68015cd0 tx=0x7f6f68018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a7872f329ab912 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:39.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f4fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:39.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.045+0000 7f6f4fe00700 1 -- 172.21.15.63:0/369544774 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68018e10 con 0x562bfdeeaea0 2024-05-11T13:34:39.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.046+0000 7f6f4fe00700 1 -- 172.21.15.63:0/369544774 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6f680223f0 con 0x562bfdeeaea0 2024-05-11T13:34:39.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.046+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f60015070 con 0x562bfdeeaea0 2024-05-11T13:34:39.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.046+0000 7f6f4fe00700 1 -- 172.21.15.63:0/369544774 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6f68030050 con 0x562bfdeeaea0 2024-05-11T13:34:39.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.046+0000 7f6f7f662b80 10 client.15477 fetch_fsmap learned FSMap version 30 2024-05-11T13:34:39.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.046+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bfdef9000 con 0x562bfdeeaea0 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f4fe00700 1 -- 172.21.15.63:0/369544774 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f60015070 con 0x562bfdeeaea0 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f4fe00700 1 -- 172.21.15.63:0/369544774 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6f68016040 con 0x562bfdeeaea0 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f7f662b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 30 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f7f662b80 10 client.15477 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f7f662b80 20 client.15477 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdefb700 0x562bfdefcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f7f662b80 4 client.15477 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.047+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 --> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bfdeaa160 con 0x562bfdefb700 2024-05-11T13:34:39.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.048+0000 7f6f6de00700 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdefb700 0x562bfdefcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.050+0000 7f6f6de00700 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdefb700 0x562bfdefcb70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ae0b51d21a384c9 server_cookie=b6b77ef037028ba2 in_seq=0 out_seq=0 2024-05-11T13:34:39.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.050+0000 7f6f4fe00700 10 client.15477 ms_handle_connect on v2:172.21.15.71:6834/1316489545 2024-05-11T13:34:39.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.052+0000 7f6f4fe00700 1 -- 172.21.15.63:0/369544774 <== mds.2 v2:172.21.15.71:6834/1316489545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562bfdeaa160 con 0x562bfdefb700 2024-05-11T13:34:39.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.052+0000 7f6f4fe00700 10 client.15477 handle_command_reply: tid=0 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.052+0000 7f6f7f662b80 1 client.15477 shutdown 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdeeaea0 msgr2=0x562bfdeeb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdeeaea0 0x562bfdeeb320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6f68015cd0 tx=0x7f6f68018000 comp rx=0 tx=0).stop 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 shutdown_connections 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdefb700 0x562bfdefcb70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdeeaea0 0x562bfdeeb320 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee1ad0 0x562bfdeed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/369544774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee77d0 0x562bfdeea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 >> 172.21.15.63:0/369544774 conn(0x562bfde0bb10 msgr2=0x562bfded2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:39.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f4fe00700 0 client.15477 ms_handle_reset on v2:172.21.15.71:6834/1316489545 2024-05-11T13:34:39.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.053+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 shutdown_connections 2024-05-11T13:34:39.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.054+0000 7f6f7f662b80 1 -- 172.21.15.63:0/369544774 wait complete. 2024-05-11T13:34:39.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.054+0000 7f6f7f662b80 20 client.15477 trim_cache size 0 max 16384 2024-05-11T13:34:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 Processor -- start 2024-05-11T13:34:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 -- start start 2024-05-11T13:34:39.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee1ad0 0x562bfdee49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee77d0 0x562bfdee4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdeeaea0 0x562bfdee7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bfdefaeb0 con 0x562bfdee77d0 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562bfdefb580 con 0x562bfdee1ad0 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.084+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562bfdefbc50 con 0x562bfdeeaea0 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee77d0 0x562bfdee4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee77d0 0x562bfdee4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43472/0 (socket says 172.21.15.63:43472) 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 -- 172.21.15.63:0/3608305605 learned_addr learned my addr 172.21.15.63:0/3608305605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6de00700 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee1ad0 0x562bfdee49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6e800700 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdeeaea0 0x562bfdee7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 -- 172.21.15.63:0/3608305605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdeeaea0 msgr2=0x562bfdee7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdeeaea0 0x562bfdee7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 -- 172.21.15.63:0/3608305605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee1ad0 msgr2=0x562bfdee49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee1ad0 0x562bfdee49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6d400700 1 -- 172.21.15.63:0/3608305605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfdefb3a0 con 0x562bfdee77d0 2024-05-11T13:34:39.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.085+0000 7f6f6de00700 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee1ad0 0x562bfdee49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:39.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.086+0000 7f6f6d400700 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee77d0 0x562bfdee4f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0a7b80 tx=0x7f6f5c022590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3fef0201cad3dc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:39.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.086+0000 7f6f5aa00700 1 -- 172.21.15.63:0/3608305605 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0a60a0 con 0x562bfdee77d0 2024-05-11T13:34:39.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.086+0000 7f6f5aa00700 1 -- 172.21.15.63:0/3608305605 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f5c0afe50 con 0x562bfdee77d0 2024-05-11T13:34:39.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.086+0000 7f6f5aa00700 1 -- 172.21.15.63:0/3608305605 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0b36e0 con 0x562bfdee77d0 2024-05-11T13:34:39.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 -- 172.21.15.63:0/3608305605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee77d0 msgr2=0x562bfdee4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee77d0 0x562bfdee4f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0a7b80 tx=0x7f6f5c022590 comp rx=0 tx=0).stop 2024-05-11T13:34:39.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 -- 172.21.15.63:0/3608305605 shutdown_connections 2024-05-11T13:34:39.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdeeaea0 0x562bfdee7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdee1ad0 0x562bfdee49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/3608305605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee77d0 0x562bfdee4f00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 -- 172.21.15.63:0/3608305605 >> 172.21.15.63:0/3608305605 conn(0x562bfde0bb10 msgr2=0x562bfded14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:39.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 -- 172.21.15.63:0/3608305605 shutdown_connections 2024-05-11T13:34:39.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.087+0000 7f6f7f662b80 1 -- 172.21.15.63:0/3608305605 wait complete. 2024-05-11T13:34:39.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 -- start start 2024-05-11T13:34:39.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 Processor -- start 2024-05-11T13:34:39.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee1ad0 0x562bfdee5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee77d0 0x562bfdee5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdeeaea0 0x562bfdee6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bfdefc9f0 con 0x562bfdee1ad0 2024-05-11T13:34:39.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562bfdefc320 con 0x562bfdeeaea0 2024-05-11T13:34:39.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.088+0000 7f6f7f662b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562bfdefd0c0 con 0x562bfdee77d0 2024-05-11T13:34:39.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f7f662b80 10 client.0 _ll_register_callbacks cb 0x562bfddd6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:39.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f6de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee1ad0 0x562bfdee5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f6de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee1ad0 0x562bfdee5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43474/0 (socket says 172.21.15.63:43474) 2024-05-11T13:34:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f6de00700 1 -- 172.21.15.63:0/2826934993 learned_addr learned my addr 172.21.15.63:0/2826934993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f6e800700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdeeaea0 0x562bfdee6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f6d400700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee77d0 0x562bfdee5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f6de00700 1 -- 172.21.15.63:0/2826934993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee77d0 msgr2=0x562bfdee5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.089+0000 7f6f6de00700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee77d0 0x562bfdee5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f6de00700 1 -- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdeeaea0 msgr2=0x562bfdee6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f6de00700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdeeaea0 0x562bfdee6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f6de00700 1 -- 172.21.15.63:0/2826934993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfdf04500 con 0x562bfdee1ad0 2024-05-11T13:34:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f6d400700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee77d0 0x562bfdee5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:34:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f6de00700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee1ad0 0x562bfdee5aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6f60001eb0 tx=0x7f6f600058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e1c14cfb2da2df server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f4fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f4fe00700 1 -- 172.21.15.63:0/2826934993 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60029020 con 0x562bfdee1ad0 2024-05-11T13:34:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f4fe00700 1 -- 172.21.15.63:0/2826934993 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f6f60025430 con 0x562bfdee1ad0 2024-05-11T13:34:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.090+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5c0afe50 con 0x562bfdee1ad0 2024-05-11T13:34:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.091+0000 7f6f4fe00700 1 -- 172.21.15.63:0/2826934993 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6f60034050 con 0x562bfdee1ad0 2024-05-11T13:34:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.091+0000 7f6f7f662b80 10 client.15483 fetch_fsmap learned FSMap version 30 2024-05-11T13:34:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.091+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bfdf66fb0 con 0x562bfdee1ad0 2024-05-11T13:34:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.092+0000 7f6f4fe00700 1 -- 172.21.15.63:0/2826934993 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f5c0afe50 con 0x562bfdee1ad0 2024-05-11T13:34:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.092+0000 7f6f4fe00700 1 -- 172.21.15.63:0/2826934993 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6f60005550 con 0x562bfdee1ad0 2024-05-11T13:34:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.092+0000 7f6f7f662b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 30 2024-05-11T13:34:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.092+0000 7f6f7f662b80 10 client.15483 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-11T13:34:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.092+0000 7f6f7f662b80 20 client.15483 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.092+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdf6b8b0 0x562bfdf6a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.092+0000 7f6f7f662b80 4 client.15483 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:34:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.093+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 --> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] -- command(tid 0: {"prefix": "status"}) -- 0x562bfde3d3b0 con 0x562bfdf6b8b0 2024-05-11T13:34:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.093+0000 7f6f6d400700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdf6b8b0 0x562bfdf6a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.094+0000 7f6f6d400700 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdf6b8b0 0x562bfdf6a590 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14cb69a919de5cd5 server_cookie=65ae338988bbf2ca in_seq=0 out_seq=0 2024-05-11T13:34:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.094+0000 7f6f4fe00700 10 client.15483 ms_handle_connect on v2:172.21.15.71:6834/1316489545 2024-05-11T13:34:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f4fe00700 1 -- 172.21.15.63:0/2826934993 <== mds.2 v2:172.21.15.71:6834/1316489545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562bfde3d3b0 con 0x562bfdf6b8b0 2024-05-11T13:34:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f4fe00700 10 client.15483 handle_command_reply: tid=0 2024-05-11T13:34:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 client.15483 shutdown 2024-05-11T13:34:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee1ad0 msgr2=0x562bfdee5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee1ad0 0x562bfdee5aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6f60001eb0 tx=0x7f6f600058e0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 shutdown_connections 2024-05-11T13:34:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bfdeeaea0 0x562bfdee6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.71:6834/1316489545,v1:172.21.15.71:6835/1316489545] conn(0x562bfdf6b8b0 0x562bfdf6a590 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfdee1ad0 0x562bfdee5aa0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f4fe00700 0 client.15483 ms_handle_reset on v2:172.21.15.71:6834/1316489545 2024-05-11T13:34:39.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 --2- 172.21.15.63:0/2826934993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562bfdee77d0 0x562bfdee5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.096+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 >> 172.21.15.63:0/2826934993 conn(0x562bfde0bb10 msgr2=0x562bfdee91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:39.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.097+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 shutdown_connections 2024-05-11T13:34:39.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.097+0000 7f6f7f662b80 1 -- 172.21.15.63:0/2826934993 wait complete. 2024-05-11T13:34:39.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.097+0000 7f6f7f662b80 20 client.15483 trim_cache size 0 max 16384 2024-05-11T13:34:39.105 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:34:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:34:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 2, 2024-05-11T13:34:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34427, 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 283.36995371099999, 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 30, 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 91, 2024-05-11T13:34:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 409.89301222099999 2024-05-11T13:34:39.107 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:34:39.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6f540b2d30 msgr2=0x7f6f540b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6f540b2d30 0x7f6f540b51f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f68012820 tx=0x7f6f68012be0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f70103e60 msgr2=0x7f6f7019fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f70103e60 0x7f6f7019fb70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c00f460 comp rx=0 tx=0).stop 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 shutdown_connections 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6f540b2d30 0x7f6f540b51f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f70109550 0x7f6f701997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f701034a0 0x7f6f7019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 --2- 172.21.15.63:0/865253287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f70103e60 0x7f6f7019fb70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.098+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 >> 172.21.15.63:0/865253287 conn(0x7f6f700fefd0 msgr2=0x7f6f70100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.099+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 shutdown_connections 2024-05-11T13:34:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:39.099+0000 7f6f6f200700 1 -- 172.21.15.63:0/865253287 wait complete. 2024-05-11T13:34:39.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:39 smithi063 ceph-mon[29809]: pgmap v26: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-05-11T13:34:39.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:39 smithi071 ceph-mon[33293]: pgmap v26: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-05-11T13:34:39.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:39 smithi165 ceph-mon[33235]: pgmap v26: 129 pgs: 129 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-05-11T13:34:40.099 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-11T13:34:40.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:40 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:40.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:40 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:40.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:40 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:40.607 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:41.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:41 smithi063 ceph-mon[29809]: pgmap v27: 129 pgs: 129 active+clean; 23 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 47 op/s 2024-05-11T13:34:41.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:41 smithi071 ceph-mon[33293]: pgmap v27: 129 pgs: 129 active+clean; 23 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 47 op/s 2024-05-11T13:34:41.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:41 smithi165 ceph-mon[33235]: pgmap v27: 129 pgs: 129 active+clean; 23 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 47 op/s 2024-05-11T13:34:42.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.977+0000 7f9da8000700 1 -- 172.21.15.63:0/1101436184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 msgr2=0x7f9da01123a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:42.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.977+0000 7f9da8000700 1 --2- 172.21.15.63:0/1101436184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01123a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800b0a0 tx=0x7f9d9800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.977+0000 7f9da8000700 1 -- 172.21.15.63:0/1101436184 shutdown_connections 2024-05-11T13:34:42.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.977+0000 7f9da8000700 1 --2- 172.21.15.63:0/1101436184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01123a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.977+0000 7f9da8000700 1 --2- 172.21.15.63:0/1101436184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da0075780 0x7f9da0075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.977+0000 7f9da8000700 1 --2- 172.21.15.63:0/1101436184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da00760a0 0x7f9da010fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.977+0000 7f9da8000700 1 -- 172.21.15.63:0/1101436184 >> 172.21.15.63:0/1101436184 conn(0x7f9da007a6b0 msgr2=0x7f9da007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:42.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.978+0000 7f9da8000700 1 -- 172.21.15.63:0/1101436184 shutdown_connections 2024-05-11T13:34:42.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.978+0000 7f9da8000700 1 -- 172.21.15.63:0/1101436184 wait complete. 2024-05-11T13:34:42.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 Processor -- start 2024-05-11T13:34:42.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 -- start start 2024-05-11T13:34:42.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da0075780 0x7f9da019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da00760a0 0x7f9da019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0115920 con 0x7f9da010ffb0 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01157a0 con 0x7f9da0075780 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.979+0000 7f9da8000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0115aa0 con 0x7f9da00760a0 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da4c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da00760a0 0x7f9da019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da5600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da0075780 0x7f9da019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51798/0 (socket says 172.21.15.63:51798) 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 -- 172.21.15.63:0/2908783938 learned_addr learned my addr 172.21.15.63:0/2908783938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 -- 172.21.15.63:0/2908783938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da00760a0 msgr2=0x7f9da019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da00760a0 0x7f9da019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 -- 172.21.15.63:0/2908783938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da0075780 msgr2=0x7f9da019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da0075780 0x7f9da019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.980+0000 7f9da6000700 1 -- 172.21.15.63:0/2908783938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da01a0f20 con 0x7f9da010ffb0 2024-05-11T13:34:42.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.981+0000 7f9da6000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01a07e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800b070 tx=0x7f9d98004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:42.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.981+0000 7f9d9e000700 1 -- 172.21.15.63:0/2908783938 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98021070 con 0x7f9da010ffb0 2024-05-11T13:34:42.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.981+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da01a11b0 con 0x7f9da010ffb0 2024-05-11T13:34:42.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.981+0000 7f9d9e000700 1 -- 172.21.15.63:0/2908783938 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d98007b80 con 0x7f9da010ffb0 2024-05-11T13:34:42.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.981+0000 7f9d9e000700 1 -- 172.21.15.63:0/2908783938 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980046e0 con 0x7f9da010ffb0 2024-05-11T13:34:42.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.981+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da01a8a90 con 0x7f9da010ffb0 2024-05-11T13:34:42.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.985+0000 7f9d9e000700 1 -- 172.21.15.63:0/2908783938 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7f9d98003a00 con 0x7f9da010ffb0 2024-05-11T13:34:42.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.986+0000 7f9d9e000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9d880b2b90 0x7f9d880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.987+0000 7f9d9e000700 1 -- 172.21.15.63:0/2908783938 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f9d9800f020 con 0x7f9da010ffb0 2024-05-11T13:34:42.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.987+0000 7f9da5600700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9d880b2b90 0x7f9d880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.987+0000 7f9db2a0bb80 1 Processor -- start 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.987+0000 7f9db2a0bb80 1 -- start start 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9da5600700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9d880b2b90 0x7f9d880b5050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009a10 tx=0x7f9d94009190 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc49010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc43300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc43d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5577efb959d0 con 0x5577efc49550 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5577efb33710 con 0x5577efc48c30 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5577efb91990 con 0x5577efc43920 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9da4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc43300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9da4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc43300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51808/0 (socket says 172.21.15.63:51808) 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9da4c00700 1 -- 172.21.15.63:0/709406973 learned_addr learned my addr 172.21.15.63:0/709406973 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:42.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.988+0000 7f9da5600700 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc49010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.989+0000 7f9da6000700 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc43d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.989+0000 7f9da4c00700 1 -- 172.21.15.63:0/709406973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 msgr2=0x5577efc43d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:42.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.989+0000 7f9da4c00700 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc43d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.989+0000 7f9da4c00700 1 -- 172.21.15.63:0/709406973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 msgr2=0x5577efc49010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:42.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.989+0000 7f9da4c00700 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc49010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.989+0000 7f9da4c00700 1 -- 172.21.15.63:0/709406973 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577efae2880 con 0x5577efc49550 2024-05-11T13:34:42.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.989+0000 7f9da4c00700 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc43300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0077f0 tx=0x7f9d8c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e218b39dff35a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:42.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.990+0000 7f9d92a00700 1 -- 172.21.15.63:0/709406973 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0125f0 con 0x5577efc49550 2024-05-11T13:34:42.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.990+0000 7f9d92a00700 1 -- 172.21.15.63:0/709406973 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d8c018070 con 0x5577efc49550 2024-05-11T13:34:42.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.990+0000 7f9d92a00700 1 -- 172.21.15.63:0/709406973 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c013c40 con 0x5577efc49550 2024-05-11T13:34:42.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.990+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/709406973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 msgr2=0x5577efc43300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.990+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc43300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0077f0 tx=0x7f9d8c007b00 comp rx=0 tx=0).stop 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/709406973 shutdown_connections 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc43d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc49010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/709406973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc43300 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/709406973 >> 172.21.15.63:0/709406973 conn(0x5577efb71b10 msgr2=0x5577efc3dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/709406973 shutdown_connections 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/709406973 wait complete. 2024-05-11T13:34:42.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.991+0000 7f9db2a0bb80 1 -- start start 2024-05-11T13:34:42.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.992+0000 7f9db2a0bb80 1 Processor -- start 2024-05-11T13:34:42.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc4f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc4c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc4cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5577efc4ffc0 con 0x5577efc49550 2024-05-11T13:34:42.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5577efafbad0 con 0x5577efc48c30 2024-05-11T13:34:42.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5577efc50140 con 0x5577efc43920 2024-05-11T13:34:42.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9db2a0bb80 10 client.0 _ll_register_callbacks cb 0x5577efb29330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:42.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9da6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc4cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9da6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc4cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51812/0 (socket says 172.21.15.63:51812) 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9da6000700 1 -- 172.21.15.63:0/1241673004 learned_addr learned my addr 172.21.15.63:0/1241673004 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.993+0000 7f9da5600700 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc4f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.994+0000 7f9da6000700 1 -- 172.21.15.63:0/1241673004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 msgr2=0x5577efc4f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.994+0000 7f9da6000700 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc4f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.994+0000 7f9da6000700 1 -- 172.21.15.63:0/1241673004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 msgr2=0x5577efc4c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.994+0000 7f9da6000700 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc4c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.994+0000 7f9da6000700 1 -- 172.21.15.63:0/1241673004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577efb5b3d0 con 0x5577efc49550 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.994+0000 7f9da6000700 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc4cbc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006070 tx=0x7f9d980aaac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0a104a553f3580 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.994+0000 7f9d87400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:42.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.995+0000 7f9d87400700 1 -- 172.21.15.63:0/1241673004 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980aaf00 con 0x5577efc49550 2024-05-11T13:34:42.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.995+0000 7f9d87400700 1 -- 172.21.15.63:0/1241673004 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f9d980ab160 con 0x5577efc49550 2024-05-11T13:34:42.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.996+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c018070 con 0x5577efc49550 2024-05-11T13:34:42.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.997+0000 7f9d87400700 1 -- 172.21.15.63:0/1241673004 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9d9806f050 con 0x5577efc49550 2024-05-11T13:34:42.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.997+0000 7f9db2a0bb80 10 client.15495 fetch_fsmap learned FSMap version 31 2024-05-11T13:34:42.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.997+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5577efc5a870 con 0x5577efc49550 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.997+0000 7f9d87400700 1 -- 172.21.15.63:0/1241673004 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d8c018070 con 0x5577efc49550 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.998+0000 7f9d87400700 1 -- 172.21.15.63:0/1241673004 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9d98011070 con 0x5577efc49550 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.998+0000 7f9db2a0bb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 31 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.998+0000 7f9db2a0bb80 10 client.15495 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.998+0000 7f9db2a0bb80 20 client.15495 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.998+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efc5cf70 0x5577efc5e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.998+0000 7f9db2a0bb80 4 client.15495 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:34:42.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.998+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 --> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577efc10010 con 0x5577efc5cf70 2024-05-11T13:34:42.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:42.999+0000 7f9da5600700 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efc5cf70 0x5577efc5e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:42.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.001+0000 7f9da5600700 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efc5cf70 0x5577efc5e3e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8daac23354f6c1a server_cookie=f4db1f97e885b73c in_seq=0 out_seq=0 2024-05-11T13:34:42.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.001+0000 7f9d87400700 10 client.15495 ms_handle_connect on v2:172.21.15.71:6836/136473158 2024-05-11T13:34:43.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.003+0000 7f9d87400700 1 -- 172.21.15.63:0/1241673004 <== mds.1 v2:172.21.15.71:6836/136473158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577efc10010 con 0x5577efc5cf70 2024-05-11T13:34:43.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.003+0000 7f9d87400700 10 client.15495 handle_command_reply: tid=0 2024-05-11T13:34:43.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.003+0000 7f9db2a0bb80 1 client.15495 shutdown 2024-05-11T13:34:43.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 msgr2=0x5577efc4cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:43.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc4cbc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006070 tx=0x7f9d980aaac0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 shutdown_connections 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc4cbc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc4f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efc5cf70 0x5577efc5e3e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/1241673004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc4c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 >> 172.21.15.63:0/1241673004 conn(0x5577efb71b10 msgr2=0x5577efc386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9d87400700 0 client.15495 ms_handle_reset on v2:172.21.15.71:6836/136473158 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 shutdown_connections 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/1241673004 wait complete. 2024-05-11T13:34:43.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.004+0000 7f9db2a0bb80 20 client.15495 trim_cache size 0 max 16384 2024-05-11T13:34:43.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 Processor -- start 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 -- start start 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc46740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc49930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5577efc5c720 con 0x5577efc49550 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5577efc5cdf0 con 0x5577efc48c30 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.030+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5577efc5d4c0 con 0x5577efc43920 2024-05-11T13:34:43.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc49930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:43.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc49930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51814/0 (socket says 172.21.15.63:51814) 2024-05-11T13:34:43.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 -- 172.21.15.63:0/3730359792 learned_addr learned my addr 172.21.15.63:0/3730359792 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:43.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da5600700 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc46740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:43.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da4c00700 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:43.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 -- 172.21.15.63:0/3730359792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 msgr2=0x5577efc46740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:43.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc46740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 -- 172.21.15.63:0/3730359792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 msgr2=0x5577efc46c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc46c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.031+0000 7f9da6000700 1 -- 172.21.15.63:0/3730359792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577efc5cc10 con 0x5577efc49550 2024-05-11T13:34:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.032+0000 7f9da6000700 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc49930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9d980a6f80 tx=0x7f9d98009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661267f5ec86a00e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.032+0000 7f9d92000700 1 -- 172.21.15.63:0/3730359792 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980af2b0 con 0x5577efc49550 2024-05-11T13:34:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.032+0000 7f9d92000700 1 -- 172.21.15.63:0/3730359792 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d980b3cd0 con 0x5577efc49550 2024-05-11T13:34:43.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.032+0000 7f9d92000700 1 -- 172.21.15.63:0/3730359792 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980a6870 con 0x5577efc49550 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/3730359792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 msgr2=0x5577efc49930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc49930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9d980a6f80 tx=0x7f9d98009c60 comp rx=0 tx=0).stop 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/3730359792 shutdown_connections 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc49550 0x5577efc49930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc43920 0x5577efc46740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/3730359792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc46c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/3730359792 >> 172.21.15.63:0/3730359792 conn(0x5577efb71b10 msgr2=0x5577efc4afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/3730359792 shutdown_connections 2024-05-11T13:34:43.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.033+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/3730359792 wait complete. 2024-05-11T13:34:43.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 -- start start 2024-05-11T13:34:43.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 Processor -- start 2024-05-11T13:34:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc43920 0x5577efc426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc42c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc49550 0x5577efc431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5577efc5e260 con 0x5577efc43920 2024-05-11T13:34:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5577efc5db90 con 0x5577efc48c30 2024-05-11T13:34:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.034+0000 7f9db2a0bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5577efc5e930 con 0x5577efc49550 2024-05-11T13:34:43.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9db2a0bb80 10 client.0 _ll_register_callbacks cb 0x5577efb29330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc43920 0x5577efc426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc43920 0x5577efc426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51820/0 (socket says 172.21.15.63:51820) 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 -- 172.21.15.63:0/2584447835 learned_addr learned my addr 172.21.15.63:0/2584447835 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 -- 172.21.15.63:0/2584447835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc49550 msgr2=0x5577efc431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc49550 0x5577efc431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 -- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 msgr2=0x5577efc42c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da4c00700 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc42c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc42c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da5600700 1 -- 172.21.15.63:0/2584447835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577efc66100 con 0x5577efc43920 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.035+0000 7f9da4c00700 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9da5600700 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc43920 0x5577efc426f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e4b0 tx=0x7f9d94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa754edb6d46f0fe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9d87400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9d87400700 1 -- 172.21.15.63:0/2584447835 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9402e070 con 0x5577efc43920 2024-05-11T13:34:43.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9d87400700 1 -- 172.21.15.63:0/2584447835 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f9d940293e0 con 0x5577efc43920 2024-05-11T13:34:43.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d980b3cd0 con 0x5577efc43920 2024-05-11T13:34:43.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9d87400700 1 -- 172.21.15.63:0/2584447835 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9d94036050 con 0x5577efc43920 2024-05-11T13:34:43.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9db2a0bb80 10 client.15501 fetch_fsmap learned FSMap version 31 2024-05-11T13:34:43.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5577efcc8a70 con 0x5577efc43920 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.036+0000 7f9d87400700 1 -- 172.21.15.63:0/2584447835 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d980b3cd0 con 0x5577efc43920 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.037+0000 7f9d87400700 1 -- 172.21.15.63:0/2584447835 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9d94031030 con 0x5577efc43920 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.037+0000 7f9db2a0bb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 31 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.037+0000 7f9db2a0bb80 10 client.15501 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.037+0000 7f9db2a0bb80 20 client.15501 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.037+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efccd810 0x5577efccc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.037+0000 7f9db2a0bb80 4 client.15501 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.037+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 --> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] -- command(tid 0: {"prefix": "status"}) -- 0x5577efba33b0 con 0x5577efccd810 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.038+0000 7f9da4c00700 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efccd810 0x5577efccc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.039+0000 7f9da4c00700 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efccd810 0x5577efccc460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=246eeb0dca588312 server_cookie=6995d222afd81a05 in_seq=0 out_seq=0 2024-05-11T13:34:43.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.039+0000 7f9d87400700 10 client.15501 ms_handle_connect on v2:172.21.15.71:6836/136473158 2024-05-11T13:34:43.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.039+0000 7f9d87400700 1 -- 172.21.15.63:0/2584447835 <== mds.1 v2:172.21.15.71:6836/136473158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5577efba33b0 con 0x5577efccd810 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.039+0000 7f9d87400700 10 client.15501 handle_command_reply: tid=0 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.039+0000 7f9db2a0bb80 1 client.15501 shutdown 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc43920 msgr2=0x5577efc426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc43920 0x5577efc426f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e4b0 tx=0x7f9d94011000 comp rx=0 tx=0).stop 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 shutdown_connections 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577efc49550 0x5577efc431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577efc43920 0x5577efc426f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:6836/136473158,v1:172.21.15.71:6837/136473158] conn(0x5577efccd810 0x5577efccc460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 --2- 172.21.15.63:0/2584447835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577efc48c30 0x5577efc42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 >> 172.21.15.63:0/2584447835 conn(0x5577efb71b10 msgr2=0x5577efc4af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9d87400700 0 client.15501 ms_handle_reset on v2:172.21.15.71:6836/136473158 2024-05-11T13:34:43.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 shutdown_connections 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 1 -- 172.21.15.63:0/2584447835 wait complete. 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.040+0000 7f9db2a0bb80 20 client.15501 trim_cache size 0 max 16384 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 1, 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34451, 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 288.33810252199999, 2024-05-11T13:34:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 31, 2024-05-11T13:34:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-11T13:34:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 91, 2024-05-11T13:34:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 401.60086740000003 2024-05-11T13:34:43.041 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:34:43.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9d880b2b90 msgr2=0x7f9d880b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:43.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9d880b2b90 0x7f9d880b5050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009a10 tx=0x7f9d94009190 comp rx=0 tx=0).stop 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 msgr2=0x7f9da01a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01a07e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800b070 tx=0x7f9d98004dc0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 shutdown_connections 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9d880b2b90 0x7f9d880b5050 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9da010ffb0 0x7f9da01a07e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da0075780 0x7f9da019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 --2- 172.21.15.63:0/2908783938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da00760a0 0x7f9da019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.041+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 >> 172.21.15.63:0/2908783938 conn(0x7f9da007a6b0 msgr2=0x7f9da0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.042+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 shutdown_connections 2024-05-11T13:34:43.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:43.042+0000 7f9da8000700 1 -- 172.21.15.63:0/2908783938 wait complete. 2024-05-11T13:34:43.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:43 smithi063 ceph-mon[29809]: pgmap v28: 129 pgs: 129 active+clean; 23 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2024-05-11T13:34:43.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:43 smithi071 ceph-mon[33293]: pgmap v28: 129 pgs: 129 active+clean; 23 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2024-05-11T13:34:43.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:43 smithi165 ceph-mon[33235]: pgmap v28: 129 pgs: 129 active+clean; 23 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2024-05-11T13:34:44.066 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-11T13:34:44.584 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:34:45.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:45 smithi063 ceph-mon[29809]: pgmap v29: 129 pgs: 129 active+clean; 28 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-05-11T13:34:45.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:45 smithi071 ceph-mon[33293]: pgmap v29: 129 pgs: 129 active+clean; 28 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-05-11T13:34:45.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:45 smithi165 ceph-mon[33235]: pgmap v29: 129 pgs: 129 active+clean; 28 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-05-11T13:34:47.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.118+0000 7f61f5400700 1 -- 172.21.15.63:0/1266938327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0109570 msgr2=0x7f61f0109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.118+0000 7f61f5400700 1 --2- 172.21.15.63:0/1266938327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0109570 0x7f61f0109950 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009990 tx=0x7f61e0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.119+0000 7f61f5400700 1 -- 172.21.15.63:0/1266938327 shutdown_connections 2024-05-11T13:34:47.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.119+0000 7f61f5400700 1 --2- 172.21.15.63:0/1266938327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f0102360 0x7f61f0110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.119+0000 7f61f5400700 1 --2- 172.21.15.63:0/1266938327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0109570 0x7f61f0109950 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.119+0000 7f61f5400700 1 --2- 172.21.15.63:0/1266938327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61f01019a0 0x7f61f0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.119+0000 7f61f5400700 1 -- 172.21.15.63:0/1266938327 >> 172.21.15.63:0/1266938327 conn(0x7f61f00788b0 msgr2=0x7f61f00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:47.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.119+0000 7f61f5400700 1 -- 172.21.15.63:0/1266938327 shutdown_connections 2024-05-11T13:34:47.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.119+0000 7f61f5400700 1 -- 172.21.15.63:0/1266938327 wait complete. 2024-05-11T13:34:47.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.120+0000 7f61f5400700 1 Processor -- start 2024-05-11T13:34:47.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.120+0000 7f61f5400700 1 -- start start 2024-05-11T13:34:47.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61f5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f01019a0 0x7f61f0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61f5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0102360 0x7f61f01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61f5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61f0109570 0x7f61f0103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61f5400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01135a0 con 0x7f61f0102360 2024-05-11T13:34:47.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61f5400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0113420 con 0x7f61f0109570 2024-05-11T13:34:47.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61f5400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0113720 con 0x7f61f01019a0 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0102360 0x7f61f01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0102360 0x7f61f01030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51844/0 (socket says 172.21.15.63:51844) 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61ef400700 1 -- 172.21.15.63:0/2404565603 learned_addr learned my addr 172.21.15.63:0/2404565603 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61efe00700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f01019a0 0x7f61f0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.121+0000 7f61f4a00700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61f0109570 0x7f61f0103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ef400700 1 -- 172.21.15.63:0/2404565603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f01019a0 msgr2=0x7f61f0102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ef400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f01019a0 0x7f61f0102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ef400700 1 -- 172.21.15.63:0/2404565603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61f0109570 msgr2=0x7f61f0103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ef400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61f0109570 0x7f61f0103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ef400700 1 -- 172.21.15.63:0/2404565603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f01a8080 con 0x7f61f0102360 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61f4a00700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61f0109570 0x7f61f0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ef400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0102360 0x7f61f01030c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00d980 tx=0x7f61dc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ecc00700 1 -- 172.21.15.63:0/2404565603 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc016070 con 0x7f61f0102360 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ecc00700 1 -- 172.21.15.63:0/2404565603 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61dc004020 con 0x7f61f0102360 2024-05-11T13:34:47.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.122+0000 7f61ecc00700 1 -- 172.21.15.63:0/2404565603 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc004bb0 con 0x7f61f0102360 2024-05-11T13:34:47.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.123+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f01a8370 con 0x7f61f0102360 2024-05-11T13:34:47.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.123+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f01a8830 con 0x7f61f0102360 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 Processor -- start 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 -- start start 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4dd9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4dd3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566a4d259d0 con 0x5566a4dd38d0 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566a4cc3710 con 0x5566a4dd8e40 2024-05-11T13:34:47.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.130+0000 7f6201377b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566a4d21990 con 0x5566a4dd9760 2024-05-11T13:34:47.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.131+0000 7f61efe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4dd3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.131+0000 7f61f4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4dd9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.131+0000 7f61efe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4dd3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:39390/0 (socket says 172.21.15.63:39390) 2024-05-11T13:34:47.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.131+0000 7f61efe00700 1 -- 172.21.15.63:0/3820856661 learned_addr learned my addr 172.21.15.63:0/3820856661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:47.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.131+0000 7f61ecc00700 1 -- 172.21.15.63:0/2404565603 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7f61dc00bb60 con 0x7f61f0102360 2024-05-11T13:34:47.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61f4a00700 1 -- 172.21.15.63:0/3820856661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 msgr2=0x5566a4dd3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61f4a00700 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4dd3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61ef400700 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61f4a00700 1 -- 172.21.15.63:0/3820856661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 msgr2=0x5566a4dd3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61f4a00700 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61f4a00700 1 -- 172.21.15.63:0/3820856661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a4c72880 con 0x5566a4dd8e40 2024-05-11T13:34:47.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61ef400700 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:34:47.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.132+0000 7f61f4a00700 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4dd9220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009d10 tx=0x7f61e80077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5903f336c0cfa87 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:47.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.133+0000 7f61e5600700 1 -- 172.21.15.63:0/3820856661 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8004bb0 con 0x5566a4dd8e40 2024-05-11T13:34:47.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.133+0000 7f61e5600700 1 -- 172.21.15.63:0/3820856661 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e8004d50 con 0x5566a4dd8e40 2024-05-11T13:34:47.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.133+0000 7f61e5600700 1 -- 172.21.15.63:0/3820856661 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8014410 con 0x5566a4dd8e40 2024-05-11T13:34:47.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.134+0000 7f61ecc00700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f61d40b2b90 0x7f61d40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.134+0000 7f61ecc00700 1 -- 172.21.15.63:0/2404565603 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f61dc0e2930 con 0x7f61f0102360 2024-05-11T13:34:47.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.134+0000 7f61efe00700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f61d40b2b90 0x7f61d40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 -- 172.21.15.63:0/3820856661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 msgr2=0x5566a4dd9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4dd9220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009d10 tx=0x7f61e80077e0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f61efe00700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f61d40b2b90 0x7f61d40b5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x5566a4dd9b60 tx=0x7f61e001a040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 -- 172.21.15.63:0/3820856661 shutdown_connections 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4dd9220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 --2- 172.21.15.63:0/3820856661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4dd3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 -- 172.21.15.63:0/3820856661 >> 172.21.15.63:0/3820856661 conn(0x5566a4d01b10 msgr2=0x5566a4dcdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 -- 172.21.15.63:0/3820856661 shutdown_connections 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.135+0000 7f6201377b80 1 -- 172.21.15.63:0/3820856661 wait complete. 2024-05-11T13:34:47.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.136+0000 7f6201377b80 1 -- start start 2024-05-11T13:34:47.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.136+0000 7f6201377b80 1 Processor -- start 2024-05-11T13:34:47.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f6201377b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd38d0 0x5566a4ddf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f6201377b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd8e40 0x5566a4ddc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f6201377b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4ddcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f6201377b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566a4de01b0 con 0x5566a4dd8e40 2024-05-11T13:34:47.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f6201377b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566a4c8bad0 con 0x5566a4dd9760 2024-05-11T13:34:47.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f6201377b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566a4de0330 con 0x5566a4dd38d0 2024-05-11T13:34:47.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f6201377b80 10 client.0 _ll_register_callbacks cb 0x5566a4cb9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:47.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f61ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd8e40 0x5566a4ddc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f61ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd8e40 0x5566a4ddc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51866/0 (socket says 172.21.15.63:51866) 2024-05-11T13:34:47.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.137+0000 7f61ef400700 1 -- 172.21.15.63:0/229150662 learned_addr learned my addr 172.21.15.63:0/229150662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61efe00700 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd38d0 0x5566a4ddf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61f4a00700 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4ddcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61ef400700 1 -- 172.21.15.63:0/229150662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd38d0 msgr2=0x5566a4ddf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61ef400700 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd38d0 0x5566a4ddf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61ef400700 1 -- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 msgr2=0x5566a4ddcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61ef400700 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4ddcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61ef400700 1 -- 172.21.15.63:0/229150662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a4ceb3d0 con 0x5566a4dd8e40 2024-05-11T13:34:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61ef400700 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd8e40 0x5566a4ddc8b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0a92d0 tx=0x7f61dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98548edd4dcb0e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.138+0000 7f61da000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.139+0000 7f61da000700 1 -- 172.21.15.63:0/229150662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0a95d0 con 0x5566a4dd8e40 2024-05-11T13:34:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.139+0000 7f61da000700 1 -- 172.21.15.63:0/229150662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f61dc0b1fa0 con 0x5566a4dd8e40 2024-05-11T13:34:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.139+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e8004d50 con 0x5566a4dd8e40 2024-05-11T13:34:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.139+0000 7f61da000700 1 -- 172.21.15.63:0/229150662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f61dc0ad980 con 0x5566a4dd8e40 2024-05-11T13:34:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.139+0000 7f6201377b80 10 client.15510 fetch_fsmap learned FSMap version 31 2024-05-11T13:34:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.139+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5566a4deaa50 con 0x5566a4dd8e40 2024-05-11T13:34:47.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.140+0000 7f61da000700 1 -- 172.21.15.63:0/229150662 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e8004d50 con 0x5566a4dd8e40 2024-05-11T13:34:47.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.140+0000 7f61da000700 1 -- 172.21.15.63:0/229150662 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f61dc022060 con 0x5566a4dd8e40 2024-05-11T13:34:47.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.140+0000 7f6201377b80 10 client.15510 fetch_fsmap finished waiting for FSMap version 31 2024-05-11T13:34:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.141+0000 7f6201377b80 10 client.15510 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:34:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.141+0000 7f6201377b80 20 client.15510 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.141+0000 7f6201377b80 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4decff0 0x5566a4dee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.141+0000 7f6201377b80 4 client.15510 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:34:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.141+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566a4da0010 con 0x5566a4decff0 2024-05-11T13:34:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.141+0000 7f61efe00700 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4decff0 0x5566a4dee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.142+0000 7f61efe00700 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4decff0 0x5566a4dee460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86afec8174e2f7a2 server_cookie=661ce5db4dfb334a in_seq=0 out_seq=0 2024-05-11T13:34:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.142+0000 7f61da000700 10 client.15510 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:34:47.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f61da000700 1 -- 172.21.15.63:0/229150662 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566a4da0010 con 0x5566a4decff0 2024-05-11T13:34:47.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f61da000700 10 client.15510 handle_command_reply: tid=0 2024-05-11T13:34:47.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 client.15510 shutdown 2024-05-11T13:34:47.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd8e40 msgr2=0x5566a4ddc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd8e40 0x5566a4ddc8b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0a92d0 tx=0x7f61dc004600 comp rx=0 tx=0).stop 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 shutdown_connections 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4decff0 0x5566a4dee460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4ddcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd38d0 0x5566a4ddf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 --2- 172.21.15.63:0/229150662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd8e40 0x5566a4ddc8b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 >> 172.21.15.63:0/229150662 conn(0x5566a4d01b10 msgr2=0x5566a4dd22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.145+0000 7f61da000700 0 client.15510 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.146+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 shutdown_connections 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.146+0000 7f6201377b80 1 -- 172.21.15.63:0/229150662 wait complete. 2024-05-11T13:34:47.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.146+0000 7f6201377b80 20 client.15510 trim_cache size 0 max 16384 2024-05-11T13:34:47.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.173+0000 7f6201377b80 1 Processor -- start 2024-05-11T13:34:47.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.173+0000 7f6201377b80 1 -- start start 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f6201377b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f6201377b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd8e40 0x5566a4ddce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f6201377b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4df7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f6201377b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566a4dec7a0 con 0x5566a4dd38d0 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f6201377b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566a4dece70 con 0x5566a4dd9760 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f6201377b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566a4ded540 con 0x5566a4dd8e40 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f61efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f61efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51874/0 (socket says 172.21.15.63:51874) 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.174+0000 7f61efe00700 1 -- 172.21.15.63:0/2225028929 learned_addr learned my addr 172.21.15.63:0/2225028929 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61ef400700 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd8e40 0x5566a4ddce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61f4a00700 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4df7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61efe00700 1 -- 172.21.15.63:0/2225028929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd8e40 msgr2=0x5566a4ddce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61efe00700 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd8e40 0x5566a4ddce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61efe00700 1 -- 172.21.15.63:0/2225028929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 msgr2=0x5566a4df7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61efe00700 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4df7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61efe00700 1 -- 172.21.15.63:0/2225028929 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a4deda30 con 0x5566a4dd38d0 2024-05-11T13:34:47.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61ef400700 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd8e40 0x5566a4ddce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:34:47.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.175+0000 7f61efe00700 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f61e000b5c0 tx=0x7f61e0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d75bc190349ec91 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:47.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.176+0000 7f61e4c00700 1 -- 172.21.15.63:0/2225028929 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e003b070 con 0x5566a4dd38d0 2024-05-11T13:34:47.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.176+0000 7f61e4c00700 1 -- 172.21.15.63:0/2225028929 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e0004c60 con 0x5566a4dd38d0 2024-05-11T13:34:47.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.176+0000 7f61e4c00700 1 -- 172.21.15.63:0/2225028929 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0035110 con 0x5566a4dd38d0 2024-05-11T13:34:47.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 -- 172.21.15.63:0/2225028929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 msgr2=0x5566a4dd3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f61e000b5c0 tx=0x7f61e0024000 comp rx=0 tx=0).stop 2024-05-11T13:34:47.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 -- 172.21.15.63:0/2225028929 shutdown_connections 2024-05-11T13:34:47.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd9760 0x5566a4df7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4dd3300 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 --2- 172.21.15.63:0/2225028929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd8e40 0x5566a4ddce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 -- 172.21.15.63:0/2225028929 >> 172.21.15.63:0/2225028929 conn(0x5566a4d01b10 msgr2=0x5566a4ddb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:47.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 -- 172.21.15.63:0/2225028929 shutdown_connections 2024-05-11T13:34:47.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.177+0000 7f6201377b80 1 -- 172.21.15.63:0/2225028929 wait complete. 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 -- start start 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 Processor -- start 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4df79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4df7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4df2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566a4dee2e0 con 0x5566a4dd38d0 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566a4dedc10 con 0x5566a4dd8e40 2024-05-11T13:34:47.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566a4dee9b0 con 0x5566a4dd9760 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.178+0000 7f6201377b80 10 client.0 _ll_register_callbacks cb 0x5566a4cb9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4df79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4df79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51882/0 (socket says 172.21.15.63:51882) 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 -- 172.21.15.63:0/2746855076 learned_addr learned my addr 172.21.15.63:0/2746855076 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61ef400700 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4df7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 -- 172.21.15.63:0/2746855076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 msgr2=0x5566a4df2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4df2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 -- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 msgr2=0x5566a4df7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4df7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.179+0000 7f61efe00700 1 -- 172.21.15.63:0/2746855076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a4df31d0 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f61efe00700 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4df79a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005850 tx=0x7f61e0005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43a2400993e13d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f61da000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f61da000700 1 -- 172.21.15.63:0/2746855076 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e003b070 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f61da000700 1 -- 172.21.15.63:0/2746855076 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f61e000bd80 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e0004c60 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f61da000700 1 -- 172.21.15.63:0/2746855076 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f61e0004aa0 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f6201377b80 10 client.15516 fetch_fsmap learned FSMap version 31 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.180+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5566a4e618b0 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f61da000700 1 -- 172.21.15.63:0/2746855076 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e0004c60 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f61da000700 1 -- 172.21.15.63:0/2746855076 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f61e0025480 con 0x5566a4dd38d0 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f6201377b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 31 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f6201377b80 10 client.15516 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T13:34:47.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f6201377b80 20 client.15516 populate_metadata read hostname 'smithi063' 2024-05-11T13:34:47.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f6201377b80 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4e65f90 0x5566a4e64be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:34:47.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f6201377b80 4 client.15516 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T13:34:47.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.181+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "status"}) -- 0x5566a4d333b0 con 0x5566a4e65f90 2024-05-11T13:34:47.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.182+0000 7f61ef400700 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4e65f90 0x5566a4e64be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:34:47.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.183+0000 7f61ef400700 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4e65f90 0x5566a4e64be0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c198f499bb6497f1 server_cookie=630419bcb85c93cc in_seq=0 out_seq=0 2024-05-11T13:34:47.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.183+0000 7f61da000700 10 client.15516 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:34:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f61da000700 1 -- 172.21.15.63:0/2746855076 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5566a4d333b0 con 0x5566a4e65f90 2024-05-11T13:34:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f61da000700 10 client.15516 handle_command_reply: tid=0 2024-05-11T13:34:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 client.15516 shutdown 2024-05-11T13:34:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 msgr2=0x5566a4df79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4df79a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005850 tx=0x7f61e0005d10 comp rx=0 tx=0).stop 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 shutdown_connections 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a4dd9760 0x5566a4df2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566a4dd38d0 0x5566a4df79a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566a4e65f90 0x5566a4e64be0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 --2- 172.21.15.63:0/2746855076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566a4dd8e40 0x5566a4df7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 >> 172.21.15.63:0/2746855076 conn(0x5566a4d01b10 msgr2=0x5566a4dc7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.184+0000 7f61da000700 0 client.15516 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.185+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 shutdown_connections 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.185+0000 7f6201377b80 1 -- 172.21.15.63:0/2746855076 wait complete. 2024-05-11T13:34:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.185+0000 7f6201377b80 20 client.15516 trim_cache size 0 max 16384 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "a0d7c9ac-0f98-11ef-bc98-c7b262605968", 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 0, 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34472, 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 368.96388474000003, 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 31, 2024-05-11T13:34:47.185 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-11T13:34:47.186 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 91, 2024-05-11T13:34:47.186 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 388.14186020699998 2024-05-11T13:34:47.186 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:34:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f61d40b2b90 msgr2=0x7f61d40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f61d40b2b90 0x7f61d40b5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x5566a4dd9b60 tx=0x7f61e001a040 comp rx=0 tx=0).stop 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0102360 msgr2=0x7f61f01030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0102360 0x7f61f01030c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00d980 tx=0x7f61dc00dd40 comp rx=0 tx=0).stop 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 shutdown_connections 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f61d40b2b90 0x7f61d40b5050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61f0109570 0x7f61f0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f01019a0 0x7f61f0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 --2- 172.21.15.63:0/2404565603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61f0102360 0x7f61f01030c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 >> 172.21.15.63:0/2404565603 conn(0x7f61f00788b0 msgr2=0x7f61f0100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 shutdown_connections 2024-05-11T13:34:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:34:47.186+0000 7f61f5400700 1 -- 172.21.15.63:0/2404565603 wait complete. 2024-05-11T13:34:47.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:47 smithi063 ceph-mon[29809]: pgmap v30: 129 pgs: 129 active+clean; 28 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 888 KiB/s wr, 57 op/s 2024-05-11T13:34:47.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:47 smithi071 ceph-mon[33293]: pgmap v30: 129 pgs: 129 active+clean; 28 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 888 KiB/s wr, 57 op/s 2024-05-11T13:34:47.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:47 smithi165 ceph-mon[33235]: pgmap v30: 129 pgs: 129 active+clean; 28 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 888 KiB/s wr, 57 op/s 2024-05-11T13:34:48.117 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-11T13:34:49.433 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:49 smithi063 ceph-mon[29809]: pgmap v31: 129 pgs: 129 active+clean; 28 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 923 KiB/s wr, 59 op/s 2024-05-11T13:34:49.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:49 smithi071 ceph-mon[33293]: pgmap v31: 129 pgs: 129 active+clean; 28 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 923 KiB/s wr, 59 op/s 2024-05-11T13:34:49.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:49 smithi165 ceph-mon[33235]: pgmap v31: 129 pgs: 129 active+clean; 28 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 923 KiB/s wr, 59 op/s 2024-05-11T13:34:51.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:51 smithi063 ceph-mon[29809]: pgmap v32: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-05-11T13:34:51.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:51 smithi071 ceph-mon[33293]: pgmap v32: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-05-11T13:34:51.566 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:51 smithi165 ceph-mon[33235]: pgmap v32: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-05-11T13:34:53.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:53 smithi063 ceph-mon[29809]: pgmap v33: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.9 MiB/s wr, 75 op/s 2024-05-11T13:34:53.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:53 smithi071 ceph-mon[33293]: pgmap v33: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.9 MiB/s wr, 75 op/s 2024-05-11T13:34:53.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:53 smithi165 ceph-mon[33235]: pgmap v33: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.9 MiB/s wr, 75 op/s 2024-05-11T13:34:55.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:55 smithi063 ceph-mon[29809]: pgmap v34: 129 pgs: 129 active+clean; 44 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 2.7 MiB/s wr, 106 op/s 2024-05-11T13:34:55.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:55 smithi071 ceph-mon[33293]: pgmap v34: 129 pgs: 129 active+clean; 44 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 2.7 MiB/s wr, 106 op/s 2024-05-11T13:34:55.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:55 smithi165 ceph-mon[33235]: pgmap v34: 129 pgs: 129 active+clean; 44 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 2.7 MiB/s wr, 106 op/s 2024-05-11T13:34:56.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:56 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:56.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:56 smithi071 ceph-mon[33293]: pgmap v35: 129 pgs: 129 active+clean; 44 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.3 MiB/s wr, 77 op/s 2024-05-11T13:34:56.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:56 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:56.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:56 smithi165 ceph-mon[33235]: pgmap v35: 129 pgs: 129 active+clean; 44 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.3 MiB/s wr, 77 op/s 2024-05-11T13:34:56.682 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:56 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:34:56.682 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:56 smithi063 ceph-mon[29809]: pgmap v35: 129 pgs: 129 active+clean; 44 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.3 MiB/s wr, 77 op/s 2024-05-11T13:34:59.313 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:34:59 smithi165 ceph-mon[33235]: pgmap v36: 129 pgs: 129 active+clean; 44 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-05-11T13:34:59.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:34:59 smithi063 ceph-mon[29809]: pgmap v36: 129 pgs: 129 active+clean; 44 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-05-11T13:34:59.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:34:59 smithi071 ceph-mon[33293]: pgmap v36: 129 pgs: 129 active+clean; 44 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-05-11T13:35:01.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:01 smithi063 ceph-mon[29809]: pgmap v37: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-05-11T13:35:01.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:01 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:01.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:01 smithi071 ceph-mon[33293]: pgmap v37: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-05-11T13:35:01.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:01 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:01.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:01 smithi165 ceph-mon[33235]: pgmap v37: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-05-11T13:35:01.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:01 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:02.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:35:02.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:35:02.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:35:03.118 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-11T13:35:03.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:35:03.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:03 smithi063 ceph-mon[29809]: pgmap v38: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-05-11T13:35:03.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:03 smithi071 ceph-mon[33293]: pgmap v38: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-05-11T13:35:03.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:03 smithi165 ceph-mon[33235]: pgmap v38: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-05-11T13:35:03.622 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:35:04.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:04 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:35:04.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:04 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:35:04.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:04 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:35:04.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:04 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:35:04.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:04 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:35:04.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:04 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:35:04.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:04 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:35:04.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:04 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:35:04.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:04 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:35:05.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:05 smithi063 ceph-mon[29809]: pgmap v39: 129 pgs: 129 active+clean; 48 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-11T13:35:05.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:05 smithi071 ceph-mon[33293]: pgmap v39: 129 pgs: 129 active+clean; 48 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-11T13:35:05.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:05 smithi165 ceph-mon[33235]: pgmap v39: 129 pgs: 129 active+clean; 48 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-11T13:35:06.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 -- 172.21.15.63:0/2122449324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb224108900 msgr2=0x7fb224110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:06.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 --2- 172.21.15.63:0/2122449324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb224108900 0x7fb224110480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00b0d0 tx=0x7fb21c00b3e0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 -- 172.21.15.63:0/2122449324 shutdown_connections 2024-05-11T13:35:06.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 --2- 172.21.15.63:0/2122449324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb224108900 0x7fb224110480 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 --2- 172.21.15.63:0/2122449324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb224103800 0x7fb224103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 --2- 172.21.15.63:0/2122449324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2241041b0 0x7fb2241081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 -- 172.21.15.63:0/2122449324 >> 172.21.15.63:0/2122449324 conn(0x7fb22407a6b0 msgr2=0x7fb22407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:06.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.009+0000 7fb228c00700 1 -- 172.21.15.63:0/2122449324 shutdown_connections 2024-05-11T13:35:06.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.011+0000 7fb228c00700 1 -- 172.21.15.63:0/2122449324 wait complete. 2024-05-11T13:35:06.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.012+0000 7fb228c00700 1 Processor -- start 2024-05-11T13:35:06.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.012+0000 7fb228c00700 1 -- start start 2024-05-11T13:35:06.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.012+0000 7fb228c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb224103800 0x7fb2241a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.012+0000 7fb228c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2241041b0 0x7fb2241a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.012+0000 7fb228c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb224108900 0x7fb22419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb228c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2241135f0 con 0x7fb2241041b0 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb228c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb224113470 con 0x7fb224103800 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb228c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb224113770 con 0x7fb224108900 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2241041b0 0x7fb2241a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2241041b0 0x7fb2241a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52406/0 (socket says 172.21.15.63:52406) 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221400700 1 -- 172.21.15.63:0/3083925577 learned_addr learned my addr 172.21.15.63:0/3083925577 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:06.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221e00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb224103800 0x7fb2241a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:06.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb222800700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb224108900 0x7fb22419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:06.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221400700 1 -- 172.21.15.63:0/3083925577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb224108900 msgr2=0x7fb22419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:06.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221400700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb224108900 0x7fb22419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221400700 1 -- 172.21.15.63:0/3083925577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb224103800 msgr2=0x7fb2241a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:06.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.013+0000 7fb221400700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb224103800 0x7fb2241a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb221400700 1 -- 172.21.15.63:0/3083925577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22419c250 con 0x7fb2241041b0 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb222800700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb224108900 0x7fb22419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb221400700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2241041b0 0x7fb2241a1e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb21000c970 tx=0x7fb21000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb21aa00700 1 -- 172.21.15.63:0/3083925577 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210007aa0 con 0x7fb2241041b0 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb21aa00700 1 -- 172.21.15.63:0/3083925577 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb210004cc0 con 0x7fb2241041b0 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb21aa00700 1 -- 172.21.15.63:0/3083925577 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210005450 con 0x7fb2241041b0 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22419c540 con 0x7fb2241041b0 2024-05-11T13:35:06.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.014+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2241a8800 con 0x7fb2241041b0 2024-05-11T13:35:06.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.017+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22406a240 con 0x7fb2241041b0 2024-05-11T13:35:06.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.019+0000 7fb21aa00700 1 -- 172.21.15.63:0/3083925577 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7fb210007c40 con 0x7fb2241041b0 2024-05-11T13:35:06.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.021+0000 7fb21aa00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb2080b2b90 0x7fb2080b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:06.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.021+0000 7fb21aa00700 1 -- 172.21.15.63:0/3083925577 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fb2100e29a0 con 0x7fb2241041b0 2024-05-11T13:35:06.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.021+0000 7fb221e00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb2080b2b90 0x7fb2080b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:06.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.022+0000 7fb221e00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb2080b2b90 0x7fb2080b5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb214005950 tx=0x7fb21400b410 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:06.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.027+0000 7fb21aa00700 1 -- 172.21.15.63:0/3083925577 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2100a9e30 con 0x7fb2241041b0 2024-05-11T13:35:06.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.385+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2241a8a30 con 0x7fb2241041b0 2024-05-11T13:35:06.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.387+0000 7fb21aa00700 1 -- 172.21.15.63:0/3083925577 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11949 (secure 0 0 0) 0x7fb2100a9730 con 0x7fb2241041b0 2024-05-11T13:35:06.386 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:06.386 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:34:59.226066+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb2080b2b90 msgr2=0x7fb2080b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb2080b2b90 0x7fb2080b5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb214005950 tx=0x7fb21400b410 comp rx=0 tx=0).stop 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2241041b0 msgr2=0x7fb2241a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2241041b0 0x7fb2241a1e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb21000c970 tx=0x7fb21000cc80 comp rx=0 tx=0).stop 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 shutdown_connections 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb2080b2b90 0x7fb2080b5050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb224108900 0x7fb22419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb224103800 0x7fb2241a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 --2- 172.21.15.63:0/3083925577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2241041b0 0x7fb2241a1e70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 >> 172.21.15.63:0/3083925577 conn(0x7fb22407a6b0 msgr2=0x7fb224101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.391+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 shutdown_connections 2024-05-11T13:35:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:06.392+0000 7fb228c00700 1 -- 172.21.15.63:0/3083925577 wait complete. 2024-05-11T13:35:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-11T13:35:07.360 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-05-11T13:35:07.361 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-05-11T13:35:07.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:07 smithi063 ceph-mon[29809]: pgmap v40: 129 pgs: 129 active+clean; 48 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 483 KiB/s wr, 56 op/s 2024-05-11T13:35:07.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:07 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3083925577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:07.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:07 smithi071 ceph-mon[33293]: pgmap v40: 129 pgs: 129 active+clean; 48 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 483 KiB/s wr, 56 op/s 2024-05-11T13:35:07.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:07 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3083925577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:07.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:07 smithi165 ceph-mon[33235]: pgmap v40: 129 pgs: 129 active+clean; 48 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 483 KiB/s wr, 56 op/s 2024-05-11T13:35:07.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:07 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3083925577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:07.850 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:35:09.407 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:09 smithi063 ceph-mon[29809]: pgmap v41: 129 pgs: 129 active+clean; 50 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 648 KiB/s wr, 87 op/s 2024-05-11T13:35:09.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:09 smithi071 ceph-mon[33293]: pgmap v41: 129 pgs: 129 active+clean; 50 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 648 KiB/s wr, 87 op/s 2024-05-11T13:35:09.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:09 smithi165 ceph-mon[33235]: pgmap v41: 129 pgs: 129 active+clean; 50 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 648 KiB/s wr, 87 op/s 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.344+0000 7ff5ae600700 1 -- 172.21.15.63:0/4240509495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8104f20 msgr2=0x7ff5a81093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.344+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4240509495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8104f20 0x7ff5a81093d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff5940099c0 tx=0x7ff594009cd0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.344+0000 7ff5ae600700 1 -- 172.21.15.63:0/4240509495 shutdown_connections 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.344+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4240509495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a8109b00 0x7ff5a8110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.344+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4240509495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a8104570 0x7ff5a8104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.344+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4240509495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8104f20 0x7ff5a81093d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.344+0000 7ff5ae600700 1 -- 172.21.15.63:0/4240509495 >> 172.21.15.63:0/4240509495 conn(0x7ff5a80fdda0 msgr2=0x7ff5a81001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.345+0000 7ff5ae600700 1 -- 172.21.15.63:0/4240509495 shutdown_connections 2024-05-11T13:35:10.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.345+0000 7ff5ae600700 1 -- 172.21.15.63:0/4240509495 wait complete. 2024-05-11T13:35:10.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.346+0000 7ff5ae600700 1 Processor -- start 2024-05-11T13:35:10.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.346+0000 7ff5ae600700 1 -- start start 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5ae600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a8104570 0x7ff5a81a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5ae600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a8104f20 0x7ff5a81a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5ae600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8109b00 0x7ff5a819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5ae600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81135a0 con 0x7ff5a8109b00 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5ae600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a8113420 con 0x7ff5a8104570 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5ae600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a8113720 con 0x7ff5a8104f20 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5adc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8109b00 0x7ff5a819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5adc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8109b00 0x7ff5a819bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41000/0 (socket says 172.21.15.63:41000) 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.347+0000 7ff5adc00700 1 -- 172.21.15.63:0/4142311489 learned_addr learned my addr 172.21.15.63:0/4142311489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:10.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5ad200700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a8104570 0x7ff5a81a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a8104f20 0x7ff5a81a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5adc00700 1 -- 172.21.15.63:0/4142311489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a8104f20 msgr2=0x7ff5a81a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5adc00700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a8104f20 0x7ff5a81a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5adc00700 1 -- 172.21.15.63:0/4142311489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a8104570 msgr2=0x7ff5a81a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5adc00700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a8104570 0x7ff5a81a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5adc00700 1 -- 172.21.15.63:0/4142311489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a819c3f0 con 0x7ff5a8109b00 2024-05-11T13:35:10.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.348+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a8104f20 0x7ff5a81a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:35:10.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.349+0000 7ff5adc00700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8109b00 0x7ff5a819bb10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000d980 tx=0x7ff5a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:10.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.349+0000 7ff5a5600700 1 -- 172.21.15.63:0/4142311489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0017070 con 0x7ff5a8109b00 2024-05-11T13:35:10.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.349+0000 7ff5a5600700 1 -- 172.21.15.63:0/4142311489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a0005020 con 0x7ff5a8109b00 2024-05-11T13:35:10.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.349+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a819c6e0 con 0x7ff5a8109b00 2024-05-11T13:35:10.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.349+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a81a89c0 con 0x7ff5a8109b00 2024-05-11T13:35:10.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.350+0000 7ff5a5600700 1 -- 172.21.15.63:0/4142311489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0009990 con 0x7ff5a8109b00 2024-05-11T13:35:10.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.353+0000 7ff5a5600700 1 -- 172.21.15.63:0/4142311489 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7ff5a0004a30 con 0x7ff5a8109b00 2024-05-11T13:35:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.355+0000 7ff5ba542b80 1 Processor -- start 2024-05-11T13:35:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.355+0000 7ff5a5600700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5900b2bc0 0x7ff5900b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.355+0000 7ff5ba542b80 1 -- start start 2024-05-11T13:35:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.355+0000 7ff5a5600700 1 -- 172.21.15.63:0/4142311489 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff5a00e2a20 con 0x7ff5a8109b00 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5900b2bc0 0x7ff5900b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e1890 0x563ad54e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ad5433bb0 con 0x563ad54e2250 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563ad53d18f0 con 0x563ad54e1890 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563ad542fb70 con 0x563ad54e7530 2024-05-11T13:35:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5adc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.356+0000 7ff5adc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54ee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41014/0 (socket says 172.21.15.63:41014) 2024-05-11T13:35:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5adc00700 1 -- 172.21.15.63:0/1792725600 learned_addr learned my addr 172.21.15.63:0/1792725600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5ad200700 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e1890 0x563ad54e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5adc00700 1 -- 172.21.15.63:0/1792725600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 msgr2=0x563ad54e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5adc00700 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5adc00700 1 -- 172.21.15.63:0/1792725600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e1890 msgr2=0x563ad54e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5adc00700 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e1890 0x563ad54e1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5adc00700 1 -- 172.21.15.63:0/1792725600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad5380a60 con 0x563ad54e2250 2024-05-11T13:35:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5900b2bc0 0x7ff5900b5080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff594009990 tx=0x7ff594005fb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.357+0000 7ff5adc00700 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54ee2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00a9650 tx=0x7ff5a0004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab8b41ffd95099 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.358+0000 7ff59a000700 1 -- 172.21.15.63:0/1792725600 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00a96c0 con 0x563ad54e2250 2024-05-11T13:35:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.358+0000 7ff59a000700 1 -- 172.21.15.63:0/1792725600 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a00ada70 con 0x563ad54e2250 2024-05-11T13:35:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.358+0000 7ff59a000700 1 -- 172.21.15.63:0/1792725600 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a008fde0 con 0x563ad54e2250 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1792725600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 msgr2=0x563ad54ee2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54ee2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00a9650 tx=0x7ff5a0004d20 comp rx=0 tx=0).stop 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1792725600 shutdown_connections 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54ee2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1792725600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e1890 0x563ad54e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1792725600 >> 172.21.15.63:0/1792725600 conn(0x563ad540fcf0 msgr2=0x563ad54dc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1792725600 shutdown_connections 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.359+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1792725600 wait complete. 2024-05-11T13:35:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 -- start start 2024-05-11T13:35:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 Processor -- start 2024-05-11T13:35:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e1890 0x563ad54edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54eb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54eb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ad54ee890 con 0x563ad54e2250 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563ad5399cb0 con 0x563ad54e7530 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563ad54eea10 con 0x563ad54e1890 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.360+0000 7ff5ba542b80 10 client.0 _ll_register_callbacks cb 0x563ad53c7510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54eb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41020/0 (socket says 172.21.15.63:41020) 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 -- 172.21.15.63:0/1614417655 learned_addr learned my addr 172.21.15.63:0/1614417655 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5ad200700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e1890 0x563ad54edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5adc00700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54eb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 -- 172.21.15.63:0/1614417655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e1890 msgr2=0x563ad54edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e1890 0x563ad54edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 -- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 msgr2=0x563ad54eb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54eb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5a7e00700 1 -- 172.21.15.63:0/1614417655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ad53f95b0 con 0x563ad54e2250 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.361+0000 7ff5adc00700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54eb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.362+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54eb330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff5a806b510 tx=0x7ff594020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd76e8a7dd88c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.362+0000 7ff58ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.362+0000 7ff58ea00700 1 -- 172.21.15.63:0/1614417655 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594003df0 con 0x563ad54e2250 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.362+0000 7ff58ea00700 1 -- 172.21.15.63:0/1614417655 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff594004830 con 0x563ad54e2250 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.362+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a00ada70 con 0x563ad54e2250 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff58ea00700 1 -- 172.21.15.63:0/1614417655 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff594004a30 con 0x563ad54e2250 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff5ba542b80 10 client.15534 fetch_fsmap learned FSMap version 33 2024-05-11T13:35:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ad54f90e0 con 0x563ad54e2250 2024-05-11T13:35:10.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff58ea00700 1 -- 172.21.15.63:0/1614417655 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a00ada70 con 0x563ad54e2250 2024-05-11T13:35:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff58ea00700 1 -- 172.21.15.63:0/1614417655 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7ff59402f480 con 0x563ad54e2250 2024-05-11T13:35:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff5ba542b80 10 client.15534 fetch_fsmap finished waiting for FSMap version 33 2024-05-11T13:35:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff5ba542b80 10 client.15534 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:35:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.363+0000 7ff5ba542b80 20 client.15534 populate_metadata read hostname 'smithi063' 2024-05-11T13:35:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.364+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad54fb7c0 0x563ad54fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.364+0000 7ff5ba542b80 4 client.15534 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:35:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.364+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ad54ae1f0 con 0x563ad54fb7c0 2024-05-11T13:35:10.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.365+0000 7ff5ad200700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad54fb7c0 0x563ad54fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.366+0000 7ff5ad200700 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad54fb7c0 0x563ad54fdbb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45507cd2b9251a0b server_cookie=c2204fc2a62aed56 in_seq=0 out_seq=0 2024-05-11T13:35:10.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.366+0000 7ff58ea00700 10 client.15534 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:10.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.368+0000 7ff58ea00700 1 -- 172.21.15.63:0/1614417655 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ad54ae1f0 con 0x563ad54fb7c0 2024-05-11T13:35:10.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.368+0000 7ff58ea00700 10 client.15534 handle_command_reply: tid=0 2024-05-11T13:35:10.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 client.15534 shutdown 2024-05-11T13:35:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 msgr2=0x563ad54eb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54eb330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff5a806b510 tx=0x7ff594020000 comp rx=0 tx=0).stop 2024-05-11T13:35:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 shutdown_connections 2024-05-11T13:35:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad54fb7c0 0x563ad54fdbb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54eb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e1890 0x563ad54edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1614417655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e2250 0x563ad54eb330 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.369+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 >> 172.21.15.63:0/1614417655 conn(0x563ad540fcf0 msgr2=0x563ad54e0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.370+0000 7ff58ea00700 0 client.15534 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.370+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 shutdown_connections 2024-05-11T13:35:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.370+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1614417655 wait complete. 2024-05-11T13:35:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.370+0000 7ff5ba542b80 20 client.15534 trim_cache size 0 max 16384 2024-05-11T13:35:10.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.395+0000 7ff5ba542b80 1 Processor -- start 2024-05-11T13:35:10.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.395+0000 7ff5ba542b80 1 -- start start 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.395+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54ea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e2250 0x563ad54f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54e5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b14a0 con 0x563ad54e1890 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b3160 con 0x563ad54e7530 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b1b70 con 0x563ad54e2250 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54ea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41034/0 (socket says 172.21.15.63:41034) 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 -- 172.21.15.63:0/1782396531 learned_addr learned my addr 172.21.15.63:0/1782396531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e2250 0x563ad54f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5adc00700 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 -- 172.21.15.63:0/1782396531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e2250 msgr2=0x563ad54f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e2250 0x563ad54f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 -- 172.21.15.63:0/1782396531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 msgr2=0x563ad54e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54e5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.396+0000 7ff5ad200700 1 -- 172.21.15.63:0/1782396531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad54dca50 con 0x563ad54e1890 2024-05-11T13:35:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.397+0000 7ff5ad200700 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54ea290 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff59c000f80 tx=0x7ff59c0091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5663ee9c1bfe2c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.397+0000 7ff599600700 1 -- 172.21.15.63:0/1782396531 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c009be0 con 0x563ad54e1890 2024-05-11T13:35:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.397+0000 7ff599600700 1 -- 172.21.15.63:0/1782396531 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff59c009d80 con 0x563ad54e1890 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.397+0000 7ff599600700 1 -- 172.21.15.63:0/1782396531 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c01e2d0 con 0x563ad54e1890 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1782396531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 msgr2=0x563ad54ea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54ea290 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff59c000f80 tx=0x7ff59c0091b0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1782396531 shutdown_connections 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e7530 0x563ad54e5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54ea290 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/1782396531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e2250 0x563ad54f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1782396531 >> 172.21.15.63:0/1782396531 conn(0x563ad540fcf0 msgr2=0x563ad54e4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1782396531 shutdown_connections 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.398+0000 7ff5ba542b80 1 -- 172.21.15.63:0/1782396531 wait complete. 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.399+0000 7ff5ba542b80 1 -- start start 2024-05-11T13:35:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.399+0000 7ff5ba542b80 1 Processor -- start 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e2250 0x563ad54f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ba542b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b29d0 con 0x563ad54e1890 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b2300 con 0x563ad54e2250 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ba542b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81af440 con 0x563ad54e7530 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ba542b80 10 client.0 _ll_register_callbacks cb 0x563ad53c7510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41042/0 (socket says 172.21.15.63:41042) 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 -- 172.21.15.63:0/3905732811 learned_addr learned my addr 172.21.15.63:0/3905732811 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e2250 0x563ad54f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 -- 172.21.15.63:0/3905732811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 msgr2=0x563ad54f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5adc00700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54f0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 -- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e2250 msgr2=0x563ad54f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e2250 0x563ad54f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.400+0000 7ff5ad200700 1 -- 172.21.15.63:0/3905732811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ad54f1220 con 0x563ad54e1890 2024-05-11T13:35:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.401+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e2250 0x563ad54f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.401+0000 7ff5ad200700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54f3c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff59c004a60 tx=0x7ff59c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e373f6c01901bc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.401+0000 7ff58ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:35:10.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.401+0000 7ff58ea00700 1 -- 172.21.15.63:0/3905732811 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c02d060 con 0x563ad54e1890 2024-05-11T13:35:10.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.401+0000 7ff58ea00700 1 -- 172.21.15.63:0/3905732811 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7ff59c01d040 con 0x563ad54e1890 2024-05-11T13:35:10.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.401+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59c009d80 con 0x563ad54e1890 2024-05-11T13:35:10.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.402+0000 7ff58ea00700 1 -- 172.21.15.63:0/3905732811 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff59c004330 con 0x563ad54e1890 2024-05-11T13:35:10.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.402+0000 7ff5ba542b80 10 client.15540 fetch_fsmap learned FSMap version 33 2024-05-11T13:35:10.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.402+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ad55668b0 con 0x563ad54e1890 2024-05-11T13:35:10.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.402+0000 7ff58ea00700 1 -- 172.21.15.63:0/3905732811 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff59c009d80 con 0x563ad54e1890 2024-05-11T13:35:10.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.403+0000 7ff58ea00700 1 -- 172.21.15.63:0/3905732811 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7ff59c01ece0 con 0x563ad54e1890 2024-05-11T13:35:10.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.403+0000 7ff5ba542b80 10 client.15540 fetch_fsmap finished waiting for FSMap version 33 2024-05-11T13:35:10.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.403+0000 7ff5ba542b80 10 client.15540 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:35:10.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.403+0000 7ff5ba542b80 20 client.15540 populate_metadata read hostname 'smithi063' 2024-05-11T13:35:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.403+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad5567050 0x563ad5569470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.403+0000 7ff5ba542b80 4 client.15540 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-05-11T13:35:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.403+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x563ad5441590 con 0x563ad5567050 2024-05-11T13:35:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.404+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad5567050 0x563ad5569470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.405+0000 7ff5a7e00700 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad5567050 0x563ad5569470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deef0b4043deb74f server_cookie=6d48f6456d1e78ef in_seq=0 out_seq=0 2024-05-11T13:35:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.405+0000 7ff58ea00700 10 client.15540 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:10.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff58ea00700 1 -- 172.21.15.63:0/3905732811 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x563ad5441590 con 0x563ad5567050 2024-05-11T13:35:10.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff58ea00700 10 client.15540 handle_command_reply: tid=0 2024-05-11T13:35:10.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 client.15540 shutdown 2024-05-11T13:35:10.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 msgr2=0x563ad54f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54f3c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff59c004a60 tx=0x7ff59c004c60 comp rx=0 tx=0).stop 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 shutdown_connections 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ad54e7530 0x563ad54f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ad54e1890 0x563ad54f3c60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x563ad5567050 0x563ad5569470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 --2- 172.21.15.63:0/3905732811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ad54e2250 0x563ad54f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.735+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 >> 172.21.15.63:0/3905732811 conn(0x563ad540fcf0 msgr2=0x563ad54e1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.736+0000 7ff58ea00700 0 client.15540 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.736+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 shutdown_connections 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.736+0000 7ff5ba542b80 1 -- 172.21.15.63:0/3905732811 wait complete. 2024-05-11T13:35:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.736+0000 7ff5ba542b80 20 client.15540 trim_cache size 0 max 16384 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 33, 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 2, 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "21273d2c": { 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 2, 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:35:10.736 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 70.0, 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:10.737 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:10.738 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:35:10.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5900b2bc0 msgr2=0x7ff5900b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5900b2bc0 0x7ff5900b5080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff594009990 tx=0x7ff594005fb0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8109b00 msgr2=0x7ff5a819bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8109b00 0x7ff5a819bb10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000d980 tx=0x7ff5a000dc90 comp rx=0 tx=0).stop 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 shutdown_connections 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5900b2bc0 0x7ff5900b5080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a8109b00 0x7ff5a819bb10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a8104570 0x7ff5a81a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 --2- 172.21.15.63:0/4142311489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a8104f20 0x7ff5a81a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 >> 172.21.15.63:0/4142311489 conn(0x7ff5a80fdda0 msgr2=0x7ff5a80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.738+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 shutdown_connections 2024-05-11T13:35:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:10.739+0000 7ff5ae600700 1 -- 172.21.15.63:0/4142311489 wait complete. 2024-05-11T13:35:11.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:11 smithi063 ceph-mon[29809]: pgmap v42: 129 pgs: 129 active+clean; 50 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 590 KiB/s wr, 114 op/s 2024-05-11T13:35:11.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:11 smithi071 ceph-mon[33293]: pgmap v42: 129 pgs: 129 active+clean; 50 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 590 KiB/s wr, 114 op/s 2024-05-11T13:35:11.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:11 smithi165 ceph-mon[33235]: pgmap v42: 129 pgs: 129 active+clean; 50 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 590 KiB/s wr, 114 op/s 2024-05-11T13:35:11.635 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '21273d2c', quiesce took 8.5 seconds. Will release after: 10 2024-05-11T13:35:13.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:13 smithi063 ceph-mon[29809]: pgmap v43: 129 pgs: 129 active+clean; 50 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 392 KiB/s wr, 92 op/s 2024-05-11T13:35:13.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:13 smithi071 ceph-mon[33293]: pgmap v43: 129 pgs: 129 active+clean; 50 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 392 KiB/s wr, 92 op/s 2024-05-11T13:35:13.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:13 smithi165 ceph-mon[33235]: pgmap v43: 129 pgs: 129 active+clean; 50 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 392 KiB/s wr, 92 op/s 2024-05-11T13:35:15.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:15 smithi063 ceph-mon[29809]: pgmap v44: 129 pgs: 129 active+clean; 51 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 498 KiB/s wr, 124 op/s 2024-05-11T13:35:15.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:15 smithi071 ceph-mon[33293]: pgmap v44: 129 pgs: 129 active+clean; 51 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 498 KiB/s wr, 124 op/s 2024-05-11T13:35:15.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:15 smithi165 ceph-mon[33235]: pgmap v44: 129 pgs: 129 active+clean; 51 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 498 KiB/s wr, 124 op/s 2024-05-11T13:35:17.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:17 smithi063 ceph-mon[29809]: pgmap v45: 129 pgs: 129 active+clean; 51 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 369 KiB/s wr, 89 op/s 2024-05-11T13:35:17.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:17 smithi071 ceph-mon[33293]: pgmap v45: 129 pgs: 129 active+clean; 51 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 369 KiB/s wr, 89 op/s 2024-05-11T13:35:17.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:17 smithi165 ceph-mon[33235]: pgmap v45: 129 pgs: 129 active+clean; 51 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 369 KiB/s wr, 89 op/s 2024-05-11T13:35:18.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:18 smithi165 ceph-mon[33235]: pgmap v46: 129 pgs: 129 active+clean; 52 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 373 KiB/s wr, 93 op/s 2024-05-11T13:35:18.681 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:18 smithi063 ceph-mon[29809]: pgmap v46: 129 pgs: 129 active+clean; 52 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 373 KiB/s wr, 93 op/s 2024-05-11T13:35:18.701 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:18 smithi071 ceph-mon[33293]: pgmap v46: 129 pgs: 129 active+clean; 52 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 373 KiB/s wr, 93 op/s 2024-05-11T13:35:21.182 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:21 smithi063 ceph-mon[29809]: pgmap v47: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 209 KiB/s wr, 65 op/s 2024-05-11T13:35:21.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:21 smithi071 ceph-mon[33293]: pgmap v47: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 209 KiB/s wr, 65 op/s 2024-05-11T13:35:21.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:21 smithi165 ceph-mon[33235]: pgmap v47: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 209 KiB/s wr, 65 op/s 2024-05-11T13:35:21.636 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:35:22.114 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:35:23.343 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:23 smithi063 ceph-mon[29809]: pgmap v48: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 109 KiB/s wr, 37 op/s 2024-05-11T13:35:23.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:23 smithi071 ceph-mon[33293]: pgmap v48: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 109 KiB/s wr, 37 op/s 2024-05-11T13:35:23.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:23 smithi165 ceph-mon[33235]: pgmap v48: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 109 KiB/s wr, 37 op/s 2024-05-11T13:35:24.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.580+0000 7fbdeb200700 1 -- 172.21.15.63:0/2768457079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde4105680 msgr2=0x7fbde4105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:24.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.580+0000 7fbdeb200700 1 --2- 172.21.15.63:0/2768457079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde4105680 0x7fbde4105b00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009990 tx=0x7fbdd4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.581+0000 7fbdeb200700 1 -- 172.21.15.63:0/2768457079 shutdown_connections 2024-05-11T13:35:24.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.581+0000 7fbdeb200700 1 --2- 172.21.15.63:0/2768457079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4106040 0x7fbde4112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.581+0000 7fbdeb200700 1 --2- 172.21.15.63:0/2768457079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde410b680 0x7fbde410ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.581+0000 7fbdeb200700 1 --2- 172.21.15.63:0/2768457079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde4105680 0x7fbde4105b00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.581+0000 7fbdeb200700 1 -- 172.21.15.63:0/2768457079 >> 172.21.15.63:0/2768457079 conn(0x7fbde4101170 msgr2=0x7fbde4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:24.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.581+0000 7fbdeb200700 1 -- 172.21.15.63:0/2768457079 shutdown_connections 2024-05-11T13:35:24.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.581+0000 7fbdeb200700 1 -- 172.21.15.63:0/2768457079 wait complete. 2024-05-11T13:35:24.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 Processor -- start 2024-05-11T13:35:24.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 -- start start 2024-05-11T13:35:24.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde4105680 0x7fbde41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:24.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4106040 0x7fbde41a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:24.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde410b680 0x7fbde419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:24.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4115730 con 0x7fbde410b680 2024-05-11T13:35:24.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41155b0 con 0x7fbde4105680 2024-05-11T13:35:24.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.582+0000 7fbdeb200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41158b0 con 0x7fbde4106040 2024-05-11T13:35:24.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde410b680 0x7fbde419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:24.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde410b680 0x7fbde419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48612/0 (socket says 172.21.15.63:48612) 2024-05-11T13:35:24.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 -- 172.21.15.63:0/4019398546 learned_addr learned my addr 172.21.15.63:0/4019398546 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:24.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbde9e00700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde4105680 0x7fbde41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:24.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbde9400700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4106040 0x7fbde41a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 -- 172.21.15.63:0/4019398546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4106040 msgr2=0x7fbde41a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4106040 0x7fbde41a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 -- 172.21.15.63:0/4019398546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde4105680 msgr2=0x7fbde41a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde4105680 0x7fbde41a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.583+0000 7fbdea800700 1 -- 172.21.15.63:0/4019398546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde419c1b0 con 0x7fbde410b680 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.584+0000 7fbdea800700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde410b680 0x7fbde419ba50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbde000ef30 tx=0x7fbde000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.584+0000 7fbddea00700 1 -- 172.21.15.63:0/4019398546 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde00092e0 con 0x7fbde410b680 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.584+0000 7fbddea00700 1 -- 172.21.15.63:0/4019398546 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbde0004bb0 con 0x7fbde410b680 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.584+0000 7fbddea00700 1 -- 172.21.15.63:0/4019398546 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004020 con 0x7fbde410b680 2024-05-11T13:35:24.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.584+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde419c440 con 0x7fbde410b680 2024-05-11T13:35:24.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.584+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde41a8760 con 0x7fbde410b680 2024-05-11T13:35:24.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.586+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde406a240 con 0x7fbde410b680 2024-05-11T13:35:24.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.588+0000 7fbddea00700 1 -- 172.21.15.63:0/4019398546 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbde001b030 con 0x7fbde410b680 2024-05-11T13:35:24.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.589+0000 7fbddea00700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fbdcc0b2a70 0x7fbdcc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:24.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.590+0000 7fbddea00700 1 -- 172.21.15.63:0/4019398546 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fbde00e2ee0 con 0x7fbde410b680 2024-05-11T13:35:24.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.590+0000 7fbde9e00700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fbdcc0b2a70 0x7fbdcc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:24.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.591+0000 7fbde9e00700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fbdcc0b2a70 0x7fbdcc0b4f30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbde419cb90 tx=0x7fbdd8009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:24.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.593+0000 7fbddea00700 1 -- 172.21.15.63:0/4019398546 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbde0022020 con 0x7fbde410b680 2024-05-11T13:35:24.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.896+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbde402e570 con 0x7fbde410b680 2024-05-11T13:35:24.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.898+0000 7fbddea00700 1 -- 172.21.15.63:0/4019398546 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11949 (secure 0 0 0) 0x7fbde0010040 con 0x7fbde410b680 2024-05-11T13:35:24.897 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:24.897 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:34:59.226066+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:35:24.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fbdcc0b2a70 msgr2=0x7fbdcc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fbdcc0b2a70 0x7fbdcc0b4f30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbde419cb90 tx=0x7fbdd8009260 comp rx=0 tx=0).stop 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde410b680 msgr2=0x7fbde419ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde410b680 0x7fbde419ba50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbde000ef30 tx=0x7fbde000c3a0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 shutdown_connections 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fbdcc0b2a70 0x7fbdcc0b4f30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde410b680 0x7fbde419ba50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde4105680 0x7fbde41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 --2- 172.21.15.63:0/4019398546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4106040 0x7fbde41a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 >> 172.21.15.63:0/4019398546 conn(0x7fbde4101170 msgr2=0x7fbde4101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 shutdown_connections 2024-05-11T13:35:24.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:24.901+0000 7fbdeb200700 1 -- 172.21.15.63:0/4019398546 wait complete. 2024-05-11T13:35:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-11T13:35:25.181 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:25 smithi063 ceph-mon[29809]: pgmap v49: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 43 op/s 2024-05-11T13:35:25.181 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:25 smithi063 ceph-mon[29809]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-11T13:35:25.182 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:25 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4019398546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:25.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:25 smithi071 ceph-mon[33293]: pgmap v49: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 43 op/s 2024-05-11T13:35:25.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:25 smithi071 ceph-mon[33293]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-11T13:35:25.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:25 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4019398546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:25.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:25 smithi165 ceph-mon[33235]: pgmap v49: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 43 op/s 2024-05-11T13:35:25.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:25 smithi165 ceph-mon[33235]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-11T13:35:25.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:25 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4019398546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:25.837 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 21273d2c --release --await' 2024-05-11T13:35:25.837 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 21273d2c --release --await 2024-05-11T13:35:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:35:27.374 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:27 smithi063 ceph-mon[29809]: pgmap v50: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-05-11T13:35:27.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:27 smithi071 ceph-mon[33293]: pgmap v50: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-05-11T13:35:27.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:27 smithi165 ceph-mon[33235]: pgmap v50: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-05-11T13:35:29.366 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:29 smithi165 ceph-mon[33235]: pgmap v51: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 KiB/s wr, 16 op/s 2024-05-11T13:35:29.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 -- 172.21.15.63:0/3403520660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc106130 msgr2=0x7f9ecc112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 --2- 172.21.15.63:0/3403520660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc106130 0x7f9ecc112640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400b0a0 tx=0x7f9ec400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 -- 172.21.15.63:0/3403520660 shutdown_connections 2024-05-11T13:35:29.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 --2- 172.21.15.63:0/3403520660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc106130 0x7f9ecc112640 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 --2- 172.21.15.63:0/3403520660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ecc10b770 0x7f9ecc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 --2- 172.21.15.63:0/3403520660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc105770 0x7f9ecc105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 -- 172.21.15.63:0/3403520660 >> 172.21.15.63:0/3403520660 conn(0x7f9ecc101260 msgr2=0x7f9ecc103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:29.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 -- 172.21.15.63:0/3403520660 shutdown_connections 2024-05-11T13:35:29.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.389+0000 7f9ed1400700 1 -- 172.21.15.63:0/3403520660 wait complete. 2024-05-11T13:35:29.389 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:29 smithi063 ceph-mon[29809]: pgmap v51: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 KiB/s wr, 16 op/s 2024-05-11T13:35:29.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.390+0000 7f9ed1400700 1 Processor -- start 2024-05-11T13:35:29.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.390+0000 7f9ed1400700 1 -- start start 2024-05-11T13:35:29.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ed1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc105770 0x7f9ecc1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ed1400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ecc106130 0x7f9ecc1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ed1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc10b770 0x7f9ecc19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ed1400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc115850 con 0x7f9ecc10b770 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ed1400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1156d0 con 0x7f9ecc105770 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ed1400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1159d0 con 0x7f9ecc106130 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ecb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc10b770 0x7f9ecc19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ecb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc10b770 0x7f9ecc19bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48632/0 (socket says 172.21.15.63:48632) 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.391+0000 7f9ecb200700 1 -- 172.21.15.63:0/1829461855 learned_addr learned my addr 172.21.15.63:0/1829461855 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9eca800700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc105770 0x7f9ecc1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ecc106130 0x7f9ecc1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ecb200700 1 -- 172.21.15.63:0/1829461855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ecc106130 msgr2=0x7f9ecc1a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ecb200700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ecc106130 0x7f9ecc1a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ecb200700 1 -- 172.21.15.63:0/1829461855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc105770 msgr2=0x7f9ecc1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ecb200700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc105770 0x7f9ecc1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ecb200700 1 -- 172.21.15.63:0/1829461855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc19c210 con 0x7f9ecc10b770 2024-05-11T13:35:29.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ecc106130 0x7f9ecc1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:29.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.392+0000 7f9ecb200700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc10b770 0x7f9ecc19bab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4004dc0 tx=0x7f9ec4004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:29.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.393+0000 7f9ec3400700 1 -- 172.21.15.63:0/1829461855 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4021020 con 0x7f9ecc10b770 2024-05-11T13:35:29.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.393+0000 7f9ec3400700 1 -- 172.21.15.63:0/1829461855 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ec4007cf0 con 0x7f9ecc10b770 2024-05-11T13:35:29.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.393+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc19c4a0 con 0x7f9ecc10b770 2024-05-11T13:35:29.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.393+0000 7f9ec3400700 1 -- 172.21.15.63:0/1829461855 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005370 con 0x7f9ecc10b770 2024-05-11T13:35:29.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.393+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc1a87c0 con 0x7f9ecc10b770 2024-05-11T13:35:29.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.396+0000 7f9ec3400700 1 -- 172.21.15.63:0/1829461855 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9ec4024020 con 0x7f9ecc10b770 2024-05-11T13:35:29.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 Processor -- start 2024-05-11T13:35:29.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 -- start start 2024-05-11T13:35:29.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9ec3400700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9eb00b29f0 0x7f9eb00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d48fd0 0x557396d493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557396c95b20 con 0x557396d48fd0 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557396c33860 con 0x557396d43f60 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557396c91ae0 con 0x557396d435a0 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.398+0000 7f9ec3400700 1 -- 172.21.15.63:0/1829461855 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f9ec400f020 con 0x7f9ecc10b770 2024-05-11T13:35:29.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d48fd0 0x557396d493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9eb00b29f0 0x7f9eb00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d48fd0 0x557396d493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48648/0 (socket says 172.21.15.63:48648) 2024-05-11T13:35:29.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 -- 172.21.15.63:0/3119917743 learned_addr learned my addr 172.21.15.63:0/3119917743 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:29.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9ecb200700 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 -- 172.21.15.63:0/3119917743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 msgr2=0x557396d43a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d43a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 -- 172.21.15.63:0/3119917743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 msgr2=0x557396d4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9eca800700 1 -- 172.21.15.63:0/3119917743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557396be29d0 con 0x557396d48fd0 2024-05-11T13:35:29.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.399+0000 7f9ecb200700 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.400+0000 7f9eca800700 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d48fd0 0x557396d493b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc008b80 tx=0x7f9ebc008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c25d289fdcfbf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.400+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9eb00b29f0 0x7f9eb00b4eb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8005fd0 tx=0x7f9eb8005ee0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.400+0000 7f9eb7e00700 1 -- 172.21.15.63:0/3119917743 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc014070 con 0x557396d48fd0 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.400+0000 7f9eb7e00700 1 -- 172.21.15.63:0/3119917743 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ebc00ed10 con 0x557396d48fd0 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.400+0000 7f9eb7e00700 1 -- 172.21.15.63:0/3119917743 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc010430 con 0x557396d48fd0 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3119917743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d48fd0 msgr2=0x557396d493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d48fd0 0x557396d493b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc008b80 tx=0x7f9ebc008e90 comp rx=0 tx=0).stop 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3119917743 shutdown_connections 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d48fd0 0x557396d493b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3119917743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d43a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3119917743 >> 172.21.15.63:0/3119917743 conn(0x557396c71c60 msgr2=0x557396d3de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3119917743 shutdown_connections 2024-05-11T13:35:29.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.401+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3119917743 wait complete. 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.402+0000 7f9edbd04b80 1 -- start start 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.402+0000 7f9edbd04b80 1 Processor -- start 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d4f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d43f60 0x557396d4ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d4d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557396d50390 con 0x557396d43f60 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557396bfbc20 con 0x557396d48fd0 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557396d50510 con 0x557396d435a0 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9edbd04b80 10 client.0 _ll_register_callbacks cb 0x557396c3cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9ec9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d43f60 0x557396d4ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9ec9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d43f60 0x557396d4ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48662/0 (socket says 172.21.15.63:48662) 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9ec9e00700 1 -- 172.21.15.63:0/332215519 learned_addr learned my addr 172.21.15.63:0/332215519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:29.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9eca800700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d4f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.403+0000 7f9ecb200700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d4d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9ec9e00700 1 -- 172.21.15.63:0/332215519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 msgr2=0x557396d4f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d4f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9ec9e00700 1 -- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 msgr2=0x557396d4d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d4d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9ec9e00700 1 -- 172.21.15.63:0/332215519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557396c5b520 con 0x557396d43f60 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9eca800700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d4f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d43f60 0x557396d4ccc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80116a0 tx=0x7f9eb80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c42fca712bdbf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9eb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9eb4c00700 1 -- 172.21.15.63:0/332215519 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb800dea0 con 0x557396d43f60 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9eb4c00700 1 -- 172.21.15.63:0/332215519 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f9eb801c410 con 0x557396d43f60 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.404+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ebc00ed10 con 0x557396d43f60 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9eb4c00700 1 -- 172.21.15.63:0/332215519 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9eb8026630 con 0x557396d43f60 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 10 client.15558 fetch_fsmap learned FSMap version 33 2024-05-11T13:35:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557396d5ac20 con 0x557396d43f60 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9eb4c00700 1 -- 172.21.15.63:0/332215519 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ebc00ed10 con 0x557396d43f60 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9eb4c00700 1 -- 172.21.15.63:0/332215519 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f9eb801d7b0 con 0x557396d43f60 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 33 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 10 client.15558 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 20 client.15558 populate_metadata read hostname 'smithi063' 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396d5d260 0x557396d5f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 4 client.15558 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.405+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557396d102b0 con 0x557396d5d260 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.406+0000 7f9eca800700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396d5d260 0x557396d5f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.407+0000 7f9eca800700 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396d5d260 0x557396d5f6d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e040a767ec3750 server_cookie=485cad9fd7a41f6d in_seq=0 out_seq=0 2024-05-11T13:35:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.407+0000 7f9eb4c00700 10 client.15558 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9eb4c00700 1 -- 172.21.15.63:0/332215519 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557396d102b0 con 0x557396d5d260 2024-05-11T13:35:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9eb4c00700 10 client.15558 handle_command_reply: tid=0 2024-05-11T13:35:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 client.15558 shutdown 2024-05-11T13:35:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d43f60 msgr2=0x557396d4ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d43f60 0x557396d4ccc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80116a0 tx=0x7f9eb80047c0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 shutdown_connections 2024-05-11T13:35:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396d5d260 0x557396d5f6d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d4d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d435a0 0x557396d4f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/332215519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d43f60 0x557396d4ccc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 >> 172.21.15.63:0/332215519 conn(0x557396c71c60 msgr2=0x557396d38960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.409+0000 7f9eb4c00700 0 client.15558 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.410+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 shutdown_connections 2024-05-11T13:35:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.410+0000 7f9edbd04b80 1 -- 172.21.15.63:0/332215519 wait complete. 2024-05-11T13:35:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.410+0000 7f9edbd04b80 20 client.15558 trim_cache size 0 max 16384 2024-05-11T13:35:29.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 Processor -- start 2024-05-11T13:35:29.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 -- start start 2024-05-11T13:35:29.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d59480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d48fd0 0x557396d38610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac00ed20 con 0x557396d435a0 2024-05-11T13:35:29.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac0109e0 con 0x557396d43f60 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.432+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac00f3f0 con 0x557396d48fd0 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d59480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d59480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48664/0 (socket says 172.21.15.63:48664) 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 -- 172.21.15.63:0/3216419601 learned_addr learned my addr 172.21.15.63:0/3216419601 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9ecb200700 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d48fd0 0x557396d38610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 -- 172.21.15.63:0/3216419601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d48fd0 msgr2=0x557396d38610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d48fd0 0x557396d38610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 -- 172.21.15.63:0/3216419601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 msgr2=0x557396d380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 -- 172.21.15.63:0/3216419601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557396d5fd50 con 0x557396d435a0 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9ecb200700 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d48fd0 0x557396d38610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:29.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:29.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.433+0000 7f9eca800700 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d59480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc008ad0 tx=0x7f9ebc005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a2d7e39d8e6cb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9eb7400700 1 -- 172.21.15.63:0/3216419601 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc027070 con 0x557396d435a0 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9eb7400700 1 -- 172.21.15.63:0/3216419601 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ebc003680 con 0x557396d435a0 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9eb7400700 1 -- 172.21.15.63:0/3216419601 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc014040 con 0x557396d435a0 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3216419601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 msgr2=0x557396d59480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d59480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc008ad0 tx=0x7f9ebc005980 comp rx=0 tx=0).stop 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3216419601 shutdown_connections 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d48fd0 0x557396d38610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d59480 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.434+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/3216419601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d43f60 0x557396d380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.435+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3216419601 >> 172.21.15.63:0/3216419601 conn(0x557396c71c60 msgr2=0x557396d472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.435+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3216419601 shutdown_connections 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.435+0000 7f9edbd04b80 1 -- 172.21.15.63:0/3216419601 wait complete. 2024-05-11T13:35:29.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.435+0000 7f9edbd04b80 1 -- start start 2024-05-11T13:35:29.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.435+0000 7f9edbd04b80 1 Processor -- start 2024-05-11T13:35:29.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d43f60 0x557396d55f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9edbd04b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac010250 con 0x557396d435a0 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac00fb80 con 0x557396d48fd0 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9edbd04b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac00ccc0 con 0x557396d43f60 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9edbd04b80 10 client.0 _ll_register_callbacks cb 0x557396c3cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9eca800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9eca800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48666/0 (socket says 172.21.15.63:48666) 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.436+0000 7f9eca800700 1 -- 172.21.15.63:0/1419936055 learned_addr learned my addr 172.21.15.63:0/1419936055 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d43f60 0x557396d55f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eca800700 1 -- 172.21.15.63:0/1419936055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d43f60 msgr2=0x557396d55f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eca800700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d43f60 0x557396d55f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9ecb200700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eca800700 1 -- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 msgr2=0x557396d529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eca800700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eca800700 1 -- 172.21.15.63:0/1419936055 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557396d3aa90 con 0x557396d435a0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d43f60 0x557396d55f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9ecb200700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eca800700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d559e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00ebd0 tx=0x7f9ebc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dae8c424644a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eb4c00700 1 -- 172.21.15.63:0/1419936055 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc014030 con 0x557396d435a0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9eb4c00700 1 -- 172.21.15.63:0/1419936055 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f9ebc003a30 con 0x557396d435a0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.437+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ebc003680 con 0x557396d435a0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9eb4c00700 1 -- 172.21.15.63:0/1419936055 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9ebc027040 con 0x557396d435a0 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 10 client.15564 fetch_fsmap learned FSMap version 33 2024-05-11T13:35:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557396d3ea60 con 0x557396d435a0 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9eb4c00700 1 -- 172.21.15.63:0/1419936055 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ebc003680 con 0x557396d435a0 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9eb4c00700 1 -- 172.21.15.63:0/1419936055 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f9ebc023450 con 0x557396d435a0 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 33 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 10 client.15564 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 20 client.15564 populate_metadata read hostname 'smithi063' 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396dc8960 0x557396dcad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 4 client.15564 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "21273d2c", "release": true, "await": true}] 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.438+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "21273d2c", "release": true, "await": true}) -- 0x557396ca3500 con 0x557396dc8960 2024-05-11T13:35:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.439+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396dc8960 0x557396dcad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.440+0000 7f9ec9e00700 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396dc8960 0x557396dcad80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26afbfaa9de251c4 server_cookie=805788526411ace4 in_seq=0 out_seq=0 2024-05-11T13:35:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.440+0000 7f9eb4c00700 10 client.15564 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:29.456 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:29 smithi071 ceph-mon[33293]: pgmap v51: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 KiB/s wr, 16 op/s 2024-05-11T13:35:29.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.487+0000 7f9eb4c00700 1 -- 172.21.15.63:0/1419936055 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x557396ca3500 con 0x557396dc8960 2024-05-11T13:35:29.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.487+0000 7f9eb4c00700 10 client.15564 handle_command_reply: tid=0 2024-05-11T13:35:29.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.487+0000 7f9edbd04b80 1 client.15564 shutdown 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 msgr2=0x557396d559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d559e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00ebd0 tx=0x7f9ebc00ecb0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 shutdown_connections 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x557396dc8960 0x557396dcad80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557396d48fd0 0x557396d529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557396d435a0 0x557396d559e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 --2- 172.21.15.63:0/1419936055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557396d43f60 0x557396d55f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 >> 172.21.15.63:0/1419936055 conn(0x557396c71c60 msgr2=0x557396d42590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9eb4c00700 0 client.15564 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.488+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 shutdown_connections 2024-05-11T13:35:29.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.489+0000 7f9edbd04b80 1 -- 172.21.15.63:0/1419936055 wait complete. 2024-05-11T13:35:29.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.489+0000 7f9edbd04b80 20 client.15564 trim_cache size 0 max 16384 2024-05-11T13:35:29.488 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:35:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 33, 2024-05-11T13:35:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:35:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 4, 2024-05-11T13:35:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "21273d2c": { 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 4, 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 70.0, 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:35:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:35:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:35:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:29.490 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9eb00b29f0 msgr2=0x7f9eb00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9eb00b29f0 0x7f9eb00b4eb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8005fd0 tx=0x7f9eb8005ee0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc10b770 msgr2=0x7f9ecc19bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc10b770 0x7f9ecc19bab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4004dc0 tx=0x7f9ec4004ea0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 shutdown_connections 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9eb00b29f0 0x7f9eb00b4eb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc10b770 0x7f9ecc19bab0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc105770 0x7f9ecc1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 --2- 172.21.15.63:0/1829461855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ecc106130 0x7f9ecc1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.490+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 >> 172.21.15.63:0/1829461855 conn(0x7f9ecc101260 msgr2=0x7f9ecc1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.491+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 shutdown_connections 2024-05-11T13:35:29.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:29.491+0000 7f9ed1400700 1 -- 172.21.15.63:0/1829461855 wait complete. 2024-05-11T13:35:29.539 INFO:teuthology.orchestra.run.smithi063.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-11T13:35:29.546 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-11T13:35:30.506 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '21273d2c', seconds elapsed: 27.4 2024-05-11T13:35:31.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:31 smithi063 ceph-mon[29809]: pgmap v52: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-05-11T13:35:31.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:31 smithi071 ceph-mon[33293]: pgmap v52: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-05-11T13:35:31.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:31 smithi165 ceph-mon[33235]: pgmap v52: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/: 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:total 2 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:34 . 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:34 .. 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 data 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data: 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:total 45 2024-05-11T13:35:31.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 . 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:34 .. 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 11 13:34 base 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 11 13:34 global 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:35 log 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_commit_ts 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_dynshmem 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 11 13:34 pg_hba.conf 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 11 13:34 pg_ident.conf 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:34 pg_logical 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 pg_multixact 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_notify 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_replslot 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_serial 2024-05-11T13:35:31.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_snapshots 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_stat 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_stat_tmp 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 pg_subtrans 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_tblspc 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_twophase 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:34 pg_wal 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 pg_xact 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 11 13:34 postgresql.auto.conf 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 11 13:34 postgresql.conf 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base: 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:total 3 2024-05-11T13:35:31.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 11 13:34 . 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 1 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 13493 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 13494 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:total 7778 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 . 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 11 13:34 .. 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 112 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 113 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 1247 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1247_fsm 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1247_vm 2024-05-11T13:35:31.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 11 13:34 1249 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1249_fsm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1249_vm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 11 13:34 1255 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1255_fsm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1255_vm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 11 13:34 1259 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1259_fsm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1259_vm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 11 13:34 13344 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13344_fsm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13344_vm 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13346 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13348 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349 2024-05-11T13:35:31.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13349_fsm 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349_vm 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13351 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13353 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13354_fsm 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354_vm 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13356 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13358 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13359_fsm 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359_vm 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13361 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13363 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1417 2024-05-11T13:35:31.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1418 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 174 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 175 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2187 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2224 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2328 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2336 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2337 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2579 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2600 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2600_fsm 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2600_vm 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2601_fsm 2024-05-11T13:35:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601_vm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2602 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2602_fsm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2602_vm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2603 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2603_fsm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2603_vm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2604 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2605 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2605_fsm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2605_vm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2606_fsm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606_vm 2024-05-11T13:35:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2607 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2607_fsm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2607_vm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 11 13:34 2608 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2608_fsm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2608_vm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 11 13:34 2609 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2609_fsm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2609_vm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2610 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2610_fsm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2610_vm 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2611 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612 2024-05-11T13:35:31.692 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2612_fsm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612_vm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2613 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2615_fsm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615_vm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2616 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2616_fsm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2616_vm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 11 13:34 2617 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2617_fsm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2617_vm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 11 13:34 2618 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2618_fsm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2618_vm 2024-05-11T13:35:31.693 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 11 13:34 2619 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2619_fsm 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2619_vm 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2620 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2650 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2651 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2652 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2653 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2654 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2655 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2656 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2657 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 11 13:34 2658 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 2659 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2660 2024-05-11T13:35:31.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2661 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2662 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2663 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2664 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2665 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2666 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2667 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2668 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2669 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2670 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 11 13:34 2673 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 11 13:34 2674 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 11 13:34 2675 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2678 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2679 2024-05-11T13:35:31.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2680 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2681 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2682 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2683 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2684 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2685 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2686 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2687 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2688 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2689 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 11 13:34 2690 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 11 13:34 2691 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2692 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2693 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2696 2024-05-11T13:35:31.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2699 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2701 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2702 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2703 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2704 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2753 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2753_fsm 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2753_vm 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2754 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2755 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2756 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2757 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2830 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2831 2024-05-11T13:35:31.697 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2832 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2833 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2834 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2835 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2836 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2837 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 11 13:34 2838 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2838_fsm 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2838_vm 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2839 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840_fsm 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2840_vm 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2841 2024-05-11T13:35:31.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2995 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2996 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3079_fsm 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079_vm 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3080 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3081 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3085 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3118 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3119 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 3164 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3256 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3257 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3258 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3350 2024-05-11T13:35:31.699 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3351 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3379 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3380 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3381 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3394 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3394_fsm 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3394_vm 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3395 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3429 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3430 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3431 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3433 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3439 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3440 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3455 2024-05-11T13:35:31.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 11 13:34 3456 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3456_fsm 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3456_vm 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3466 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3467 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3468 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3501 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3502 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3503 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3534 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3541_fsm 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541_vm 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3542 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3574 2024-05-11T13:35:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3575 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3576 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3596 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3597 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3598 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3599 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3600_fsm 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600_vm 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3601_fsm 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601_vm 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602 2024-05-11T13:35:31.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3602_fsm 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602_vm 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603_fsm 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3603_vm 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3604 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3605 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3606 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3607 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3608 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 3609 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3712 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3764_fsm 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764_vm 2024-05-11T13:35:31.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3766 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3767 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3997 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4143 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4144 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4145 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4146 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4147 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4148 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4149 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4150 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4151 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4152 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4153 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4154 2024-05-11T13:35:31.704 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4155 2024-05-11T13:35:31.705 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4156 2024-05-11T13:35:31.705 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4157 2024-05-11T13:35:31.705 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4158 2024-05-11T13:35:33.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:33 smithi063 ceph-mon[29809]: pgmap v53: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-05-11T13:35:33.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:33 smithi071 ceph-mon[33293]: pgmap v53: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-05-11T13:35:33.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:33 smithi165 ceph-mon[33235]: pgmap v53: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-05-11T13:35:33.588 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4159 2024-05-11T13:35:33.588 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4160 2024-05-11T13:35:33.588 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4161 2024-05-11T13:35:33.588 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4162 2024-05-11T13:35:33.588 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4163 2024-05-11T13:35:33.588 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4164 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4165 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4166 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4167 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4168 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4169 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4170 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4171 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4172 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4173 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4174 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 5002 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 548 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 549 2024-05-11T13:35:33.589 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6102 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6104 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6106 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6110 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6111 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6112 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6113 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6117 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 826 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 827 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 828 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:35:33.590 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:total 7778 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 . 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 11 13:34 .. 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 112 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 113 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 1247 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1247_fsm 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1247_vm 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 11 13:34 1249 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1249_fsm 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1249_vm 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 11 13:34 1255 2024-05-11T13:35:33.591 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1255_fsm 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1255_vm 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 11 13:34 1259 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1259_fsm 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1259_vm 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 11 13:34 13344 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13344_fsm 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13344_vm 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13346 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13348 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349 2024-05-11T13:35:33.592 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13349_fsm 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349_vm 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13351 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13353 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13354_fsm 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354_vm 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13356 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13358 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13359_fsm 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359_vm 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13361 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13363 2024-05-11T13:35:33.593 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1417 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1418 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 174 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 175 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2187 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2224 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2328 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2336 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2337 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2579 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2600 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2600_fsm 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2600_vm 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601 2024-05-11T13:35:33.594 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2601_fsm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601_vm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2602 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2602_fsm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2602_vm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2603 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2603_fsm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2603_vm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2604 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2605 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2605_fsm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2605_vm 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606 2024-05-11T13:35:33.595 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2606_fsm 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606_vm 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2607 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2607_fsm 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2607_vm 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 11 13:34 2608 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2608_fsm 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2608_vm 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 11 13:34 2609 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2609_fsm 2024-05-11T13:35:33.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2609_vm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2610 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2610_fsm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2610_vm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2611 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2612_fsm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612_vm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2613 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2615_fsm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615_vm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2616 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2616_fsm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2616_vm 2024-05-11T13:35:33.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 11 13:34 2617 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2617_fsm 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2617_vm 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 11 13:34 2618 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2618_fsm 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2618_vm 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 11 13:34 2619 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2619_fsm 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2619_vm 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2620 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2650 2024-05-11T13:35:33.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2651 2024-05-11T13:35:33.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2652 2024-05-11T13:35:33.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2653 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2654 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2655 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2656 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2657 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 11 13:34 2658 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 2659 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2660 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2661 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2662 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2663 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2664 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2665 2024-05-11T13:35:33.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2666 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2667 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2668 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2669 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2670 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 11 13:34 2673 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 11 13:34 2674 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 11 13:34 2675 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2678 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2679 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2680 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2681 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2682 2024-05-11T13:35:33.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2683 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2684 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2685 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2686 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2687 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2688 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2689 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 11 13:34 2690 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 11 13:34 2691 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2692 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2693 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2696 2024-05-11T13:35:33.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2699 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2701 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2702 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2703 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2704 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2753 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2753_fsm 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2753_vm 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2754 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2755 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2756 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2757 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2830 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2831 2024-05-11T13:35:33.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2832 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2833 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2834 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2835 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2836 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2837 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 11 13:34 2838 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2838_fsm 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2838_vm 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2839 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840_fsm 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2840_vm 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2841 2024-05-11T13:35:33.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2995 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2996 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3079_fsm 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079_vm 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3080 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3081 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3085 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3118 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3119 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 3164 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3256 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3257 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3258 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3350 2024-05-11T13:35:33.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3351 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3379 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3380 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3381 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3394 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3394_fsm 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3394_vm 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3395 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3429 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3430 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3431 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3433 2024-05-11T13:35:33.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3439 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3440 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3455 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 11 13:34 3456 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3456_fsm 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3456_vm 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3466 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3467 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3468 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3501 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3502 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3503 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3534 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3541_fsm 2024-05-11T13:35:33.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541_vm 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3542 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3574 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3575 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3576 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3596 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3597 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3598 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3599 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3600_fsm 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600_vm 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3601_fsm 2024-05-11T13:35:33.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601_vm 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3602_fsm 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602_vm 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603_fsm 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3603_vm 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3604 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3605 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3606 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3607 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3608 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 3609 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3712 2024-05-11T13:35:33.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3764_fsm 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764_vm 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3766 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3767 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3997 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4143 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4144 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4145 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4146 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4147 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4148 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4149 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4150 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4151 2024-05-11T13:35:33.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4152 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4153 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4154 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4155 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4156 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4157 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4158 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4159 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4160 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4161 2024-05-11T13:35:33.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4162 2024-05-11T13:35:35.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:35 smithi063 ceph-mon[29809]: pgmap v54: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-11T13:35:35.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:35 smithi071 ceph-mon[33293]: pgmap v54: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-11T13:35:35.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:35 smithi165 ceph-mon[33235]: pgmap v54: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4163 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4164 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4165 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4166 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4167 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4168 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4169 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4170 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4171 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4172 2024-05-11T13:35:35.870 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4173 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4174 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 5002 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 548 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 549 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6102 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6104 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6106 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6110 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6111 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6112 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6113 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6117 2024-05-11T13:35:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 826 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 827 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 828 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:total 7778 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 . 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 11 13:34 .. 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 112 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 113 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 1247 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1247_fsm 2024-05-11T13:35:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1247_vm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 11 13:34 1249 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1249_fsm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1249_vm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 11 13:34 1255 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1255_fsm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1255_vm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 11 13:34 1259 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1259_fsm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1259_vm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 11 13:34 13344 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13344_fsm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13344_vm 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13346 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13348 2024-05-11T13:35:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13349_fsm 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349_vm 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13351 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13353 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13354_fsm 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354_vm 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13356 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13358 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13359_fsm 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359_vm 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13361 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13363 2024-05-11T13:35:35.874 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1417 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1418 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 174 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 175 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2187 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2224 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2328 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2336 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2337 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2579 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2600 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2600_fsm 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2600_vm 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2601_fsm 2024-05-11T13:35:35.875 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601_vm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2602 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2602_fsm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2602_vm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2603 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2603_fsm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2603_vm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2604 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2605 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2605_fsm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2605_vm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2606_fsm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606_vm 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2607 2024-05-11T13:35:35.876 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2607_fsm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2607_vm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 11 13:34 2608 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2608_fsm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2608_vm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 11 13:34 2609 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2609_fsm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2609_vm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2610 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2610_fsm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2610_vm 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2611 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612 2024-05-11T13:35:35.877 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2612_fsm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612_vm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2613 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2615_fsm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615_vm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2616 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2616_fsm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2616_vm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 11 13:34 2617 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2617_fsm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2617_vm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 11 13:34 2618 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2618_fsm 2024-05-11T13:35:35.878 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2618_vm 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 11 13:34 2619 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2619_fsm 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2619_vm 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2620 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2650 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2651 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2652 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2653 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2654 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2655 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2656 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2657 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 11 13:34 2658 2024-05-11T13:35:35.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 2659 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2660 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2661 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2662 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2663 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2664 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2665 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2666 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2667 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2668 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2669 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2670 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 11 13:34 2673 2024-05-11T13:35:35.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 11 13:34 2674 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 11 13:34 2675 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2678 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2679 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2680 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2681 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2682 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2683 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2684 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2685 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2686 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2687 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2688 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2689 2024-05-11T13:35:35.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 11 13:34 2690 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 11 13:34 2691 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2692 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2693 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2696 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2699 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2701 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2702 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2703 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2704 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2753 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2753_fsm 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2753_vm 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2754 2024-05-11T13:35:35.882 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2755 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2756 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2757 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2830 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2831 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2832 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2833 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2834 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2835 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2836 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2837 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 11 13:34 2838 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2838_fsm 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2838_vm 2024-05-11T13:35:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2839 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840_fsm 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2840_vm 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2841 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2995 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2996 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3079_fsm 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079_vm 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3080 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3081 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3085 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3118 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3119 2024-05-11T13:35:35.884 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 3164 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3256 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3257 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3258 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3350 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3351 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3379 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3380 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3381 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3394 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3394_fsm 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3394_vm 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3395 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3429 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3430 2024-05-11T13:35:35.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3431 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3433 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3439 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3440 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3455 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 11 13:34 3456 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3456_fsm 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3456_vm 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3466 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3467 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3468 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3501 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3502 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3503 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3534 2024-05-11T13:35:35.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3541_fsm 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541_vm 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3542 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3574 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3575 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3576 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3596 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3597 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3598 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3599 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3600_fsm 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600_vm 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601 2024-05-11T13:35:35.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3601_fsm 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601_vm 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3602_fsm 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602_vm 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603_fsm 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3603_vm 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3604 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3605 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3606 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3607 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3608 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 3609 2024-05-11T13:35:35.888 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3712 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3764_fsm 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764_vm 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3766 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3767 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3997 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4143 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4144 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4145 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4146 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4147 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4148 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4149 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4150 2024-05-11T13:35:35.889 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4151 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4152 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4153 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4154 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4155 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4156 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4157 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4158 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4159 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4160 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4161 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4162 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4163 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4164 2024-05-11T13:35:35.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4165 2024-05-11T13:35:35.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4166 2024-05-11T13:35:35.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4167 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4168 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4169 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4170 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4171 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4172 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4173 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4174 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 5002 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 548 2024-05-11T13:35:36.209 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 549 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6102 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6104 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6106 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6110 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6111 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6112 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6113 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6117 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 826 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 827 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 828 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:35:36.210 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/global: 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:total 538 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 11 13:34 . 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1213 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1213_fsm 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1213_vm 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1214 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1214_fsm 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1214_vm 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 1232 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 1233 2024-05-11T13:35:36.211 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1260 2024-05-11T13:35:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1260_fsm 2024-05-11T13:35:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1260_vm 2024-05-11T13:35:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1261 2024-05-11T13:35:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1261_fsm 2024-05-11T13:35:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1261_vm 2024-05-11T13:35:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1262 2024-05-11T13:35:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1262_fsm 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1262_vm 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2396 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2396_fsm 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2396_vm 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2397 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2671 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2672 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2676 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2677 2024-05-11T13:35:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2694 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2695 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2697 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2698 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2846 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2847 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2964 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2965 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2966 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2967 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3592 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3593 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4060 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4061 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4175 2024-05-11T13:35:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4176 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4177 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4178 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4181 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4182 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4183 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4184 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4185 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4186 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6000 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6001 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6002 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6100 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6114 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6115 2024-05-11T13:35:36.226 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 pg_control 2024-05-11T13:35:36.227 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:35:36.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.227 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/log: 2024-05-11T13:35:36.227 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.227 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:35 . 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-11T13:35:36.508 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:total 3 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:34 . 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 mappings 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:34 replorigin_checkpoint 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 snapshots 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.509 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:34 .. 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:34 .. 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:total 2 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 . 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 members 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 offsets 2024-05-11T13:35:36.510 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:total 9 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 . 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 .. 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 0000 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:total 9 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 . 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 .. 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 0000 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-11T13:35:36.511 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.512 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.513 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:total 9 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 . 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 0000 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.612 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:total 16386 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:34 . 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:34 000000010000000000000001 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 archive_status 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-11T13:35:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:34 .. 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:total 9 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 . 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 11 13:35 .. 2024-05-11T13:35:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 0000 2024-05-11T13:35:36.621 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-11T13:35:37.324 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:37 smithi063 ceph-mon[29809]: pgmap v55: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-11T13:35:37.324 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:37 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:37.334 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-11T13:35:37.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:37 smithi071 ceph-mon[33293]: pgmap v55: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-11T13:35:37.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:37 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:37.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:37 smithi165 ceph-mon[33235]: pgmap v55: 129 pgs: 129 active+clean; 52 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-11T13:35:37.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:37 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:dropping old tables... 2024-05-11T13:35:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-11T13:35:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-11T13:35:37.759 INFO:teuthology.orchestra.run.smithi063.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-11T13:35:37.759 INFO:teuthology.orchestra.run.smithi063.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-11T13:35:37.759 INFO:teuthology.orchestra.run.smithi063.stderr:creating tables... 2024-05-11T13:35:38.109 INFO:teuthology.orchestra.run.smithi063.stderr:generating data (client-side)... 2024-05-11T13:35:39.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:39 smithi063 ceph-mon[29809]: pgmap v56: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-05-11T13:35:39.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:39 smithi071 ceph-mon[33293]: pgmap v56: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-05-11T13:35:39.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:39 smithi165 ceph-mon[33235]: pgmap v56: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-05-11T13:35:40.957 INFO:teuthology.orchestra.run.smithi063.stderr:100000 of 3200000 tuples (3%) done (elapsed 2.55 s, remaining 79.17 s) 2024-05-11T13:35:41.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:41 smithi063 ceph-mon[29809]: pgmap v57: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-05-11T13:35:41.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:41 smithi071 ceph-mon[33293]: pgmap v57: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-05-11T13:35:41.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:41 smithi165 ceph-mon[33235]: pgmap v57: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-05-11T13:35:42.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:42 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:42.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:42 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:42.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:42 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:43.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:43 smithi063 ceph-mon[29809]: pgmap v58: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-05-11T13:35:43.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:43 smithi071 ceph-mon[33293]: pgmap v58: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-05-11T13:35:43.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:43 smithi165 ceph-mon[33235]: pgmap v58: 129 pgs: 129 active+clean; 52 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 18 op/s 2024-05-11T13:35:44.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:44 smithi165 ceph-mon[33235]: pgmap v59: 129 pgs: 129 active+clean; 96 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.0 MiB/s wr, 31 op/s 2024-05-11T13:35:44.681 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:44 smithi063 ceph-mon[29809]: pgmap v59: 129 pgs: 129 active+clean; 96 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.0 MiB/s wr, 31 op/s 2024-05-11T13:35:44.700 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:44 smithi071 ceph-mon[33293]: pgmap v59: 129 pgs: 129 active+clean; 96 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.0 MiB/s wr, 31 op/s 2024-05-11T13:35:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:200000 of 3200000 tuples (6%) done (elapsed 6.82 s, remaining 102.26 s) 2024-05-11T13:35:47.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:47 smithi063 ceph-mon[29809]: pgmap v60: 129 pgs: 129 active+clean; 96 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-05-11T13:35:47.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:47 smithi071 ceph-mon[33293]: pgmap v60: 129 pgs: 129 active+clean; 96 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-05-11T13:35:47.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:47 smithi165 ceph-mon[33235]: pgmap v60: 129 pgs: 129 active+clean; 96 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-05-11T13:35:49.079 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-11T13:35:49.080 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-05-11T13:35:49.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:35:49.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:49 smithi063 ceph-mon[29809]: pgmap v61: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.3 MiB/s wr, 36 op/s 2024-05-11T13:35:49.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:49 smithi071 ceph-mon[33293]: pgmap v61: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.3 MiB/s wr, 36 op/s 2024-05-11T13:35:49.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:49 smithi165 ceph-mon[33235]: pgmap v61: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.3 MiB/s wr, 36 op/s 2024-05-11T13:35:49.608 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:35:49.950 INFO:teuthology.orchestra.run.smithi063.stderr:300000 of 3200000 tuples (9%) done (elapsed 11.55 s, remaining 111.61 s) 2024-05-11T13:35:51.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:51 smithi063 ceph-mon[29809]: pgmap v62: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.2 MiB/s wr, 23 op/s 2024-05-11T13:35:51.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:51 smithi071 ceph-mon[33293]: pgmap v62: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.2 MiB/s wr, 23 op/s 2024-05-11T13:35:51.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:51 smithi165 ceph-mon[33235]: pgmap v62: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.2 MiB/s wr, 23 op/s 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.056+0000 7fab8ca00700 1 -- 172.21.15.63:0/628418880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 msgr2=0x7fab88109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.056+0000 7fab8ca00700 1 --2- 172.21.15.63:0/628418880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab88109920 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fab780099c0 tx=0x7fab78009cd0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.056+0000 7fab8ca00700 1 -- 172.21.15.63:0/628418880 shutdown_connections 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.056+0000 7fab8ca00700 1 --2- 172.21.15.63:0/628418880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab88103e50 0x7fab88110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.056+0000 7fab8ca00700 1 --2- 172.21.15.63:0/628418880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab88109920 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.056+0000 7fab8ca00700 1 --2- 172.21.15.63:0/628418880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab88103490 0x7fab88103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.056+0000 7fab8ca00700 1 -- 172.21.15.63:0/628418880 >> 172.21.15.63:0/628418880 conn(0x7fab880fef60 msgr2=0x7fab88101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.057+0000 7fab8ca00700 1 -- 172.21.15.63:0/628418880 shutdown_connections 2024-05-11T13:35:52.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.057+0000 7fab8ca00700 1 -- 172.21.15.63:0/628418880 wait complete. 2024-05-11T13:35:52.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.058+0000 7fab8ca00700 1 Processor -- start 2024-05-11T13:35:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.058+0000 7fab8ca00700 1 -- start start 2024-05-11T13:35:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab8ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab88103490 0x7fab881a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab8ca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab88103e50 0x7fab881a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab8ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab8819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab8ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fab88113520 con 0x7fab88109540 2024-05-11T13:35:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab8ca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab881133a0 con 0x7fab88103490 2024-05-11T13:35:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab8ca00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab881136a0 con 0x7fab88103e50 2024-05-11T13:35:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab87200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab8819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab87200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab8819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52188/0 (socket says 172.21.15.63:52188) 2024-05-11T13:35:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab87200700 1 -- 172.21.15.63:0/14183424 learned_addr learned my addr 172.21.15.63:0/14183424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab86800700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab88103490 0x7fab881a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.059+0000 7fab85e00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab88103e50 0x7fab881a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab87200700 1 -- 172.21.15.63:0/14183424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab88103e50 msgr2=0x7fab881a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab87200700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab88103e50 0x7fab881a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab87200700 1 -- 172.21.15.63:0/14183424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab88103490 msgr2=0x7fab881a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab87200700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab88103490 0x7fab881a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab87200700 1 -- 172.21.15.63:0/14183424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8819c120 con 0x7fab88109540 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab85e00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab88103e50 0x7fab881a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab87200700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab8819b9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fab80007d70 tx=0x7fab8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.060+0000 7fab7f400700 1 -- 172.21.15.63:0/14183424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80016020 con 0x7fab88109540 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.061+0000 7fab7f400700 1 -- 172.21.15.63:0/14183424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab8000ad40 con 0x7fab88109540 2024-05-11T13:35:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.061+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8819c410 con 0x7fab88109540 2024-05-11T13:35:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.061+0000 7fab7f400700 1 -- 172.21.15.63:0/14183424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab800115e0 con 0x7fab88109540 2024-05-11T13:35:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.061+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab881a86d0 con 0x7fab88109540 2024-05-11T13:35:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.063+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8806a240 con 0x7fab88109540 2024-05-11T13:35:52.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.065+0000 7fab7f400700 1 -- 172.21.15.63:0/14183424 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fab80019020 con 0x7fab88109540 2024-05-11T13:35:52.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.067+0000 7fab7f400700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fab6c0b2a70 0x7fab6c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:52.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.067+0000 7fab7f400700 1 -- 172.21.15.63:0/14183424 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fab800e2750 con 0x7fab88109540 2024-05-11T13:35:52.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.068+0000 7fab86800700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fab6c0b2a70 0x7fab6c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:52.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.069+0000 7fab86800700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fab6c0b2a70 0x7fab6c0b4f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab78009990 tx=0x7fab78005fb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:52.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.073+0000 7fab7f400700 1 -- 172.21.15.63:0/14183424 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab80020020 con 0x7fab88109540 2024-05-11T13:35:52.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.401+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab881a8960 con 0x7fab88109540 2024-05-11T13:35:52.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.403+0000 7fab7f400700 1 -- 172.21.15.63:0/14183424 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11956 (secure 0 0 0) 0x7fab800a9eb0 con 0x7fab88109540 2024-05-11T13:35:52.400 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:35:52.400 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:35:40.705654+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34451,34472,34427,34403,24472]},"id":1}]} 2024-05-11T13:35:52.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.406+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fab6c0b2a70 msgr2=0x7fab6c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:52.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.406+0000 7fab8ca00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fab6c0b2a70 0x7fab6c0b4f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab78009990 tx=0x7fab78005fb0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 msgr2=0x7fab8819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:52.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab8819b9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fab80007d70 tx=0x7fab8000a340 comp rx=0 tx=0).stop 2024-05-11T13:35:52.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 shutdown_connections 2024-05-11T13:35:52.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fab6c0b2a70 0x7fab6c0b4f30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab88109540 0x7fab8819b9c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab88103490 0x7fab881a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 --2- 172.21.15.63:0/14183424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab88103e50 0x7fab881a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:52.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.407+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 >> 172.21.15.63:0/14183424 conn(0x7fab880fef60 msgr2=0x7fab88100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:52.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.409+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 shutdown_connections 2024-05-11T13:35:52.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:52.409+0000 7fab8ca00700 1 -- 172.21.15.63:0/14183424 wait complete. 2024-05-11T13:35:52.407 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-11T13:35:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:400000 of 3200000 tuples (12%) done (elapsed 14.34 s, remaining 100.38 s) 2024-05-11T13:35:53.366 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:53 smithi063 ceph-mon[29809]: pgmap v63: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.2 MiB/s wr, 22 op/s 2024-05-11T13:35:53.366 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:53 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/14183424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:53.370 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0' 2024-05-11T13:35:53.370 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0 2024-05-11T13:35:53.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:53 smithi071 ceph-mon[33293]: pgmap v63: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.2 MiB/s wr, 22 op/s 2024-05-11T13:35:53.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/14183424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:53.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:53 smithi165 ceph-mon[33235]: pgmap v63: 129 pgs: 129 active+clean; 107 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.2 MiB/s wr, 22 op/s 2024-05-11T13:35:53.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:53 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/14183424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:35:53.889 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:35:55.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:55 smithi063 ceph-mon[29809]: pgmap v64: 129 pgs: 129 active+clean; 143 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 32 op/s 2024-05-11T13:35:55.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:55 smithi071 ceph-mon[33293]: pgmap v64: 129 pgs: 129 active+clean; 143 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 32 op/s 2024-05-11T13:35:55.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:55 smithi165 ceph-mon[33235]: pgmap v64: 129 pgs: 129 active+clean; 143 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 32 op/s 2024-05-11T13:35:56.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 -- 172.21.15.63:0/2016539899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74281078c0 msgr2=0x7f7428112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 --2- 172.21.15.63:0/2016539899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74281078c0 0x7f7428112550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f742000b0a0 tx=0x7f742000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 -- 172.21.15.63:0/2016539899 shutdown_connections 2024-05-11T13:35:56.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 --2- 172.21.15.63:0/2016539899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74281078c0 0x7f7428112550 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 --2- 172.21.15.63:0/2016539899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74281058f0 0x7f7428105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 --2- 172.21.15.63:0/2016539899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428106210 0x7f7428107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 -- 172.21.15.63:0/2016539899 >> 172.21.15.63:0/2016539899 conn(0x7f7428101060 msgr2=0x7f7428103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 -- 172.21.15.63:0/2016539899 shutdown_connections 2024-05-11T13:35:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.416+0000 7f742ca00700 1 -- 172.21.15.63:0/2016539899 wait complete. 2024-05-11T13:35:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 Processor -- start 2024-05-11T13:35:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 -- start start 2024-05-11T13:35:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74281058f0 0x7f74281a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428106210 0x7f74281a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74281078c0 0x7f742819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7428115720 con 0x7f7428106210 2024-05-11T13:35:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74281155a0 con 0x7f74281058f0 2024-05-11T13:35:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.418+0000 7f742ca00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f74281158a0 con 0x7f74281078c0 2024-05-11T13:35:56.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428106210 0x7f74281a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428106210 0x7f74281a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52218/0 (socket says 172.21.15.63:52218) 2024-05-11T13:35:56.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 -- 172.21.15.63:0/3370266926 learned_addr learned my addr 172.21.15.63:0/3370266926 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:56.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7427e00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74281078c0 0x7f742819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7427400700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74281058f0 0x7f74281a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 -- 172.21.15.63:0/3370266926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74281078c0 msgr2=0x7f742819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74281078c0 0x7f742819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 -- 172.21.15.63:0/3370266926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74281058f0 msgr2=0x7f74281a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74281058f0 0x7f74281a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.419+0000 7f7426a00700 1 -- 172.21.15.63:0/3370266926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742819c100 con 0x7f7428106210 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.420+0000 7f7426a00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428106210 0x7f74281a1cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f741400d980 tx=0x7f741400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.420+0000 7f741fe00700 1 -- 172.21.15.63:0/3370266926 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414017070 con 0x7f7428106210 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.420+0000 7f741fe00700 1 -- 172.21.15.63:0/3370266926 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7414009dc0 con 0x7f7428106210 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.420+0000 7f741fe00700 1 -- 172.21.15.63:0/3370266926 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741400f3c0 con 0x7f7428106210 2024-05-11T13:35:56.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.420+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742819c390 con 0x7f7428106210 2024-05-11T13:35:56.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.420+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74281a8660 con 0x7f7428106210 2024-05-11T13:35:56.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.426+0000 7f741fe00700 1 -- 172.21.15.63:0/3370266926 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7414009930 con 0x7f7428106210 2024-05-11T13:35:56.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 Processor -- start 2024-05-11T13:35:56.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 -- start start 2024-05-11T13:35:56.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70ba9f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baa00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e70b9f1bb0 con 0x55e70baa5100 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e70b98f8f0 con 0x55e70ba9fc50 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.427+0000 7f74389ccb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e70b9edb70 con 0x55e70baa5a20 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f7427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f7427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52232/0 (socket says 172.21.15.63:52232) 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f7427e00700 1 -- 172.21.15.63:0/1308985791 learned_addr learned my addr 172.21.15.63:0/1308985791 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f741fe00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f740c0b2b20 0x7f740c0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f7427400700 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70ba9f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f7426a00700 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baa00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f741fe00700 1 -- 172.21.15.63:0/3370266926 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f74140e2460 con 0x7f7428106210 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.428+0000 7f7427e00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f740c0b2b20 0x7f740c0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.429+0000 7f7427e00700 1 -- 172.21.15.63:0/1308985791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 msgr2=0x55e70ba9f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.429+0000 7f7427e00700 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70ba9f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.429+0000 7f7427e00700 1 -- 172.21.15.63:0/1308985791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 msgr2=0x55e70baa00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.429+0000 7f7427e00700 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baa00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.429+0000 7f7427e00700 1 -- 172.21.15.63:0/1308985791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70b93ea60 con 0x55e70baa5100 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.429+0000 7f7427e00700 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa54e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7420004b10 tx=0x7f7420004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac797a5404c97ff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.429+0000 7f741cc00700 1 -- 172.21.15.63:0/1308985791 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200079c0 con 0x55e70baa5100 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.430+0000 7f741cc00700 1 -- 172.21.15.63:0/1308985791 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7420007b60 con 0x55e70baa5100 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.430+0000 7f741cc00700 1 -- 172.21.15.63:0/1308985791 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742001edb0 con 0x55e70baa5100 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.430+0000 7f7427e00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f740c0b2b20 0x7f740c0b4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f742002c000 tx=0x7f7420029000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 -- 172.21.15.63:0/1308985791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 msgr2=0x55e70baa54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa54e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7420004b10 tx=0x7f7420004b40 comp rx=0 tx=0).stop 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 -- 172.21.15.63:0/1308985791 shutdown_connections 2024-05-11T13:35:56.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baa00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa54e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 --2- 172.21.15.63:0/1308985791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70ba9f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 -- 172.21.15.63:0/1308985791 >> 172.21.15.63:0/1308985791 conn(0x55e70b9cdcf0 msgr2=0x55e70ba9a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:56.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 -- 172.21.15.63:0/1308985791 shutdown_connections 2024-05-11T13:35:56.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.431+0000 7f74389ccb80 1 -- 172.21.15.63:0/1308985791 wait complete. 2024-05-11T13:35:56.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.432+0000 7f74389ccb80 1 -- start start 2024-05-11T13:35:56.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.432+0000 7f74389ccb80 1 Processor -- start 2024-05-11T13:35:56.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70baa8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f74389ccb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e70baac540 con 0x55e70baa5100 2024-05-11T13:35:56.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f74389ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e70b957cb0 con 0x55e70ba9fc50 2024-05-11T13:35:56.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f74389ccb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e70baac6c0 con 0x55e70baa5a20 2024-05-11T13:35:56.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f74389ccb80 10 client.0 _ll_register_callbacks cb 0x55e70b998e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:35:56.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52234/0 (socket says 172.21.15.63:52234) 2024-05-11T13:35:56.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.433+0000 7f7426a00700 1 -- 172.21.15.63:0/4195071641 learned_addr learned my addr 172.21.15.63:0/4195071641 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:56.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7427400700 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7427e00700 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70baa8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7426a00700 1 -- 172.21.15.63:0/4195071641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 msgr2=0x55e70baa8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7426a00700 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70baa8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7426a00700 1 -- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 msgr2=0x55e70baab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7426a00700 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7426a00700 1 -- 172.21.15.63:0/4195071641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70b9b75b0 con 0x55e70baa5100 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7426a00700 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa8ab0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74140a6620 tx=0x7f7414072c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1efdb6aac9faffe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7411600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.434+0000 7f7411600700 1 -- 172.21.15.63:0/4195071641 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414020040 con 0x55e70baa5100 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.435+0000 7f7411600700 1 -- 172.21.15.63:0/4195071641 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f7414010460 con 0x55e70baa5100 2024-05-11T13:35:56.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.435+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7420007b60 con 0x55e70baa5100 2024-05-11T13:35:56.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.435+0000 7f7411600700 1 -- 172.21.15.63:0/4195071641 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f74140a8ec0 con 0x55e70baa5100 2024-05-11T13:35:56.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.435+0000 7f74389ccb80 10 client.15582 fetch_fsmap learned FSMap version 35 2024-05-11T13:35:56.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.435+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e70bab6d40 con 0x55e70baa5100 2024-05-11T13:35:56.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.435+0000 7f7411600700 1 -- 172.21.15.63:0/4195071641 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7420007b60 con 0x55e70baa5100 2024-05-11T13:35:56.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.435+0000 7f7411600700 1 -- 172.21.15.63:0/4195071641 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8410+0+0 (secure 0 0 0) 0x7f74140e2120 con 0x55e70baa5100 2024-05-11T13:35:56.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.436+0000 7f74389ccb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 35 2024-05-11T13:35:56.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.436+0000 7f74389ccb80 10 client.15582 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:35:56.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.436+0000 7f74389ccb80 20 client.15582 populate_metadata read hostname 'smithi063' 2024-05-11T13:35:56.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.436+0000 7f74389ccb80 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bab92d0 0x55e70babb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.436+0000 7f74389ccb80 4 client.15582 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:35:56.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.436+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e70ba6c340 con 0x55e70bab92d0 2024-05-11T13:35:56.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.436+0000 7f7427400700 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bab92d0 0x55e70babb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.437+0000 7f7427400700 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bab92d0 0x55e70babb730 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf3b549b437b527 server_cookie=3fe4532cce90d6d8 in_seq=0 out_seq=0 2024-05-11T13:35:56.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.437+0000 7f7411600700 10 client.15582 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:56.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.439+0000 7f7411600700 1 -- 172.21.15.63:0/4195071641 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e70ba6c340 con 0x55e70bab92d0 2024-05-11T13:35:56.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.439+0000 7f7411600700 10 client.15582 handle_command_reply: tid=0 2024-05-11T13:35:56.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.439+0000 7f74389ccb80 1 client.15582 shutdown 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 msgr2=0x55e70baa8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa8ab0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74140a6620 tx=0x7f7414072c50 comp rx=0 tx=0).stop 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 shutdown_connections 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70baa5a20 0x55e70baa8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bab92d0 0x55e70babb730 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70ba9fc50 0x55e70baab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 --2- 172.21.15.63:0/4195071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baa8ab0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f7411600700 0 client.15582 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 >> 172.21.15.63:0/4195071641 conn(0x55e70b9cdcf0 msgr2=0x55e70ba9e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:56.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 shutdown_connections 2024-05-11T13:35:56.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 1 -- 172.21.15.63:0/4195071641 wait complete. 2024-05-11T13:35:56.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.440+0000 7f74389ccb80 20 client.15582 trim_cache size 0 max 16384 2024-05-11T13:35:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:500000 of 3200000 tuples (15%) done (elapsed 18.05 s, remaining 97.48 s) 2024-05-11T13:35:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.470+0000 7f74389ccb80 1 Processor -- start 2024-05-11T13:35:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.470+0000 7f74389ccb80 1 -- start start 2024-05-11T13:35:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.470+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 0x55e70baa8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.470+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5100 0x55e70bab55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5a20 0x55e70baa73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f74389ccb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b14a0 con 0x55e70baa5a20 2024-05-11T13:35:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f74389ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b3220 con 0x55e70baa5100 2024-05-11T13:35:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f74389ccb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b1c30 con 0x55e70ba9fc50 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f7427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5a20 0x55e70baa73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f7427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5a20 0x55e70baa73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52246/0 (socket says 172.21.15.63:52246) 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f7427e00700 1 -- 172.21.15.63:0/3521474850 learned_addr learned my addr 172.21.15.63:0/3521474850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.471+0000 7f7426a00700 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5100 0x55e70bab55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.472+0000 7f7427e00700 1 -- 172.21.15.63:0/3521474850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 msgr2=0x55e70baa8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.472+0000 7f7427e00700 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 0x55e70baa8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.472+0000 7f7427e00700 1 -- 172.21.15.63:0/3521474850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5100 msgr2=0x55e70bab55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.472+0000 7f7427e00700 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5100 0x55e70bab55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.472+0000 7f7427e00700 1 -- 172.21.15.63:0/3521474850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70babbdb0 con 0x55e70baa5a20 2024-05-11T13:35:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.472+0000 7f7427e00700 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5a20 0x55e70baa73a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f74200095f0 tx=0x7f7420009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55c5aca4f2af752 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.472+0000 7f7413e00700 1 -- 172.21.15.63:0/3521474850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420038020 con 0x55e70baa5a20 2024-05-11T13:35:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.473+0000 7f7413e00700 1 -- 172.21.15.63:0/3521474850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f742001ed90 con 0x55e70baa5a20 2024-05-11T13:35:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.473+0000 7f7413e00700 1 -- 172.21.15.63:0/3521474850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742001e8f0 con 0x55e70baa5a20 2024-05-11T13:35:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.473+0000 7f74389ccb80 1 -- 172.21.15.63:0/3521474850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5a20 msgr2=0x55e70baa73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.473+0000 7f74389ccb80 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5a20 0x55e70baa73a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f74200095f0 tx=0x7f7420009040 comp rx=0 tx=0).stop 2024-05-11T13:35:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 -- 172.21.15.63:0/3521474850 shutdown_connections 2024-05-11T13:35:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5a20 0x55e70baa73a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 0x55e70baa8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 --2- 172.21.15.63:0/3521474850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5100 0x55e70bab55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 -- 172.21.15.63:0/3521474850 >> 172.21.15.63:0/3521474850 conn(0x55e70b9cdcf0 msgr2=0x55e70ba9ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:56.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 -- 172.21.15.63:0/3521474850 shutdown_connections 2024-05-11T13:35:56.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 -- 172.21.15.63:0/3521474850 wait complete. 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 -- start start 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.474+0000 7f74389ccb80 1 Processor -- start 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 0x55e70baaf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baaf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f74389ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5a20 0x55e70bab1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f74389ccb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b2a90 con 0x55e70baa5100 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f74389ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b23c0 con 0x55e70baa5a20 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f74389ccb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f74281af440 con 0x55e70ba9fc50 2024-05-11T13:35:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f74389ccb80 10 client.0 _ll_register_callbacks cb 0x55e70b998e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:35:56.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baaf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baaf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52260/0 (socket says 172.21.15.63:52260) 2024-05-11T13:35:56.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.475+0000 7f7426a00700 1 -- 172.21.15.63:0/2711584025 learned_addr learned my addr 172.21.15.63:0/2711584025 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:35:56.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7427400700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 0x55e70baaf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7427e00700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5a20 0x55e70bab1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7426a00700 1 -- 172.21.15.63:0/2711584025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 msgr2=0x55e70baaf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7426a00700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 0x55e70baaf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7426a00700 1 -- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5a20 msgr2=0x55e70bab1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7426a00700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5a20 0x55e70bab1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7426a00700 1 -- 172.21.15.63:0/2711584025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70bab1b80 con 0x55e70baa5100 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7427e00700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5a20 0x55e70bab1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7426a00700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baaf620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74140a9340 tx=0x7f74140a89f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cc86b4160db61a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7411600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.476+0000 7f7411600700 1 -- 172.21.15.63:0/2711584025 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414020070 con 0x55e70baa5100 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f7411600700 1 -- 172.21.15.63:0/2711584025 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f74140a59a0 con 0x55e70baa5100 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e70b9ff590 con 0x55e70baa5100 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f7411600700 1 -- 172.21.15.63:0/2711584025 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f74140a9ab0 con 0x55e70baa5100 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f74389ccb80 10 client.15588 fetch_fsmap learned FSMap version 35 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e70bb257a0 con 0x55e70baa5100 2024-05-11T13:35:56.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f7411600700 1 -- 172.21.15.63:0/2711584025 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e70b9ff590 con 0x55e70baa5100 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f7411600700 1 -- 172.21.15.63:0/2711584025 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8410+0+0 (secure 0 0 0) 0x7f74140a3ad0 con 0x55e70baa5100 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f74389ccb80 10 client.15588 fetch_fsmap finished waiting for FSMap version 35 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.477+0000 7f74389ccb80 10 client.15588 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.478+0000 7f74389ccb80 20 client.15588 populate_metadata read hostname 'smithi063' 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.478+0000 7f74389ccb80 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bb25d00 0x55e70bb28120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.478+0000 7f74389ccb80 4 client.15588 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}] 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.478+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}) -- 0x55e70b9f9630 con 0x55e70bb25d00 2024-05-11T13:35:56.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.478+0000 7f7427400700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bb25d00 0x55e70bb28120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:35:56.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.479+0000 7f7427400700 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bb25d00 0x55e70bb28120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b833f551d7189 server_cookie=769657133605e806 in_seq=0 out_seq=0 2024-05-11T13:35:56.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.479+0000 7f7411600700 10 client.15588 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:56.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.837+0000 7f7411600700 1 -- 172.21.15.63:0/2711584025 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55e70b9f9630 con 0x55e70bb25d00 2024-05-11T13:35:56.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.837+0000 7f7411600700 10 client.15588 handle_command_reply: tid=0 2024-05-11T13:35:56.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.837+0000 7f74389ccb80 1 client.15588 shutdown 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.837+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 msgr2=0x55e70baaf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.837+0000 7f74389ccb80 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baaf620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74140a9340 tx=0x7f74140a89f0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 shutdown_connections 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e70bb25d00 0x55e70bb28120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e70baa5a20 0x55e70bab1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e70ba9fc50 0x55e70baaf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 --2- 172.21.15.63:0/2711584025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e70baa5100 0x55e70baaf620 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f7411600700 0 client.15588 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 >> 172.21.15.63:0/2711584025 conn(0x55e70b9cdcf0 msgr2=0x55e70baa5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:56.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 shutdown_connections 2024-05-11T13:35:56.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.838+0000 7f74389ccb80 1 -- 172.21.15.63:0/2711584025 wait complete. 2024-05-11T13:35:56.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.839+0000 7f74389ccb80 20 client.15588 trim_cache size 0 max 16384 2024-05-11T13:35:56.837 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:35:56.837 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 35, 2024-05-11T13:35:56.837 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:35:56.837 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 6, 2024-05-11T13:35:56.837 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "c68aee52": { 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 6, 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.6, 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:35:56.838 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:35:56.839 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:35:56.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:56.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:56.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:56.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:56.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:35:56.839 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:35:56.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.840+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f740c0b2b20 msgr2=0x7f740c0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.840+0000 7f742ca00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f740c0b2b20 0x7f740c0b4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f742002c000 tx=0x7f7420029000 comp rx=0 tx=0).stop 2024-05-11T13:35:56.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.840+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428106210 msgr2=0x7f74281a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:35:56.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.840+0000 7f742ca00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428106210 0x7f74281a1cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f741400d980 tx=0x7f741400dc90 comp rx=0 tx=0).stop 2024-05-11T13:35:56.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.840+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 shutdown_connections 2024-05-11T13:35:56.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.841+0000 7f742ca00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f740c0b2b20 0x7f740c0b4fe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.841+0000 7f742ca00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74281078c0 0x7f742819b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.841+0000 7f742ca00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74281058f0 0x7f74281a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.841+0000 7f742ca00700 1 --2- 172.21.15.63:0/3370266926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428106210 0x7f74281a1cd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:35:56.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.841+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 >> 172.21.15.63:0/3370266926 conn(0x7f7428101060 msgr2=0x7f7428101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:35:56.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.841+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 shutdown_connections 2024-05-11T13:35:56.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:35:56.841+0000 7f742ca00700 1 -- 172.21.15.63:0/3370266926 wait complete. 2024-05-11T13:35:57.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:57 smithi063 ceph-mon[29809]: pgmap v65: 129 pgs: 129 active+clean; 143 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.8 MiB/s wr, 19 op/s 2024-05-11T13:35:57.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:57 smithi071 ceph-mon[33293]: pgmap v65: 129 pgs: 129 active+clean; 143 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.8 MiB/s wr, 19 op/s 2024-05-11T13:35:57.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:57 smithi165 ceph-mon[33235]: pgmap v65: 129 pgs: 129 active+clean; 143 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.8 MiB/s wr, 19 op/s 2024-05-11T13:35:57.740 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c68aee52', quiesce took 8.7 seconds. Will release after: 25.6 2024-05-11T13:35:58.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:58 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:58.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:58 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:58.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:58 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:35:59.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:35:59 smithi063 ceph-mon[29809]: pgmap v66: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-05-11T13:35:59.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:35:59 smithi071 ceph-mon[33293]: pgmap v66: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-05-11T13:35:59.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:35:59 smithi165 ceph-mon[33235]: pgmap v66: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-05-11T13:36:01.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:01 smithi063 ceph-mon[29809]: pgmap v67: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 19 op/s 2024-05-11T13:36:01.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:01 smithi071 ceph-mon[33293]: pgmap v67: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 19 op/s 2024-05-11T13:36:01.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:01 smithi165 ceph-mon[33235]: pgmap v67: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 19 op/s 2024-05-11T13:36:03.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:03 smithi063 ceph-mon[29809]: pgmap v68: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 19 op/s 2024-05-11T13:36:03.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:03 smithi071 ceph-mon[33293]: pgmap v68: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 19 op/s 2024-05-11T13:36:03.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:03 smithi165 ceph-mon[33235]: pgmap v68: 129 pgs: 129 active+clean; 171 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 19 op/s 2024-05-11T13:36:04.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:04 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:36:04.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:04 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:36:04.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:04 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:36:05.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:05 smithi063 ceph-mon[29809]: pgmap v69: 129 pgs: 129 active+clean; 174 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 39 op/s 2024-05-11T13:36:05.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:05 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:36:05.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:05 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:36:05.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:05 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:36:05.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:05 smithi071 ceph-mon[33293]: pgmap v69: 129 pgs: 129 active+clean; 174 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 39 op/s 2024-05-11T13:36:05.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:05 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:36:05.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:05 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:36:05.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:05 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:36:05.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:05 smithi165 ceph-mon[33235]: pgmap v69: 129 pgs: 129 active+clean; 174 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 39 op/s 2024-05-11T13:36:05.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:05 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:36:05.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:05 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:36:05.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:05 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:36:07.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:07 smithi063 ceph-mon[29809]: pgmap v70: 129 pgs: 129 active+clean; 174 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-05-11T13:36:07.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:07 smithi071 ceph-mon[33293]: pgmap v70: 129 pgs: 129 active+clean; 174 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-05-11T13:36:07.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:07 smithi165 ceph-mon[33235]: pgmap v70: 129 pgs: 129 active+clean; 174 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-05-11T13:36:09.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:09 smithi063 ceph-mon[29809]: pgmap v71: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 34 op/s 2024-05-11T13:36:09.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:09 smithi071 ceph-mon[33293]: pgmap v71: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 34 op/s 2024-05-11T13:36:09.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:09 smithi165 ceph-mon[33235]: pgmap v71: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 34 op/s 2024-05-11T13:36:11.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:11 smithi071 ceph-mon[33293]: pgmap v72: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 637 KiB/s wr, 24 op/s 2024-05-11T13:36:11.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:11 smithi165 ceph-mon[33235]: pgmap v72: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 637 KiB/s wr, 24 op/s 2024-05-11T13:36:11.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:11 smithi063 ceph-mon[29809]: pgmap v72: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 637 KiB/s wr, 24 op/s 2024-05-11T13:36:12.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:12 smithi165 ceph-mon[33235]: pgmap v73: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 637 KiB/s wr, 24 op/s 2024-05-11T13:36:12.681 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:12 smithi063 ceph-mon[29809]: pgmap v73: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 637 KiB/s wr, 24 op/s 2024-05-11T13:36:12.700 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:12 smithi071 ceph-mon[33293]: pgmap v73: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 637 KiB/s wr, 24 op/s 2024-05-11T13:36:15.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:15 smithi063 ceph-mon[29809]: pgmap v74: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 639 KiB/s wr, 29 op/s 2024-05-11T13:36:15.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:15 smithi071 ceph-mon[33293]: pgmap v74: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 639 KiB/s wr, 29 op/s 2024-05-11T13:36:15.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:15 smithi165 ceph-mon[33235]: pgmap v74: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 639 KiB/s wr, 29 op/s 2024-05-11T13:36:17.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:17 smithi063 ceph-mon[29809]: pgmap v75: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-05-11T13:36:17.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:17 smithi071 ceph-mon[33293]: pgmap v75: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-05-11T13:36:17.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:17 smithi165 ceph-mon[33235]: pgmap v75: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-05-11T13:36:19.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:19 smithi063 ceph-mon[29809]: pgmap v76: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-05-11T13:36:19.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:19 smithi071 ceph-mon[33293]: pgmap v76: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-05-11T13:36:19.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:19 smithi165 ceph-mon[33235]: pgmap v76: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-05-11T13:36:21.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:21 smithi063 ceph-mon[29809]: pgmap v77: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-11T13:36:21.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:21 smithi071 ceph-mon[33293]: pgmap v77: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-11T13:36:21.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:21 smithi165 ceph-mon[33235]: pgmap v77: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-11T13:36:23.341 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:36:23.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:23 smithi063 ceph-mon[29809]: pgmap v78: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-11T13:36:23.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:23 smithi071 ceph-mon[33293]: pgmap v78: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-11T13:36:23.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:23 smithi165 ceph-mon[33235]: pgmap v78: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-11T13:36:23.811 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:36:25.181 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:25 smithi063 ceph-mon[29809]: pgmap v79: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-05-11T13:36:25.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:25 smithi071 ceph-mon[33293]: pgmap v79: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-05-11T13:36:25.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:25 smithi165 ceph-mon[33235]: pgmap v79: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-05-11T13:36:26.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 -- 172.21.15.63:0/885318175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5406bbf0 msgr2=0x7f2d541103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:26.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 --2- 172.21.15.63:0/885318175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5406bbf0 0x7f2d541103e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00b0a0 tx=0x7f2d4c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 -- 172.21.15.63:0/885318175 shutdown_connections 2024-05-11T13:36:26.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 --2- 172.21.15.63:0/885318175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5406bbf0 0x7f2d541103e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 --2- 172.21.15.63:0/885318175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d541095c0 0x7f2d541099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 --2- 172.21.15.63:0/885318175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d5406b230 0x7f2d5406b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 -- 172.21.15.63:0/885318175 >> 172.21.15.63:0/885318175 conn(0x7f2d54079400 msgr2=0x7f2d54079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:26.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 -- 172.21.15.63:0/885318175 shutdown_connections 2024-05-11T13:36:26.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.249+0000 7f2d5be00700 1 -- 172.21.15.63:0/885318175 wait complete. 2024-05-11T13:36:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.250+0000 7f2d5be00700 1 Processor -- start 2024-05-11T13:36:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d5be00700 1 -- start start 2024-05-11T13:36:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d5be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d5406b230 0x7f2d5419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d5be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d5406bbf0 0x7f2d5419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d5be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d541095c0 0x7f2d541a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d5be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541135b0 con 0x7f2d541095c0 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d5be00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54113430 con 0x7f2d5406bbf0 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d5be00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54113730 con 0x7f2d5406b230 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.251+0000 7f2d59e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d541095c0 0x7f2d541a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d541095c0 0x7f2d541a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38626/0 (socket says 172.21.15.63:38626) 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 -- 172.21.15.63:0/558343396 learned_addr learned my addr 172.21.15.63:0/558343396 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59400700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d5406b230 0x7f2d5419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d58a00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d5406bbf0 0x7f2d5419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 -- 172.21.15.63:0/558343396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d5406b230 msgr2=0x7f2d5419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d5406b230 0x7f2d5419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 -- 172.21.15.63:0/558343396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d5406bbf0 msgr2=0x7f2d5419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d5406bbf0 0x7f2d5419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 -- 172.21.15.63:0/558343396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d541a0d30 con 0x7f2d541095c0 2024-05-11T13:36:26.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59400700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d5406b230 0x7f2d5419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:36:26.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.252+0000 7f2d59e00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d541095c0 0x7f2d541a05f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00b830 tx=0x7f2d4c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:26.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.253+0000 7f2d52000700 1 -- 172.21.15.63:0/558343396 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c021070 con 0x7f2d541095c0 2024-05-11T13:36:26.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.253+0000 7f2d52000700 1 -- 172.21.15.63:0/558343396 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d4c007c80 con 0x7f2d541095c0 2024-05-11T13:36:26.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.253+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d541a0fc0 con 0x7f2d541095c0 2024-05-11T13:36:26.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.253+0000 7f2d52000700 1 -- 172.21.15.63:0/558343396 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c005260 con 0x7f2d541095c0 2024-05-11T13:36:26.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.253+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d541a88a0 con 0x7f2d541095c0 2024-05-11T13:36:26.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.254+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5406a240 con 0x7f2d541095c0 2024-05-11T13:36:26.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.257+0000 7f2d52000700 1 -- 172.21.15.63:0/558343396 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2d4c024070 con 0x7f2d541095c0 2024-05-11T13:36:26.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.258+0000 7f2d52000700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2d3c0b2a70 0x7f2d3c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:26.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.259+0000 7f2d52000700 1 -- 172.21.15.63:0/558343396 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f2d4c00f020 con 0x7f2d541095c0 2024-05-11T13:36:26.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.259+0000 7f2d59400700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2d3c0b2a70 0x7f2d3c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:26.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.260+0000 7f2d59400700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2d3c0b2a70 0x7f2d3c0b4f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005e10 tx=0x7f2d48005d60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.262+0000 7f2d52000700 1 -- 172.21.15.63:0/558343396 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d4c02b020 con 0x7f2d541095c0 2024-05-11T13:36:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.553+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d541a8a70 con 0x7f2d541095c0 2024-05-11T13:36:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.555+0000 7f2d52000700 1 -- 172.21.15.63:0/558343396 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11959 (secure 0 0 0) 0x7f2d4c0abae0 con 0x7f2d541095c0 2024-05-11T13:36:26.551 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:36:26.551 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:35:56.457541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:36:26.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2d3c0b2a70 msgr2=0x7f2d3c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:26.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2d3c0b2a70 0x7f2d3c0b4f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005e10 tx=0x7f2d48005d60 comp rx=0 tx=0).stop 2024-05-11T13:36:26.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d541095c0 msgr2=0x7f2d541a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:26.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d541095c0 0x7f2d541a05f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00b830 tx=0x7f2d4c004db0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 shutdown_connections 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2d3c0b2a70 0x7f2d3c0b4f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d541095c0 0x7f2d541a05f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d5406b230 0x7f2d5419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 --2- 172.21.15.63:0/558343396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d5406bbf0 0x7f2d5419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.558+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 >> 172.21.15.63:0/558343396 conn(0x7f2d54079400 msgr2=0x7f2d54100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.559+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 shutdown_connections 2024-05-11T13:36:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:26.559+0000 7f2d5be00700 1 -- 172.21.15.63:0/558343396 wait complete. 2024-05-11T13:36:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 36 2024-05-11T13:36:27.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:27 smithi063 ceph-mon[29809]: pgmap v80: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-11T13:36:27.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:27 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/558343396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:36:27.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:27 smithi071 ceph-mon[33293]: pgmap v80: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-11T13:36:27.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:27 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/558343396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:36:27.506 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id c68aee52 --release --await' 2024-05-11T13:36:27.506 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id c68aee52 --release --await 2024-05-11T13:36:27.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:27 smithi165 ceph-mon[33235]: pgmap v80: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-11T13:36:27.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:27 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/558343396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:36:27.981 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:36:29.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:29 smithi063 ceph-mon[29809]: pgmap v81: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-11T13:36:29.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:29 smithi071 ceph-mon[33293]: pgmap v81: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-11T13:36:29.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:29 smithi165 ceph-mon[33235]: pgmap v81: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-11T13:36:30.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 -- 172.21.15.63:0/766864036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103460 msgr2=0x7fd6281038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 --2- 172.21.15.63:0/766864036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103460 0x7fd6281038e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd610009990 tx=0x7fd610009ca0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 -- 172.21.15.63:0/766864036 shutdown_connections 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 --2- 172.21.15.63:0/766864036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628103e20 0x7fd628110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 --2- 172.21.15.63:0/766864036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6281094f0 0x7fd6281098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 --2- 172.21.15.63:0/766864036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103460 0x7fd6281038e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 -- 172.21.15.63:0/766864036 >> 172.21.15.63:0/766864036 conn(0x7fd6280fef70 msgr2=0x7fd628101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 -- 172.21.15.63:0/766864036 shutdown_connections 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.496+0000 7fd62cc00700 1 -- 172.21.15.63:0/766864036 wait complete. 2024-05-11T13:36:30.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.497+0000 7fd62cc00700 1 Processor -- start 2024-05-11T13:36:30.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.498+0000 7fd62cc00700 1 -- start start 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.498+0000 7fd62cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628103460 0x7fd628075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.498+0000 7fd62cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103e20 0x7fd628075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.498+0000 7fd62cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6281094f0 0x7fd6280762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.498+0000 7fd62cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd628113530 con 0x7fd628103e20 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.498+0000 7fd62cc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281133b0 con 0x7fd6281094f0 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.498+0000 7fd62cc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281136b0 con 0x7fd628103460 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103e20 0x7fd628075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103e20 0x7fd628075d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40610/0 (socket says 172.21.15.63:40610) 2024-05-11T13:36:30.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 -- 172.21.15.63:0/3104209827 learned_addr learned my addr 172.21.15.63:0/3104209827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:30.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd627400700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628103460 0x7fd628075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd627e00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6281094f0 0x7fd6280762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 -- 172.21.15.63:0/3104209827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628103460 msgr2=0x7fd628075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628103460 0x7fd628075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 -- 172.21.15.63:0/3104209827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6281094f0 msgr2=0x7fd6280762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6281094f0 0x7fd6280762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd61fe00700 1 -- 172.21.15.63:0/3104209827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281a7fc0 con 0x7fd628103e20 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.499+0000 7fd627400700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628103460 0x7fd628075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.500+0000 7fd61fe00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103e20 0x7fd628075d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd610005910 tx=0x7fd6100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.500+0000 7fd624c00700 1 -- 172.21.15.63:0/3104209827 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610027070 con 0x7fd628103e20 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.500+0000 7fd624c00700 1 -- 172.21.15.63:0/3104209827 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd61000fc00 con 0x7fd628103e20 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.500+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281a8250 con 0x7fd628103e20 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.500+0000 7fd624c00700 1 -- 172.21.15.63:0/3104209827 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610022d40 con 0x7fd628103e20 2024-05-11T13:36:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.500+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281a8700 con 0x7fd628103e20 2024-05-11T13:36:30.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.504+0000 7fd624c00700 1 -- 172.21.15.63:0/3104209827 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd61002a070 con 0x7fd628103e20 2024-05-11T13:36:30.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.505+0000 7fd638ab7b80 1 Processor -- start 2024-05-11T13:36:30.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.505+0000 7fd638ab7b80 1 -- start start 2024-05-11T13:36:30.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd827010 0x561edd827490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd8279d0 0x561edd833a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561edd770b20 con 0x561edd827010 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561edd70e860 con 0x561edd8279d0 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561edd76cae0 con 0x561edd82ccb0 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd627400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd827010 0x561edd827490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd627400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd827010 0x561edd827490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40620/0 (socket says 172.21.15.63:40620) 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd627400700 1 -- 172.21.15.63:0/1642726727 learned_addr learned my addr 172.21.15.63:0/1642726727 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.506+0000 7fd624c00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd60c0b2a70 0x7fd60c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd61fe00700 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd8279d0 0x561edd833a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627e00700 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627400700 1 -- 172.21.15.63:0/1642726727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 msgr2=0x561edd82d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627400700 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627400700 1 -- 172.21.15.63:0/1642726727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd8279d0 msgr2=0x561edd833a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627400700 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd8279d0 0x561edd833a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627400700 1 -- 172.21.15.63:0/1642726727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561edd6bd9d0 con 0x561edd827010 2024-05-11T13:36:30.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627e00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd60c0b2a70 0x7fd60c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd624c00700 1 -- 172.21.15.63:0/3104209827 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd6100e3720 con 0x7fd628103e20 2024-05-11T13:36:30.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.507+0000 7fd627400700 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd827010 0x561edd827490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd61800bd10 tx=0x7fd61800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c13bf1ef883e37a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:30.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.508+0000 7fd61cc00700 1 -- 172.21.15.63:0/1642726727 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800be60 con 0x561edd827010 2024-05-11T13:36:30.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.508+0000 7fd61cc00700 1 -- 172.21.15.63:0/1642726727 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd61800fe50 con 0x561edd827010 2024-05-11T13:36:30.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.508+0000 7fd61cc00700 1 -- 172.21.15.63:0/1642726727 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180106a0 con 0x561edd827010 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1642726727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd827010 msgr2=0x561edd827490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd627e00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd60c0b2a70 0x7fd60c0b4f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd620009550 tx=0x7fd62000b040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd827010 0x561edd827490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd61800bd10 tx=0x7fd61800f640 comp rx=0 tx=0).stop 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1642726727 shutdown_connections 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd8279d0 0x561edd833a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1642726727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd827010 0x561edd827490 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1642726727 >> 172.21.15.63:0/1642726727 conn(0x561edd74cc60 msgr2=0x561edd819480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1642726727 shutdown_connections 2024-05-11T13:36:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.509+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1642726727 wait complete. 2024-05-11T13:36:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.510+0000 7fd638ab7b80 1 -- start start 2024-05-11T13:36:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.510+0000 7fd638ab7b80 1 Processor -- start 2024-05-11T13:36:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd833650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd830a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd830f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561edd834010 con 0x561edd8279d0 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561edd6d6c20 con 0x561edd827010 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561edd834190 con 0x561edd82ccb0 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd638ab7b80 10 client.0 _ll_register_callbacks cb 0x561edd704480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd830a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd830a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40628/0 (socket says 172.21.15.63:40628) 2024-05-11T13:36:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.511+0000 7fd61fe00700 1 -- 172.21.15.63:0/2839009350 learned_addr learned my addr 172.21.15.63:0/2839009350 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd627400700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd833650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd627e00700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd830f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd61fe00700 1 -- 172.21.15.63:0/2839009350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 msgr2=0x561edd830f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd61fe00700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd830f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd61fe00700 1 -- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 msgr2=0x561edd833650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd61fe00700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd833650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd61fe00700 1 -- 172.21.15.63:0/2839009350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561edd736520 con 0x561edd8279d0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd627400700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd833650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd627e00700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd830f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd61fe00700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd830a40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd6100e0ae0 tx=0x7fd610073f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e01bf2b515d7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd615600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd615600700 1 -- 172.21.15.63:0/2839009350 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6100aa1b0 con 0x561edd8279d0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd615600700 1 -- 172.21.15.63:0/2839009350 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd610022470 con 0x561edd8279d0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.512+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61800fe50 con 0x561edd8279d0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd615600700 1 -- 172.21.15.63:0/2839009350 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd61007a400 con 0x561edd8279d0 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd638ab7b80 10 client.15606 fetch_fsmap learned FSMap version 36 2024-05-11T13:36:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561edd83e7b0 con 0x561edd8279d0 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd615600700 1 -- 172.21.15.63:0/2839009350 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd61800fe50 con 0x561edd8279d0 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd615600700 1 -- 172.21.15.63:0/2839009350 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8418+0+0 (secure 0 0 0) 0x7fd6100afbe0 con 0x561edd8279d0 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd638ab7b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 36 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd638ab7b80 10 client.15606 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd638ab7b80 20 client.15606 populate_metadata read hostname 'smithi063' 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.513+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd840cd0 0x561edd843110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.514+0000 7fd638ab7b80 4 client.15606 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.514+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561edd7eb160 con 0x561edd840cd0 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.514+0000 7fd627400700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd840cd0 0x561edd843110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.515+0000 7fd627400700 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd840cd0 0x561edd843110 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2ba974fa568e81 server_cookie=e2478603824441c1 in_seq=0 out_seq=0 2024-05-11T13:36:30.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.515+0000 7fd615600700 10 client.15606 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:30.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.517+0000 7fd615600700 1 -- 172.21.15.63:0/2839009350 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561edd7eb160 con 0x561edd840cd0 2024-05-11T13:36:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.517+0000 7fd615600700 10 client.15606 handle_command_reply: tid=0 2024-05-11T13:36:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.517+0000 7fd638ab7b80 1 client.15606 shutdown 2024-05-11T13:36:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 msgr2=0x561edd830a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd830a40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd6100e0ae0 tx=0x7fd610073f40 comp rx=0 tx=0).stop 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 shutdown_connections 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd830f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd840cd0 0x561edd843110 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd833650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2839009350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd830a40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd615600700 0 client.15606 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 >> 172.21.15.63:0/2839009350 conn(0x561edd74cc60 msgr2=0x561edd826000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 shutdown_connections 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2839009350 wait complete. 2024-05-11T13:36:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.518+0000 7fd638ab7b80 20 client.15606 trim_cache size 0 max 16384 2024-05-11T13:36:30.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.540+0000 7fd638ab7b80 1 Processor -- start 2024-05-11T13:36:30.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.540+0000 7fd638ab7b80 1 -- start start 2024-05-11T13:36:30.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd83d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd82c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281acff0 con 0x561edd8279d0 2024-05-11T13:36:30.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281aedd0 con 0x561edd827010 2024-05-11T13:36:30.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281ad780 con 0x561edd82ccb0 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd82c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd82c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40634/0 (socket says 172.21.15.63:40634) 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.541+0000 7fd61fe00700 1 -- 172.21.15.63:0/2274498401 learned_addr learned my addr 172.21.15.63:0/2274498401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd627e00700 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd627400700 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd83d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd61fe00700 1 -- 172.21.15.63:0/2274498401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 msgr2=0x561edd82fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd61fe00700 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd61fe00700 1 -- 172.21.15.63:0/2274498401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 msgr2=0x561edd83d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd61fe00700 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd83d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd61fe00700 1 -- 172.21.15.63:0/2274498401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561edd843790 con 0x561edd8279d0 2024-05-11T13:36:30.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd61fe00700 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd82c200 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd6100aa3a0 tx=0x7fd610005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51535a0029275e04 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:30.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd617e00700 1 -- 172.21.15.63:0/2274498401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6100aa6a0 con 0x561edd8279d0 2024-05-11T13:36:30.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd617e00700 1 -- 172.21.15.63:0/2274498401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6100aa1b0 con 0x561edd8279d0 2024-05-11T13:36:30.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.542+0000 7fd617e00700 1 -- 172.21.15.63:0/2274498401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6100a9e10 con 0x561edd8279d0 2024-05-11T13:36:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2274498401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 msgr2=0x561edd82c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd82c200 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd6100aa3a0 tx=0x7fd610005ac0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2274498401 shutdown_connections 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd82fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd83d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/2274498401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd82c200 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2274498401 >> 172.21.15.63:0/2274498401 conn(0x561edd74cc60 msgr2=0x561edd82f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2274498401 shutdown_connections 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.543+0000 7fd638ab7b80 1 -- 172.21.15.63:0/2274498401 wait complete. 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.544+0000 7fd638ab7b80 1 -- start start 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.544+0000 7fd638ab7b80 1 Processor -- start 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.544+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd8393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd839920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd638ab7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd8363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281ae640 con 0x561edd8279d0 2024-05-11T13:36:30.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281adf10 con 0x561edd827010 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd638ab7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281aafb0 con 0x561edd82ccb0 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd638ab7b80 10 client.0 _ll_register_callbacks cb 0x561edd704480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd839920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd839920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40642/0 (socket says 172.21.15.63:40642) 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 -- 172.21.15.63:0/1687091197 learned_addr learned my addr 172.21.15.63:0/1687091197 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd627400700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd8393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd627e00700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd8363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 -- 172.21.15.63:0/1687091197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 msgr2=0x561edd8363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd8363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 -- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 msgr2=0x561edd8393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd8393e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd61fe00700 1 -- 172.21.15.63:0/1687091197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561edd815d70 con 0x561edd8279d0 2024-05-11T13:36:30.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.545+0000 7fd627e00700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd8363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd61fe00700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd839920 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd61009fd20 tx=0x7fd6100aa120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699e9c26b1c28b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd615600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd615600700 1 -- 172.21.15.63:0/1687091197 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6100a9e10 con 0x561edd8279d0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6100aa1b0 con 0x561edd8279d0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd615600700 1 -- 172.21.15.63:0/1687091197 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd6100a6c90 con 0x561edd8279d0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd615600700 1 -- 172.21.15.63:0/1687091197 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd6100aada0 con 0x561edd8279d0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd638ab7b80 10 client.15612 fetch_fsmap learned FSMap version 36 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.546+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561edd815d70 con 0x561edd8279d0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd615600700 1 -- 172.21.15.63:0/1687091197 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6100aa1b0 con 0x561edd8279d0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd615600700 1 -- 172.21.15.63:0/1687091197 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8418+0+0 (secure 0 0 0) 0x7fd610090bc0 con 0x561edd8279d0 2024-05-11T13:36:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd638ab7b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 36 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd638ab7b80 10 client.15612 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd638ab7b80 20 client.15612 populate_metadata read hostname 'smithi063' 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd8abf20 0x561edd8ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd638ab7b80 4 client.15612 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c68aee52", "release": true, "await": true}] 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.547+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c68aee52", "release": true, "await": true}) -- 0x561edd77e500 con 0x561edd8abf20 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.548+0000 7fd627400700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd8abf20 0x561edd8ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.549+0000 7fd627400700 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd8abf20 0x561edd8ae340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98dc5c38f2e63bd1 server_cookie=f81801f2fff3b9dc in_seq=0 out_seq=0 2024-05-11T13:36:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.549+0000 7fd615600700 10 client.15612 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.614+0000 7fd615600700 1 -- 172.21.15.63:0/1687091197 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x561edd77e500 con 0x561edd8abf20 2024-05-11T13:36:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.614+0000 7fd615600700 10 client.15612 handle_command_reply: tid=0 2024-05-11T13:36:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.614+0000 7fd638ab7b80 1 client.15612 shutdown 2024-05-11T13:36:30.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 msgr2=0x561edd839920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd839920 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd61009fd20 tx=0x7fd6100aa120 comp rx=0 tx=0).stop 2024-05-11T13:36:30.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 shutdown_connections 2024-05-11T13:36:30.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561edd82ccb0 0x561edd8363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561edd8abf20 0x561edd8ae340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561edd827010 0x561edd8393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 --2- 172.21.15.63:0/1687091197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561edd8279d0 0x561edd839920 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 >> 172.21.15.63:0/1687091197 conn(0x561edd74cc60 msgr2=0x561edd82afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.615+0000 7fd615600700 0 client.15612 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.616+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 shutdown_connections 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.616+0000 7fd638ab7b80 1 -- 172.21.15.63:0/1687091197 wait complete. 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.616+0000 7fd638ab7b80 20 client.15612 trim_cache size 0 max 16384 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 36, 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:36:30.614 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 8, 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "c68aee52": { 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 8, 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.6, 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:36:30.615 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:36:30.616 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:36:30.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:30.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:30.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:30.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:30.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:30.616 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.617+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd60c0b2a70 msgr2=0x7fd60c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.617+0000 7fd62cc00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd60c0b2a70 0x7fd60c0b4f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd620009550 tx=0x7fd62000b040 comp rx=0 tx=0).stop 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.617+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103e20 msgr2=0x7fd628075d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.617+0000 7fd62cc00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103e20 0x7fd628075d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd610005910 tx=0x7fd6100047b0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.618+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 shutdown_connections 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.618+0000 7fd62cc00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd60c0b2a70 0x7fd60c0b4f30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.618+0000 7fd62cc00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6281094f0 0x7fd6280762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.618+0000 7fd62cc00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628103460 0x7fd628075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.618+0000 7fd62cc00700 1 --2- 172.21.15.63:0/3104209827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd628103e20 0x7fd628075d90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:30.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.618+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 >> 172.21.15.63:0/3104209827 conn(0x7fd6280fef70 msgr2=0x7fd6280ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:30.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.618+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 shutdown_connections 2024-05-11T13:36:30.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:30.619+0000 7fd62cc00700 1 -- 172.21.15.63:0/3104209827 wait complete. 2024-05-11T13:36:31.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:31 smithi063 ceph-mon[29809]: pgmap v82: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:36:31.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:31 smithi071 ceph-mon[33293]: pgmap v82: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:36:31.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:31 smithi165 ceph-mon[33235]: pgmap v82: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:36:31.620 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c68aee52', seconds elapsed: 42.5 2024-05-11T13:36:31.620 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-05-11T13:36:31.620 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:36:32.137 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:36:33.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:33 smithi063 ceph-mon[29809]: pgmap v83: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:36:33.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:33 smithi071 ceph-mon[33293]: pgmap v83: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:36:33.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:33 smithi165 ceph-mon[33235]: pgmap v83: 129 pgs: 129 active+clean; 174 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:36:33.675 INFO:teuthology.orchestra.run.smithi063.stderr:600000 of 3200000 tuples (18%) done (elapsed 55.27 s, remaining 239.51 s) 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 -- 172.21.15.63:0/2776991292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c10b790 msgr2=0x7fcb6c10bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 --2- 172.21.15.63:0/2776991292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c10b790 0x7fcb6c10bb70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000b0a0 tx=0x7fcb6000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 -- 172.21.15.63:0/2776991292 shutdown_connections 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 --2- 172.21.15.63:0/2776991292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb6c106150 0x7fcb6c112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 --2- 172.21.15.63:0/2776991292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c10b790 0x7fcb6c10bb70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 --2- 172.21.15.63:0/2776991292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c105790 0x7fcb6c105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 -- 172.21.15.63:0/2776991292 >> 172.21.15.63:0/2776991292 conn(0x7fcb6c101280 msgr2=0x7fcb6c1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 -- 172.21.15.63:0/2776991292 shutdown_connections 2024-05-11T13:36:34.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.479+0000 7fcb73c00700 1 -- 172.21.15.63:0/2776991292 wait complete. 2024-05-11T13:36:34.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 Processor -- start 2024-05-11T13:36:34.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 -- start start 2024-05-11T13:36:34.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb6c105790 0x7fcb6c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:34.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c106150 0x7fcb6c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:34.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c10b790 0x7fcb6c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:34.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c115870 con 0x7fcb6c106150 2024-05-11T13:36:34.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c1156f0 con 0x7fcb6c10b790 2024-05-11T13:36:34.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.481+0000 7fcb73c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c1159f0 con 0x7fcb6c105790 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c106150 0x7fcb6c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c106150 0x7fcb6c19c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40658/0 (socket says 172.21.15.63:40658) 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 -- 172.21.15.63:0/3081050818 learned_addr learned my addr 172.21.15.63:0/3081050818 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb71200700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb6c105790 0x7fcb6c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb71c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c10b790 0x7fcb6c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 -- 172.21.15.63:0/3081050818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb6c105790 msgr2=0x7fcb6c19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb6c105790 0x7fcb6c19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 -- 172.21.15.63:0/3081050818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c10b790 msgr2=0x7fcb6c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c10b790 0x7fcb6c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb6be00700 1 -- 172.21.15.63:0/3081050818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c1a0db0 con 0x7fcb6c106150 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.482+0000 7fcb71c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c10b790 0x7fcb6c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:36:34.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.483+0000 7fcb6be00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c106150 0x7fcb6c19c2e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800e9a0 tx=0x7fcb5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:34.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.483+0000 7fcb69600700 1 -- 172.21.15.63:0/3081050818 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800cbd0 con 0x7fcb6c106150 2024-05-11T13:36:34.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.483+0000 7fcb69600700 1 -- 172.21.15.63:0/3081050818 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb5800cd70 con 0x7fcb6c106150 2024-05-11T13:36:34.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.483+0000 7fcb69600700 1 -- 172.21.15.63:0/3081050818 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58018680 con 0x7fcb6c106150 2024-05-11T13:36:34.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.483+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c1a1040 con 0x7fcb6c106150 2024-05-11T13:36:34.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.483+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c1a8920 con 0x7fcb6c106150 2024-05-11T13:36:34.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.485+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6c06a240 con 0x7fcb6c106150 2024-05-11T13:36:34.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.490+0000 7fcb69600700 1 -- 172.21.15.63:0/3081050818 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcb58010430 con 0x7fcb6c106150 2024-05-11T13:36:34.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.492+0000 7fcb69600700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb540b2a70 0x7fcb540b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:34.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.492+0000 7fcb69600700 1 -- 172.21.15.63:0/3081050818 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fcb580e2fd0 con 0x7fcb6c106150 2024-05-11T13:36:34.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.492+0000 7fcb71200700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb540b2a70 0x7fcb540b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:34.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.493+0000 7fcb69600700 1 -- 172.21.15.63:0/3081050818 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb580aa6d0 con 0x7fcb6c106150 2024-05-11T13:36:34.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.493+0000 7fcb71200700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb540b2a70 0x7fcb540b4f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009fd0 tx=0x7fcb60006210 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:34.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.808+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb6c02e570 con 0x7fcb6c106150 2024-05-11T13:36:34.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.809+0000 7fcb69600700 1 -- 172.21.15.63:0/3081050818 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11959 (secure 0 0 0) 0x7fcb580a9fd0 con 0x7fcb6c106150 2024-05-11T13:36:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:36:34.807 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:35:56.457541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:36:34.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb540b2a70 msgr2=0x7fcb540b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:34.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb540b2a70 0x7fcb540b4f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009fd0 tx=0x7fcb60006210 comp rx=0 tx=0).stop 2024-05-11T13:36:34.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c106150 msgr2=0x7fcb6c19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:34.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c106150 0x7fcb6c19c2e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800e9a0 tx=0x7fcb5800ecb0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 shutdown_connections 2024-05-11T13:36:34.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcb540b2a70 0x7fcb540b4f30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c10b790 0x7fcb6c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb6c105790 0x7fcb6c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 --2- 172.21.15.63:0/3081050818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb6c106150 0x7fcb6c19c2e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:34.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 >> 172.21.15.63:0/3081050818 conn(0x7fcb6c101280 msgr2=0x7fcb6c101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:34.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 shutdown_connections 2024-05-11T13:36:34.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:34.813+0000 7fcb73c00700 1 -- 172.21.15.63:0/3081050818 wait complete. 2024-05-11T13:36:34.812 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 36 2024-05-11T13:36:35.419 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:35 smithi063 ceph-mon[29809]: pgmap v84: 129 pgs: 129 active+clean; 199 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 MiB/s wr, 13 op/s 2024-05-11T13:36:35.419 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:35 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3081050818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:36:35.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:35 smithi071 ceph-mon[33293]: pgmap v84: 129 pgs: 129 active+clean; 199 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 MiB/s wr, 13 op/s 2024-05-11T13:36:35.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:35 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3081050818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:36:35.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:35 smithi165 ceph-mon[33235]: pgmap v84: 129 pgs: 129 active+clean; 199 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 MiB/s wr, 13 op/s 2024-05-11T13:36:35.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:35 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3081050818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:36:35.945 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0' 2024-05-11T13:36:35.946 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0 2024-05-11T13:36:36.461 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:36:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:700000 of 3200000 tuples (21%) done (elapsed 58.84 s, remaining 210.13 s) 2024-05-11T13:36:37.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:37 smithi063 ceph-mon[29809]: pgmap v85: 129 pgs: 129 active+clean; 199 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 MiB/s wr, 9 op/s 2024-05-11T13:36:37.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:37 smithi071 ceph-mon[33293]: pgmap v85: 129 pgs: 129 active+clean; 199 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 MiB/s wr, 9 op/s 2024-05-11T13:36:37.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:37 smithi165 ceph-mon[33235]: pgmap v85: 129 pgs: 129 active+clean; 199 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 MiB/s wr, 9 op/s 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 -- 172.21.15.63:0/2771373894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50940760a0 msgr2=0x7f509410fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 --2- 172.21.15.63:0/2771373894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50940760a0 0x7f509410fa70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5080009990 tx=0x7f5080009ca0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 -- 172.21.15.63:0/2771373894 shutdown_connections 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 --2- 172.21.15.63:0/2771373894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f509410ffb0 0x7f50941123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 --2- 172.21.15.63:0/2771373894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5094075780 0x7f5094075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 --2- 172.21.15.63:0/2771373894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50940760a0 0x7f509410fa70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 -- 172.21.15.63:0/2771373894 >> 172.21.15.63:0/2771373894 conn(0x7f509407a6b0 msgr2=0x7f509407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.917+0000 7f5099400700 1 -- 172.21.15.63:0/2771373894 shutdown_connections 2024-05-11T13:36:38.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.918+0000 7f5099400700 1 -- 172.21.15.63:0/2771373894 wait complete. 2024-05-11T13:36:38.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 Processor -- start 2024-05-11T13:36:38.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 -- start start 2024-05-11T13:36:38.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5094075780 0x7f50941a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50940760a0 0x7f50941a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f509410ffb0 0x7f509419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5094115840 con 0x7f5094075780 2024-05-11T13:36:38.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50941156c0 con 0x7f50940760a0 2024-05-11T13:36:38.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.919+0000 7f5099400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50941159c0 con 0x7f509410ffb0 2024-05-11T13:36:38.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.920+0000 7f5091e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50940760a0 0x7f50941a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.920+0000 7f5091e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50940760a0 0x7f50941a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:41838/0 (socket says 172.21.15.63:41838) 2024-05-11T13:36:38.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.920+0000 7f5091e00700 1 -- 172.21.15.63:0/3837800936 learned_addr learned my addr 172.21.15.63:0/3837800936 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:38.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.920+0000 7f5093200700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f509410ffb0 0x7f509419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.921+0000 7f5092800700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5094075780 0x7f50941a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.921+0000 7f5091e00700 1 -- 172.21.15.63:0/3837800936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f509410ffb0 msgr2=0x7f509419baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:38.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.921+0000 7f5091e00700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f509410ffb0 0x7f509419baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.921+0000 7f5091e00700 1 -- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5094075780 msgr2=0x7f50941a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:38.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.921+0000 7f5091e00700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5094075780 0x7f50941a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.921+0000 7f5091e00700 1 -- 172.21.15.63:0/3837800936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509419c200 con 0x7f50940760a0 2024-05-11T13:36:38.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.922+0000 7f5091e00700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50940760a0 0x7f50941a1e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50800057c0 tx=0x7f50800057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:38.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.922+0000 7f508b400700 1 -- 172.21.15.63:0/3837800936 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080027070 con 0x7f50940760a0 2024-05-11T13:36:38.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.922+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509419c490 con 0x7f50940760a0 2024-05-11T13:36:38.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.922+0000 7f508b400700 1 -- 172.21.15.63:0/3837800936 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f508000fc00 con 0x7f50940760a0 2024-05-11T13:36:38.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.922+0000 7f508b400700 1 -- 172.21.15.63:0/3837800936 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080022b30 con 0x7f50940760a0 2024-05-11T13:36:38.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.922+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50941a87b0 con 0x7f50940760a0 2024-05-11T13:36:38.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.928+0000 7f50a3e1cb80 1 Processor -- start 2024-05-11T13:36:38.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.928+0000 7f508b400700 1 -- 172.21.15.63:0/3837800936 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f508002a070 con 0x7f50940760a0 2024-05-11T13:36:38.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f50a3e1cb80 1 -- start start 2024-05-11T13:36:38.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a88740440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a8873aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88746e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558a8868cbb0 con 0x558a88740060 2024-05-11T13:36:38.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558a8862a8f0 con 0x558a8873aff0 2024-05-11T13:36:38.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558a88688b70 con 0x558a8873a630 2024-05-11T13:36:38.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.930+0000 7f508b400700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50780b2aa0 0x7f50780b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f5093200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a88740440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f508b400700 1 -- 172.21.15.63:0/3837800936 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f50800e3910 con 0x7f50940760a0 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f5093200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a88740440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40696/0 (socket says 172.21.15.63:40696) 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f5093200700 1 -- 172.21.15.63:0/4066855131 learned_addr learned my addr 172.21.15.63:0/4066855131 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f5093200700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50780b2aa0 0x7f50780b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f5091e00700 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88746e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f5092800700 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a8873aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.931+0000 7f5093200700 1 -- 172.21.15.63:0/4066855131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 msgr2=0x558a8873aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.932+0000 7f5093200700 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a8873aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.932+0000 7f5093200700 1 -- 172.21.15.63:0/4066855131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 msgr2=0x558a88746e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.932+0000 7f5093200700 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88746e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.932+0000 7f5093200700 1 -- 172.21.15.63:0/4066855131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a885d9a60 con 0x558a88740060 2024-05-11T13:36:38.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.932+0000 7f5092800700 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a8873aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:36:38.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.933+0000 7f5093200700 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a88740440 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f508c0077c0 tx=0x7f508c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4a1f2f25631466 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:38.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.933+0000 7f5093200700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50780b2aa0 0x7f50780b4f60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f508c010000 tx=0x7f508c01d000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:38.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.933+0000 7f507fe00700 1 -- 172.21.15.63:0/4066855131 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c019dd0 con 0x558a88740060 2024-05-11T13:36:38.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.933+0000 7f507fe00700 1 -- 172.21.15.63:0/4066855131 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f508c004bb0 con 0x558a88740060 2024-05-11T13:36:38.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.933+0000 7f507fe00700 1 -- 172.21.15.63:0/4066855131 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c022410 con 0x558a88740060 2024-05-11T13:36:38.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.934+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/4066855131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 msgr2=0x558a88740440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:38.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.934+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a88740440 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f508c0077c0 tx=0x7f508c007ad0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.935+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/4066855131 shutdown_connections 2024-05-11T13:36:38.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.935+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88746e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.935+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a88740440 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.935+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/4066855131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a8873aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.935+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/4066855131 >> 172.21.15.63:0/4066855131 conn(0x558a88668cf0 msgr2=0x558a88734f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:38.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.935+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/4066855131 shutdown_connections 2024-05-11T13:36:38.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.935+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/4066855131 wait complete. 2024-05-11T13:36:38.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.936+0000 7f50a3e1cb80 1 -- start start 2024-05-11T13:36:38.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.936+0000 7f50a3e1cb80 1 Processor -- start 2024-05-11T13:36:38.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a88746b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a88743eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a887443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558a88747360 con 0x558a8873aff0 2024-05-11T13:36:38.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558a885f2cb0 con 0x558a8873a630 2024-05-11T13:36:38.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558a887474e0 con 0x558a88740060 2024-05-11T13:36:38.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f50a3e1cb80 10 client.0 _ll_register_callbacks cb 0x558a88633e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:36:38.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f5091e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a88743eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f5091e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a88743eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40702/0 (socket says 172.21.15.63:40702) 2024-05-11T13:36:38.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f5091e00700 1 -- 172.21.15.63:0/2077275135 learned_addr learned my addr 172.21.15.63:0/2077275135 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:38.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f5092800700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a88746b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.937+0000 7f5093200700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a887443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f5091e00700 1 -- 172.21.15.63:0/2077275135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 msgr2=0x558a887443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f5091e00700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a887443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f5091e00700 1 -- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 msgr2=0x558a88746b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f5091e00700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a88746b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f5091e00700 1 -- 172.21.15.63:0/2077275135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a886525b0 con 0x558a8873aff0 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f5093200700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a887443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f5091e00700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a88743eb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f50800a7b60 tx=0x7f5080009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b113be7e4a3c8f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f507cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.938+0000 7f507cc00700 1 -- 172.21.15.63:0/2077275135 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50800aa400 con 0x558a8873aff0 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.939+0000 7f507cc00700 1 -- 172.21.15.63:0/2077275135 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f50800094b0 con 0x558a8873aff0 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.939+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f508c004bb0 con 0x558a8873aff0 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.939+0000 7f507cc00700 1 -- 172.21.15.63:0/2077275135 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f50800aaff0 con 0x558a8873aff0 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.939+0000 7f50a3e1cb80 10 client.15630 fetch_fsmap learned FSMap version 36 2024-05-11T13:36:38.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.939+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558a88751bb0 con 0x558a8873aff0 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.939+0000 7f507cc00700 1 -- 172.21.15.63:0/2077275135 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f508c004bb0 con 0x558a8873aff0 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.940+0000 7f507cc00700 1 -- 172.21.15.63:0/2077275135 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8418+0+0 (secure 0 0 0) 0x7f50800aabe0 con 0x558a8873aff0 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.940+0000 7f50a3e1cb80 10 client.15630 fetch_fsmap finished waiting for FSMap version 36 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.940+0000 7f50a3e1cb80 10 client.15630 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.940+0000 7f50a3e1cb80 20 client.15630 populate_metadata read hostname 'smithi063' 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.940+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a88754240 0x558a88756660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.940+0000 7f50a3e1cb80 4 client.15630 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.940+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a88707340 con 0x558a88754240 2024-05-11T13:36:38.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.941+0000 7f5092800700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a88754240 0x558a88756660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.942+0000 7f5092800700 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a88754240 0x558a88756660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50047045e6bd9ee9 server_cookie=41c62475e7eff487 in_seq=0 out_seq=0 2024-05-11T13:36:38.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.942+0000 7f507cc00700 10 client.15630 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f507cc00700 1 -- 172.21.15.63:0/2077275135 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a88707340 con 0x558a88754240 2024-05-11T13:36:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f507cc00700 10 client.15630 handle_command_reply: tid=0 2024-05-11T13:36:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 client.15630 shutdown 2024-05-11T13:36:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 msgr2=0x558a88743eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a88743eb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f50800a7b60 tx=0x7f5080009ca0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 shutdown_connections 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a887443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a88754240 0x558a88756660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a88746b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/2077275135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a88743eb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.944+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 >> 172.21.15.63:0/2077275135 conn(0x558a88668cf0 msgr2=0x558a8872f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.945+0000 7f507cc00700 0 client.15630 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.945+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 shutdown_connections 2024-05-11T13:36:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.945+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/2077275135 wait complete. 2024-05-11T13:36:38.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.945+0000 7f50a3e1cb80 20 client.15630 trim_cache size 0 max 16384 2024-05-11T13:36:38.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.981+0000 7f50a3e1cb80 1 Processor -- start 2024-05-11T13:36:38.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.982+0000 7f50a3e1cb80 1 -- start start 2024-05-11T13:36:38.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.982+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a887444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.982+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a887503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.982+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a8875bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.982+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50941b1600 con 0x558a8873aff0 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.982+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50941b33e0 con 0x558a8873a630 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.982+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50941b1d90 con 0x558a88740060 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a887503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a887503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40704/0 (socket says 172.21.15.63:40704) 2024-05-11T13:36:38.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 -- 172.21.15.63:0/720107773 learned_addr learned my addr 172.21.15.63:0/720107773 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:38.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5092800700 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a887444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5093200700 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a8875bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 -- 172.21.15.63:0/720107773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 msgr2=0x558a8875bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a8875bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 -- 172.21.15.63:0/720107773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 msgr2=0x558a887444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a887444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.983+0000 7f5091e00700 1 -- 172.21.15.63:0/720107773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a88735af0 con 0x558a8873aff0 2024-05-11T13:36:38.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.984+0000 7f5091e00700 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a887503b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5080005f50 tx=0x7f5080005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e026e708b3b860 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:38.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.984+0000 7f507f400700 1 -- 172.21.15.63:0/720107773 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50800aa400 con 0x558a8873aff0 2024-05-11T13:36:38.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.984+0000 7f507f400700 1 -- 172.21.15.63:0/720107773 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50800aa8f0 con 0x558a8873aff0 2024-05-11T13:36:38.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.984+0000 7f507f400700 1 -- 172.21.15.63:0/720107773 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080005190 con 0x558a8873aff0 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/720107773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 msgr2=0x558a887503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a887503b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5080005f50 tx=0x7f5080005860 comp rx=0 tx=0).stop 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/720107773 shutdown_connections 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a88740060 0x558a8875bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873a630 0x558a887444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/720107773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a8873aff0 0x558a887503b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/720107773 >> 172.21.15.63:0/720107773 conn(0x558a88668cf0 msgr2=0x558a88739eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:38.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.985+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/720107773 shutdown_connections 2024-05-11T13:36:38.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.986+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/720107773 wait complete. 2024-05-11T13:36:38.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.986+0000 7f50a3e1cb80 1 -- start start 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.986+0000 7f50a3e1cb80 1 Processor -- start 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a88749700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88749c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f50a3e1cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a8874a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50941b2c50 con 0x558a88740060 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50941b2520 con 0x558a8873aff0 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f50a3e1cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50941af5a0 con 0x558a8873a630 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f50a3e1cb80 10 client.0 _ll_register_callbacks cb 0x558a88633e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.987+0000 7f5092800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a88749700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a8874a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a8874a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40714/0 (socket says 172.21.15.63:40714) 2024-05-11T13:36:38.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 -- 172.21.15.63:0/3390082575 learned_addr learned my addr 172.21.15.63:0/3390082575 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:36:38.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5091e00700 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88749c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 -- 172.21.15.63:0/3390082575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 msgr2=0x558a88749700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a88749700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 -- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 msgr2=0x558a88749c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:36:38.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88749c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:38.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.988+0000 7f5093200700 1 -- 172.21.15.63:0/3390082575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a88731b20 con 0x558a88740060 2024-05-11T13:36:38.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.989+0000 7f5093200700 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a8874a180 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f508c0077c0 tx=0x7f508c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cbcf7d5817ed42 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:36:38.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.989+0000 7f507cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.990+0000 7f507cc00700 1 -- 172.21.15.63:0/3390082575 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c003c10 con 0x558a88740060 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.990+0000 7f507cc00700 1 -- 172.21.15.63:0/3390082575 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f508c003db0 con 0x558a88740060 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.990+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50800aa8f0 con 0x558a88740060 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.990+0000 7f507cc00700 1 -- 172.21.15.63:0/3390082575 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f508c03c050 con 0x558a88740060 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.990+0000 7f50a3e1cb80 10 client.15636 fetch_fsmap learned FSMap version 36 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.990+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558a887bfb20 con 0x558a88740060 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f507cc00700 1 -- 172.21.15.63:0/3390082575 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50800aa8f0 con 0x558a88740060 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f507cc00700 1 -- 172.21.15.63:0/3390082575 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8418+0+0 (secure 0 0 0) 0x7f508c018040 con 0x558a88740060 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f50a3e1cb80 10 client.15636 fetch_fsmap finished waiting for FSMap version 36 2024-05-11T13:36:38.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f50a3e1cb80 10 client.15636 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:36:38.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f50a3e1cb80 20 client.15636 populate_metadata read hostname 'smithi063' 2024-05-11T13:36:38.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a887c0010 0x558a887c2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:36:38.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f50a3e1cb80 4 client.15636 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}] 2024-05-11T13:36:38.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.991+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}) -- 0x558a8869a590 con 0x558a887c0010 2024-05-11T13:36:38.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.992+0000 7f5092800700 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a887c0010 0x558a887c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:36:38.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.992+0000 7f5092800700 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a887c0010 0x558a887c2430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722d3de027bf401c server_cookie=3494b0bdc96d0a28 in_seq=0 out_seq=0 2024-05-11T13:36:38.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:38.992+0000 7f507cc00700 10 client.15636 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:39.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.241+0000 7f507cc00700 1 -- 172.21.15.63:0/3390082575 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558a8869a590 con 0x558a887c0010 2024-05-11T13:36:39.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.241+0000 7f507cc00700 10 client.15636 handle_command_reply: tid=0 2024-05-11T13:36:39.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 client.15636 shutdown 2024-05-11T13:36:39.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 msgr2=0x558a8874a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:39.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a8874a180 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f508c0077c0 tx=0x7f508c01a000 comp rx=0 tx=0).stop 2024-05-11T13:36:39.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 shutdown_connections 2024-05-11T13:36:39.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a88740060 0x558a8874a180 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a8873a630 0x558a88749700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558a887c0010 0x558a887c2430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 --2- 172.21.15.63:0/3390082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a8873aff0 0x558a88749c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f507cc00700 0 client.15636 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.242+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 >> 172.21.15.63:0/3390082575 conn(0x558a88668cf0 msgr2=0x558a8873dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.243+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 shutdown_connections 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.243+0000 7f50a3e1cb80 1 -- 172.21.15.63:0/3390082575 wait complete. 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.243+0000 7f50a3e1cb80 20 client.15636 trim_cache size 0 max 16384 2024-05-11T13:36:39.241 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 36, 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 10, 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "3ac0d618": { 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 10, 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:36:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.6, 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:36:39.244 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50780b2aa0 msgr2=0x7f50780b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50780b2aa0 0x7f50780b4f60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f508c010000 tx=0x7f508c01d000 comp rx=0 tx=0).stop 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50940760a0 msgr2=0x7f50941a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50940760a0 0x7f50941a1e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50800057c0 tx=0x7f50800057f0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 shutdown_connections 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50780b2aa0 0x7f50780b4f60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f509410ffb0 0x7f509419baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5094075780 0x7f50941a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 --2- 172.21.15.63:0/3837800936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50940760a0 0x7f50941a1e20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:36:39.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.245+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 >> 172.21.15.63:0/3837800936 conn(0x7f509407a6b0 msgr2=0x7f5094100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:36:39.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.246+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 shutdown_connections 2024-05-11T13:36:39.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:36:39.246+0000 7f5099400700 1 -- 172.21.15.63:0/3837800936 wait complete. 2024-05-11T13:36:39.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:39 smithi063 ceph-mon[29809]: pgmap v86: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 19 op/s 2024-05-11T13:36:39.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:39 smithi071 ceph-mon[33293]: pgmap v86: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 19 op/s 2024-05-11T13:36:39.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:39 smithi165 ceph-mon[33235]: pgmap v86: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 19 op/s 2024-05-11T13:36:40.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3ac0d618', quiesce took 8.5 seconds. Will release after: 25.6 2024-05-11T13:36:41.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:41 smithi071 ceph-mon[33293]: pgmap v87: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 15 op/s 2024-05-11T13:36:41.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:41 smithi165 ceph-mon[33235]: pgmap v87: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 15 op/s 2024-05-11T13:36:41.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:41 smithi063 ceph-mon[29809]: pgmap v87: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 15 op/s 2024-05-11T13:36:43.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:43 smithi063 ceph-mon[29809]: pgmap v88: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 15 op/s 2024-05-11T13:36:43.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:43 smithi071 ceph-mon[33293]: pgmap v88: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 15 op/s 2024-05-11T13:36:43.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:43 smithi165 ceph-mon[33235]: pgmap v88: 129 pgs: 129 active+clean; 239 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 15 op/s 2024-05-11T13:36:44.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:44 smithi165 ceph-mon[33235]: pgmap v89: 129 pgs: 129 active+clean; 246 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 24 op/s 2024-05-11T13:36:44.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:44 smithi063 ceph-mon[29809]: pgmap v89: 129 pgs: 129 active+clean; 246 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 24 op/s 2024-05-11T13:36:44.700 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:44 smithi071 ceph-mon[33293]: pgmap v89: 129 pgs: 129 active+clean; 246 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 24 op/s 2024-05-11T13:36:47.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:47 smithi063 ceph-mon[29809]: pgmap v90: 129 pgs: 129 active+clean; 246 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 18 op/s 2024-05-11T13:36:47.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:47 smithi071 ceph-mon[33293]: pgmap v90: 129 pgs: 129 active+clean; 246 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 18 op/s 2024-05-11T13:36:47.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:47 smithi165 ceph-mon[33235]: pgmap v90: 129 pgs: 129 active+clean; 246 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 18 op/s 2024-05-11T13:36:49.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:49 smithi063 ceph-mon[29809]: pgmap v91: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 23 op/s 2024-05-11T13:36:49.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:49 smithi071 ceph-mon[33293]: pgmap v91: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 23 op/s 2024-05-11T13:36:49.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:49 smithi165 ceph-mon[33235]: pgmap v91: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 23 op/s 2024-05-11T13:36:50.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:50 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:36:50.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:50 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:36:50.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:50 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:36:51.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:51 smithi063 ceph-mon[29809]: pgmap v92: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-05-11T13:36:51.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:51 smithi071 ceph-mon[33293]: pgmap v92: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-05-11T13:36:51.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:51 smithi165 ceph-mon[33235]: pgmap v92: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-05-11T13:36:53.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:53 smithi063 ceph-mon[29809]: pgmap v93: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-05-11T13:36:53.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:53 smithi071 ceph-mon[33293]: pgmap v93: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-05-11T13:36:53.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:53 smithi165 ceph-mon[33235]: pgmap v93: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-05-11T13:36:55.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:55 smithi063 ceph-mon[29809]: pgmap v94: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 18 op/s 2024-05-11T13:36:55.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:55 smithi071 ceph-mon[33293]: pgmap v94: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 18 op/s 2024-05-11T13:36:55.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:55 smithi165 ceph-mon[33235]: pgmap v94: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 18 op/s 2024-05-11T13:36:57.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:57 smithi063 ceph-mon[29809]: pgmap v95: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 526 KiB/s wr, 10 op/s 2024-05-11T13:36:57.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:57 smithi071 ceph-mon[33293]: pgmap v95: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 526 KiB/s wr, 10 op/s 2024-05-11T13:36:57.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:57 smithi165 ceph-mon[33235]: pgmap v95: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 526 KiB/s wr, 10 op/s 2024-05-11T13:36:59.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:36:59 smithi063 ceph-mon[29809]: pgmap v96: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 528 KiB/s wr, 14 op/s 2024-05-11T13:36:59.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:36:59 smithi071 ceph-mon[33293]: pgmap v96: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 528 KiB/s wr, 14 op/s 2024-05-11T13:36:59.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:36:59 smithi165 ceph-mon[33235]: pgmap v96: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 528 KiB/s wr, 14 op/s 2024-05-11T13:37:01.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:01 smithi063 ceph-mon[29809]: pgmap v97: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-11T13:37:01.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:01 smithi071 ceph-mon[33293]: pgmap v97: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-11T13:37:01.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:01 smithi165 ceph-mon[33235]: pgmap v97: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-11T13:37:03.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:03 smithi063 ceph-mon[29809]: pgmap v98: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-11T13:37:03.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:03 smithi071 ceph-mon[33293]: pgmap v98: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-11T13:37:03.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:03 smithi165 ceph-mon[33235]: pgmap v98: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-11T13:37:05.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:05 smithi071 ceph-mon[33293]: pgmap v99: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-05-11T13:37:05.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:05 smithi165 ceph-mon[33235]: pgmap v99: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-05-11T13:37:05.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:05 smithi063 ceph-mon[29809]: pgmap v99: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-05-11T13:37:05.767 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:37:06.227 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:37:06.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:06 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:37:06.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:06 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:37:06.574 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:06 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:37:07.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:07 smithi071 ceph-mon[33293]: pgmap v100: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-11T13:37:07.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:07 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:37:07.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:07 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:37:07.450 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:07 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:37:07.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:07 smithi165 ceph-mon[33235]: pgmap v100: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-11T13:37:07.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:07 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:37:07.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:07 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:37:07.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:07 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:37:07.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:07 smithi063 ceph-mon[29809]: pgmap v100: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-11T13:37:07.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:07 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:37:07.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:07 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:37:07.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:07 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:37:08.337 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:08 smithi063 ceph-mon[29809]: pgmap v101: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-05-11T13:37:08.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:08 smithi165 ceph-mon[33235]: pgmap v101: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-05-11T13:37:08.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 -- 172.21.15.63:0/727131860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50108900 msgr2=0x7fef50110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 --2- 172.21.15.63:0/727131860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50108900 0x7fef50110480 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fef480099c0 tx=0x7fef48009cd0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 -- 172.21.15.63:0/727131860 shutdown_connections 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 --2- 172.21.15.63:0/727131860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50108900 0x7fef50110480 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 --2- 172.21.15.63:0/727131860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef50103800 0x7fef50103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 --2- 172.21.15.63:0/727131860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef501041b0 0x7fef501081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 -- 172.21.15.63:0/727131860 >> 172.21.15.63:0/727131860 conn(0x7fef5007a6b0 msgr2=0x7fef5007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 -- 172.21.15.63:0/727131860 shutdown_connections 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.637+0000 7fef55200700 1 -- 172.21.15.63:0/727131860 wait complete. 2024-05-11T13:37:08.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.639+0000 7fef55200700 1 Processor -- start 2024-05-11T13:37:08.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.639+0000 7fef55200700 1 -- start start 2024-05-11T13:37:08.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef55200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50103800 0x7fef501a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:08.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef55200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef501041b0 0x7fef501a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:08.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef55200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef50108900 0x7fef5019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:08.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef55200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef501135a0 con 0x7fef50103800 2024-05-11T13:37:08.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef55200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113420 con 0x7fef501041b0 2024-05-11T13:37:08.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef55200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113720 con 0x7fef50108900 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef4de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50103800 0x7fef501a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef4de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50103800 0x7fef501a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42526/0 (socket says 172.21.15.63:42526) 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef4de00700 1 -- 172.21.15.63:0/138161665 learned_addr learned my addr 172.21.15.63:0/138161665 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.640+0000 7fef45400700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef501041b0 0x7fef501a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4e800700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef50108900 0x7fef5019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4de00700 1 -- 172.21.15.63:0/138161665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef50108900 msgr2=0x7fef5019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4de00700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef50108900 0x7fef5019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4de00700 1 -- 172.21.15.63:0/138161665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef501041b0 msgr2=0x7fef501a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4de00700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef501041b0 0x7fef501a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4de00700 1 -- 172.21.15.63:0/138161665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5019c200 con 0x7fef50103800 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4e800700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef50108900 0x7fef5019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:37:08.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef45400700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef501041b0 0x7fef501a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:37:08.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.641+0000 7fef4de00700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50103800 0x7fef501a18e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef40007f10 tx=0x7fef4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:08.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.642+0000 7fef47400700 1 -- 172.21.15.63:0/138161665 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4000dc90 con 0x7fef50103800 2024-05-11T13:37:08.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.642+0000 7fef47400700 1 -- 172.21.15.63:0/138161665 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef4000de30 con 0x7fef50103800 2024-05-11T13:37:08.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.642+0000 7fef47400700 1 -- 172.21.15.63:0/138161665 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40005a50 con 0x7fef50103800 2024-05-11T13:37:08.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.642+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5019c4f0 con 0x7fef50103800 2024-05-11T13:37:08.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.642+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501a87b0 con 0x7fef50103800 2024-05-11T13:37:08.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.643+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5006a240 con 0x7fef50103800 2024-05-11T13:37:08.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.646+0000 7fef47400700 1 -- 172.21.15.63:0/138161665 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fef4001a020 con 0x7fef50103800 2024-05-11T13:37:08.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.647+0000 7fef47400700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fef340b2c10 0x7fef340b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:08.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.647+0000 7fef47400700 1 -- 172.21.15.63:0/138161665 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fef400e2250 con 0x7fef50103800 2024-05-11T13:37:08.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.648+0000 7fef45400700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fef340b2c10 0x7fef340b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:08.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.648+0000 7fef45400700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fef340b2c10 0x7fef340b50d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef38005950 tx=0x7fef380058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:08.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.652+0000 7fef47400700 1 -- 172.21.15.63:0/138161665 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef40010070 con 0x7fef50103800 2024-05-11T13:37:08.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:08 smithi071 ceph-mon[33293]: pgmap v101: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-05-11T13:37:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.942+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef501a89e0 con 0x7fef50103800 2024-05-11T13:37:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.943+0000 7fef47400700 1 -- 172.21.15.63:0/138161665 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11955 (secure 0 0 0) 0x7fef400a9840 con 0x7fef50103800 2024-05-11T13:37:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:37:08.940 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:36:49.061519+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:37:08.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.946+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fef340b2c10 msgr2=0x7fef340b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fef340b2c10 0x7fef340b50d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef38005950 tx=0x7fef380058e0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50103800 msgr2=0x7fef501a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50103800 0x7fef501a18e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef40007f10 tx=0x7fef4000d3b0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 shutdown_connections 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fef340b2c10 0x7fef340b50d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef50108900 0x7fef5019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef50103800 0x7fef501a18e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 --2- 172.21.15.63:0/138161665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef501041b0 0x7fef501a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:08.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 >> 172.21.15.63:0/138161665 conn(0x7fef5007a6b0 msgr2=0x7fef50101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:08.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 shutdown_connections 2024-05-11T13:37:08.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:08.947+0000 7fef55200700 1 -- 172.21.15.63:0/138161665 wait complete. 2024-05-11T13:37:08.946 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 37 2024-05-11T13:37:09.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:09 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/138161665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:37:09.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:09 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/138161665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:37:09.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:09 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/138161665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:37:09.877 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 3ac0d618 --release --await' 2024-05-11T13:37:09.877 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 3ac0d618 --release --await 2024-05-11T13:37:10.334 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:37:10.526 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:10 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:10.526 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:10 smithi063 ceph-mon[29809]: pgmap v102: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-11T13:37:10.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:10 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:10.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:10 smithi165 ceph-mon[33235]: pgmap v102: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-11T13:37:10.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:10 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:10.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:10 smithi071 ceph-mon[33293]: pgmap v102: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 -- 172.21.15.63:0/4240756680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c1078b0 msgr2=0x7fd12c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 --2- 172.21.15.63:0/4240756680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c1078b0 0x7fd12c107d60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b0a0 tx=0x7fd12800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 -- 172.21.15.63:0/4240756680 shutdown_connections 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 --2- 172.21.15.63:0/4240756680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c1078b0 0x7fd12c107d60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 --2- 172.21.15.63:0/4240756680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd12c1024a0 0x7fd12c102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 --2- 172.21.15.63:0/4240756680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c102dc0 0x7fd12c107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 -- 172.21.15.63:0/4240756680 >> 172.21.15.63:0/4240756680 conn(0x7fd12c0fdf00 msgr2=0x7fd12c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:12.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.784+0000 7fd133200700 1 -- 172.21.15.63:0/4240756680 shutdown_connections 2024-05-11T13:37:12.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.785+0000 7fd133200700 1 -- 172.21.15.63:0/4240756680 wait complete. 2024-05-11T13:37:12.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 Processor -- start 2024-05-11T13:37:12.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 -- start start 2024-05-11T13:37:12.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c1024a0 0x7fd12c1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c102dc0 0x7fd12c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd12c1078b0 0x7fd12c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c119bd0 con 0x7fd12c102dc0 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c119a50 con 0x7fd12c1078b0 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.786+0000 7fd133200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c119d50 con 0x7fd12c1024a0 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c102dc0 0x7fd12c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c102dc0 0x7fd12c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40396/0 (socket says 172.21.15.63:40396) 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 -- 172.21.15.63:0/3263569847 learned_addr learned my addr 172.21.15.63:0/3263569847 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131e00700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c1024a0 0x7fd12c1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd132800700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd12c1078b0 0x7fd12c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 -- 172.21.15.63:0/3263569847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c1024a0 msgr2=0x7fd12c1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c1024a0 0x7fd12c1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 -- 172.21.15.63:0/3263569847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd12c1078b0 msgr2=0x7fd12c1a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd12c1078b0 0x7fd12c1a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.787+0000 7fd131400700 1 -- 172.21.15.63:0/3263569847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c1a5110 con 0x7fd12c102dc0 2024-05-11T13:37:12.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.788+0000 7fd131400700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c102dc0 0x7fd12c1a0640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d750 tx=0x7fd11c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:12.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.788+0000 7fd126a00700 1 -- 172.21.15.63:0/3263569847 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c016070 con 0x7fd12c102dc0 2024-05-11T13:37:12.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.788+0000 7fd126a00700 1 -- 172.21.15.63:0/3263569847 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd11c00fd20 con 0x7fd12c102dc0 2024-05-11T13:37:12.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.788+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c1a53a0 con 0x7fd12c102dc0 2024-05-11T13:37:12.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.788+0000 7fd126a00700 1 -- 172.21.15.63:0/3263569847 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00e3b0 con 0x7fd12c102dc0 2024-05-11T13:37:12.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.788+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c1acc80 con 0x7fd12c102dc0 2024-05-11T13:37:12.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.792+0000 7fd126a00700 1 -- 172.21.15.63:0/3263569847 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd11c00f890 con 0x7fd12c102dc0 2024-05-11T13:37:12.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.793+0000 7fd13f0adb80 1 Processor -- start 2024-05-11T13:37:12.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.793+0000 7fd13f0adb80 1 -- start start 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.793+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa46800 0x55caeaa46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa53260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55caea998b20 con 0x55caeaa46800 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55caea936860 con 0x55caeaa471c0 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55caea994ae0 con 0x55caeaa4c4a0 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd126a00700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd1140b29f0 0x7fd1140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd131e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa46800 0x55caeaa46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd131e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa46800 0x55caeaa46c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40406/0 (socket says 172.21.15.63:40406) 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd131e00700 1 -- 172.21.15.63:0/319886018 learned_addr learned my addr 172.21.15.63:0/319886018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:37:12.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd132800700 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd126a00700 1 -- 172.21.15.63:0/3263569847 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd11c0e25b0 con 0x7fd12c102dc0 2024-05-11T13:37:12.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd131400700 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa53260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.794+0000 7fd132800700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd1140b29f0 0x7fd1140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd131e00700 1 -- 172.21.15.63:0/319886018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa4c4a0 msgr2=0x55caeaa4c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd131e00700 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd131e00700 1 -- 172.21.15.63:0/319886018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 msgr2=0x55caeaa53260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd131e00700 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa53260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd131e00700 1 -- 172.21.15.63:0/319886018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caea8e59d0 con 0x55caeaa46800 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd131e00700 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa46800 0x55caeaa46c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd12000bd70 tx=0x7fd1200106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0ab59374045f39 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd132800700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd1140b29f0 0x7fd1140b4eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd128009e60 tx=0x7fd128006210 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.795+0000 7fd11b400700 1 -- 172.21.15.63:0/319886018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120010e10 con 0x55caeaa46800 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.796+0000 7fd11b400700 1 -- 172.21.15.63:0/319886018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd12001c910 con 0x55caeaa46800 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.796+0000 7fd11b400700 1 -- 172.21.15.63:0/319886018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200117a0 con 0x55caeaa46800 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.796+0000 7fd13f0adb80 1 -- 172.21.15.63:0/319886018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa46800 msgr2=0x55caeaa46c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.796+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa46800 0x55caeaa46c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd12000bd70 tx=0x7fd1200106b0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.796+0000 7fd13f0adb80 1 -- 172.21.15.63:0/319886018 shutdown_connections 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.796+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa53260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.797+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.797+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/319886018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa46800 0x55caeaa46c80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.797+0000 7fd13f0adb80 1 -- 172.21.15.63:0/319886018 >> 172.21.15.63:0/319886018 conn(0x55caea974c60 msgr2=0x55caeaa41480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.797+0000 7fd13f0adb80 1 -- 172.21.15.63:0/319886018 shutdown_connections 2024-05-11T13:37:12.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.797+0000 7fd13f0adb80 1 -- 172.21.15.63:0/319886018 wait complete. 2024-05-11T13:37:12.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.797+0000 7fd13f0adb80 1 -- start start 2024-05-11T13:37:12.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.797+0000 7fd13f0adb80 1 Processor -- start 2024-05-11T13:37:12.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa471c0 0x55caeaa50250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa4c4a0 0x55caeaa50790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55caeaa53800 con 0x55caeaa471c0 2024-05-11T13:37:12.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55caea8fec20 con 0x55caeaa4c4a0 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55caeaa53980 con 0x55caeaa46800 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd13f0adb80 10 client.0 _ll_register_callbacks cb 0x55caea92c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd131400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa471c0 0x55caeaa50250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd131400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa471c0 0x55caeaa50250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40408/0 (socket says 172.21.15.63:40408) 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.798+0000 7fd131400700 1 -- 172.21.15.63:0/2497403807 learned_addr learned my addr 172.21.15.63:0/2497403807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd131e00700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd132800700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa4c4a0 0x55caeaa50790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd131400700 1 -- 172.21.15.63:0/2497403807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 msgr2=0x55caeaa52eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd131400700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa52eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd131400700 1 -- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa4c4a0 msgr2=0x55caeaa50790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd131400700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa4c4a0 0x55caeaa50790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd131400700 1 -- 172.21.15.63:0/2497403807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caea95e520 con 0x55caeaa471c0 2024-05-11T13:37:12.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd132800700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa4c4a0 0x55caeaa50790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd131400700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa471c0 0x55caeaa50250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d720 tx=0x7fd11c0dfe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b45ad7f29f81bf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd10fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.799+0000 7fd10fe00700 1 -- 172.21.15.63:0/2497403807 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c01f040 con 0x55caeaa471c0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd10fe00700 1 -- 172.21.15.63:0/2497403807 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd11c0b0de0 con 0x55caeaa471c0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd12001c910 con 0x55caeaa471c0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd10fe00700 1 -- 172.21.15.63:0/2497403807 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd11c0a8c10 con 0x55caeaa471c0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd13f0adb80 10 client.15654 fetch_fsmap learned FSMap version 38 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55caeaa5e0b0 con 0x55caeaa471c0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd10fe00700 1 -- 172.21.15.63:0/2497403807 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd12001c910 con 0x55caeaa471c0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd10fe00700 1 -- 172.21.15.63:0/2497403807 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8406+0+0 (secure 0 0 0) 0x7fd11c0b1e70 con 0x55caeaa471c0 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd13f0adb80 10 client.15654 fetch_fsmap finished waiting for FSMap version 38 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd13f0adb80 10 client.15654 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.800+0000 7fd13f0adb80 20 client.15654 populate_metadata read hostname 'smithi063' 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.801+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaa60640 0x55caeaa62a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.801+0000 7fd13f0adb80 4 client.15654 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:37:12.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.801+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caeaa13160 con 0x55caeaa60640 2024-05-11T13:37:12.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.801+0000 7fd131e00700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaa60640 0x55caeaa62a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.802+0000 7fd131e00700 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaa60640 0x55caeaa62a30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c3478dec120bba server_cookie=d90e1e057774ed39 in_seq=0 out_seq=0 2024-05-11T13:37:12.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.802+0000 7fd10fe00700 10 client.15654 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:37:12.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.804+0000 7fd10fe00700 1 -- 172.21.15.63:0/2497403807 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55caeaa13160 con 0x55caeaa60640 2024-05-11T13:37:12.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.804+0000 7fd10fe00700 10 client.15654 handle_command_reply: tid=0 2024-05-11T13:37:12.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.804+0000 7fd13f0adb80 1 client.15654 shutdown 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa471c0 msgr2=0x55caeaa50250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa471c0 0x55caeaa50250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d720 tx=0x7fd11c0dfe70 comp rx=0 tx=0).stop 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 shutdown_connections 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaa60640 0x55caeaa62a30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa4c4a0 0x55caeaa50790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa52eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2497403807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa471c0 0x55caeaa50250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 >> 172.21.15.63:0/2497403807 conn(0x55caea974c60 msgr2=0x55caeaa457f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd10fe00700 0 client.15654 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 shutdown_connections 2024-05-11T13:37:12.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2497403807 wait complete. 2024-05-11T13:37:12.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.805+0000 7fd13f0adb80 20 client.15654 trim_cache size 0 max 16384 2024-05-11T13:37:12.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.827+0000 7fd13f0adb80 1 Processor -- start 2024-05-11T13:37:12.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd13f0adb80 1 -- start start 2024-05-11T13:37:12.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa5c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa3b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1b5880 con 0x55caeaa4c4a0 2024-05-11T13:37:12.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1b75d0 con 0x55caeaa471c0 2024-05-11T13:37:12.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1b6010 con 0x55caeaa46800 2024-05-11T13:37:12.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd132800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd132800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40412/0 (socket says 172.21.15.63:40412) 2024-05-11T13:37:12.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.828+0000 7fd132800700 1 -- 172.21.15.63:0/4091509854 learned_addr learned my addr 172.21.15.63:0/4091509854 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:37:12.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd131e00700 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa5c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd131400700 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa3b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd132800700 1 -- 172.21.15.63:0/4091509854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 msgr2=0x55caeaa5c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd132800700 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa5c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd132800700 1 -- 172.21.15.63:0/4091509854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 msgr2=0x55caeaa3b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd132800700 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa3b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd132800700 1 -- 172.21.15.63:0/4091509854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caeaa46410 con 0x55caeaa4c4a0 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd131400700 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa3b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.829+0000 7fd132800700 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4f250 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd128015000 tx=0x7fd128009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f328c0f224a35b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd11aa00700 1 -- 172.21.15.63:0/4091509854 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128032070 con 0x55caeaa4c4a0 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd11aa00700 1 -- 172.21.15.63:0/4091509854 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd128007c30 con 0x55caeaa4c4a0 2024-05-11T13:37:12.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd11aa00700 1 -- 172.21.15.63:0/4091509854 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1280058e0 con 0x55caeaa4c4a0 2024-05-11T13:37:12.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd13f0adb80 1 -- 172.21.15.63:0/4091509854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 msgr2=0x55caeaa4f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4f250 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd128015000 tx=0x7fd128009be0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd13f0adb80 1 -- 172.21.15.63:0/4091509854 shutdown_connections 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaa4f250 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa5c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/4091509854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa3b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.830+0000 7fd13f0adb80 1 -- 172.21.15.63:0/4091509854 >> 172.21.15.63:0/4091509854 conn(0x55caea974c60 msgr2=0x55caeaa4eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.831+0000 7fd13f0adb80 1 -- 172.21.15.63:0/4091509854 shutdown_connections 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.831+0000 7fd13f0adb80 1 -- 172.21.15.63:0/4091509854 wait complete. 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.831+0000 7fd13f0adb80 1 -- start start 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.831+0000 7fd13f0adb80 1 Processor -- start 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa55030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa55570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd13f0adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaac3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1b6ed0 con 0x55caeaa4c4a0 2024-05-11T13:37:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1b67a0 con 0x55caeaa471c0 2024-05-11T13:37:12.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd13f0adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1b3820 con 0x55caeaa46800 2024-05-11T13:37:12.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd13f0adb80 10 client.0 _ll_register_callbacks cb 0x55caea92c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:37:12.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd132800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaac3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd132800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaac3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40414/0 (socket says 172.21.15.63:40414) 2024-05-11T13:37:12.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd132800700 1 -- 172.21.15.63:0/2639273284 learned_addr learned my addr 172.21.15.63:0/2639273284 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:37:12.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd131e00700 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa55030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.832+0000 7fd131400700 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa55570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd132800700 1 -- 172.21.15.63:0/2639273284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 msgr2=0x55caeaa55030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd132800700 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa55030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd132800700 1 -- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 msgr2=0x55caeaa55570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd132800700 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa55570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd132800700 1 -- 172.21.15.63:0/2639273284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caeaac44f0 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd132800700 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaac3eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b3f0 tx=0x7fd128007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29872c542cc27a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd10fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd10fe00700 1 -- 172.21.15.63:0/2639273284 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128032030 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd10fe00700 1 -- 172.21.15.63:0/2639273284 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fd128005770 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.833+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd128007c30 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd10fe00700 1 -- 172.21.15.63:0/2639273284 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd128043050 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 10 client.15660 fetch_fsmap learned FSMap version 38 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55caeaacb810 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd10fe00700 1 -- 172.21.15.63:0/2639273284 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd128007c30 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd10fe00700 1 -- 172.21.15.63:0/2639273284 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8406+0+0 (secure 0 0 0) 0x7fd128011070 con 0x55caeaa4c4a0 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 10 client.15660 fetch_fsmap finished waiting for FSMap version 38 2024-05-11T13:37:12.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 10 client.15660 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:37:12.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 20 client.15660 populate_metadata read hostname 'smithi063' 2024-05-11T13:37:12.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaacbec0 0x55caeaace2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:37:12.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 4 client.15660 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3ac0d618", "release": true, "await": true}] 2024-05-11T13:37:12.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.834+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3ac0d618", "release": true, "await": true}) -- 0x55caea9a6500 con 0x55caeaacbec0 2024-05-11T13:37:12.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.835+0000 7fd131e00700 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaacbec0 0x55caeaace2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:37:12.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.836+0000 7fd131e00700 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaacbec0 0x55caeaace2e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4f9a75a90e1e6a server_cookie=c6aa12b2f4be5d03 in_seq=0 out_seq=0 2024-05-11T13:37:12.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.836+0000 7fd10fe00700 10 client.15660 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:37:12.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.902+0000 7fd10fe00700 1 -- 172.21.15.63:0/2639273284 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55caea9a6500 con 0x55caeaacbec0 2024-05-11T13:37:12.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.902+0000 7fd10fe00700 10 client.15660 handle_command_reply: tid=0 2024-05-11T13:37:12.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.902+0000 7fd13f0adb80 1 client.15660 shutdown 2024-05-11T13:37:12.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.903+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 msgr2=0x55caeaac3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.903+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaac3eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b3f0 tx=0x7fd128007880 comp rx=0 tx=0).stop 2024-05-11T13:37:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.903+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 shutdown_connections 2024-05-11T13:37:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caeaa4c4a0 0x55caeaac3eb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55caeaa46800 0x55caeaa55030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55caeaacbec0 0x55caeaace2e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 1 --2- 172.21.15.63:0/2639273284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55caeaa471c0 0x55caeaa55570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 >> 172.21.15.63:0/2639273284 conn(0x55caea974c60 msgr2=0x55caeaa4a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd10fe00700 0 client.15660 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 shutdown_connections 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 1 -- 172.21.15.63:0/2639273284 wait complete. 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.904+0000 7fd13f0adb80 20 client.15660 trim_cache size 0 max 16384 2024-05-11T13:37:12.902 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 38, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 12, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "3ac0d618": { 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 12, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.6, 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:37:12.903 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:37:12.904 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd1140b29f0 msgr2=0x7fd1140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd1140b29f0 0x7fd1140b4eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd128009e60 tx=0x7fd128006210 comp rx=0 tx=0).stop 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c102dc0 msgr2=0x7fd12c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c102dc0 0x7fd12c1a0640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d750 tx=0x7fd11c00da60 comp rx=0 tx=0).stop 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 shutdown_connections 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd1140b29f0 0x7fd1140b4eb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd12c1078b0 0x7fd12c1a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c1024a0 0x7fd12c1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 --2- 172.21.15.63:0/3263569847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd12c102dc0 0x7fd12c1a0640 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:37:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.906+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 >> 172.21.15.63:0/3263569847 conn(0x7fd12c0fdf00 msgr2=0x7fd12c105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:37:12.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.907+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 shutdown_connections 2024-05-11T13:37:12.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:37:12.908+0000 7fd133200700 1 -- 172.21.15.63:0/3263569847 wait complete. 2024-05-11T13:37:13.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:13 smithi063 ceph-mon[29809]: pgmap v103: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-11T13:37:13.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:13 smithi071 ceph-mon[33293]: pgmap v103: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-11T13:37:13.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:13 smithi165 ceph-mon[33235]: pgmap v103: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-11T13:37:13.864 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3ac0d618', seconds elapsed: 42.2 2024-05-11T13:37:15.133 INFO:teuthology.orchestra.run.smithi063.stderr:800000 of 3200000 tuples (25%) done (elapsed 96.73 s, remaining 290.19 s) 2024-05-11T13:37:15.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:15 smithi063 ceph-mon[29809]: pgmap v104: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-05-11T13:37:15.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:15 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:15.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:15 smithi071 ceph-mon[33293]: pgmap v104: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-05-11T13:37:15.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:15 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:15.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:15 smithi165 ceph-mon[33235]: pgmap v104: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-05-11T13:37:15.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:15 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:17.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:17 smithi063 ceph-mon[29809]: pgmap v105: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-11T13:37:17.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:17 smithi071 ceph-mon[33293]: pgmap v105: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-11T13:37:17.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:17 smithi165 ceph-mon[33235]: pgmap v105: 129 pgs: 129 active+clean; 252 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-11T13:37:19.425 INFO:teuthology.orchestra.run.smithi063.stderr:900000 of 3200000 tuples (28%) done (elapsed 101.02 s, remaining 258.17 s) 2024-05-11T13:37:19.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:19 smithi063 ceph-mon[29809]: pgmap v106: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.2 MiB/s wr, 18 op/s 2024-05-11T13:37:19.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:19 smithi071 ceph-mon[33293]: pgmap v106: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.2 MiB/s wr, 18 op/s 2024-05-11T13:37:19.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:19 smithi165 ceph-mon[33235]: pgmap v106: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.2 MiB/s wr, 18 op/s 2024-05-11T13:37:21.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:21 smithi063 ceph-mon[29809]: pgmap v107: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 13 op/s 2024-05-11T13:37:21.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:21 smithi071 ceph-mon[33293]: pgmap v107: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 13 op/s 2024-05-11T13:37:21.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:21 smithi165 ceph-mon[33235]: pgmap v107: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 13 op/s 2024-05-11T13:37:23.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:23 smithi063 ceph-mon[29809]: pgmap v108: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 13 op/s 2024-05-11T13:37:23.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:23 smithi071 ceph-mon[33293]: pgmap v108: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 13 op/s 2024-05-11T13:37:23.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:23 smithi165 ceph-mon[33235]: pgmap v108: 129 pgs: 129 active+clean; 293 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 13 op/s 2024-05-11T13:37:24.448 INFO:teuthology.orchestra.run.smithi063.stderr:1000000 of 3200000 tuples (31%) done (elapsed 106.05 s, remaining 233.30 s) 2024-05-11T13:37:25.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:25 smithi063 ceph-mon[29809]: pgmap v109: 129 pgs: 129 active+clean; 322 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.2 MiB/s wr, 21 op/s 2024-05-11T13:37:25.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:25 smithi071 ceph-mon[33293]: pgmap v109: 129 pgs: 129 active+clean; 322 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.2 MiB/s wr, 21 op/s 2024-05-11T13:37:25.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:25 smithi165 ceph-mon[33235]: pgmap v109: 129 pgs: 129 active+clean; 322 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.2 MiB/s wr, 21 op/s 2024-05-11T13:37:27.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:27 smithi063 ceph-mon[29809]: pgmap v110: 129 pgs: 129 active+clean; 322 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 17 op/s 2024-05-11T13:37:27.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:27 smithi071 ceph-mon[33293]: pgmap v110: 129 pgs: 129 active+clean; 322 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 17 op/s 2024-05-11T13:37:27.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:27 smithi165 ceph-mon[33235]: pgmap v110: 129 pgs: 129 active+clean; 322 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 17 op/s 2024-05-11T13:37:29.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:29 smithi063 ceph-mon[29809]: pgmap v111: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-05-11T13:37:29.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:29 smithi071 ceph-mon[33293]: pgmap v111: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-05-11T13:37:29.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:29 smithi165 ceph-mon[33235]: pgmap v111: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-05-11T13:37:29.855 INFO:teuthology.orchestra.run.smithi063.stderr:1100000 of 3200000 tuples (34%) done (elapsed 111.45 s, remaining 212.77 s) 2024-05-11T13:37:31.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:31 smithi071 ceph-mon[33293]: pgmap v112: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:37:31.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:31 smithi165 ceph-mon[33235]: pgmap v112: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:37:31.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:31 smithi063 ceph-mon[29809]: pgmap v112: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:37:32.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:32 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:32.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:32 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:32.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:32 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:33.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:33 smithi165 ceph-mon[33235]: pgmap v113: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:37:33.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:33 smithi063 ceph-mon[29809]: pgmap v113: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:37:33.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:33 smithi071 ceph-mon[33293]: pgmap v113: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:37:34.109 INFO:teuthology.orchestra.run.smithi063.stderr:1200000 of 3200000 tuples (37%) done (elapsed 115.71 s, remaining 192.85 s) 2024-05-11T13:37:34.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:34 smithi165 ceph-mon[33235]: pgmap v114: 129 pgs: 129 active+clean; 365 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-05-11T13:37:34.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:34 smithi063 ceph-mon[29809]: pgmap v114: 129 pgs: 129 active+clean; 365 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-05-11T13:37:34.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:34 smithi071 ceph-mon[33293]: pgmap v114: 129 pgs: 129 active+clean; 365 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-05-11T13:37:37.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:37 smithi063 ceph-mon[29809]: pgmap v115: 129 pgs: 129 active+clean; 365 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-05-11T13:37:37.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:37 smithi071 ceph-mon[33293]: pgmap v115: 129 pgs: 129 active+clean; 365 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-05-11T13:37:37.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:37 smithi165 ceph-mon[33235]: pgmap v115: 129 pgs: 129 active+clean; 365 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-05-11T13:37:38.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:38 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:38.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:38 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:38.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:38 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:39.012 INFO:teuthology.orchestra.run.smithi063.stderr:1300000 of 3200000 tuples (40%) done (elapsed 120.61 s, remaining 176.28 s) 2024-05-11T13:37:39.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:39 smithi063 ceph-mon[29809]: pgmap v116: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:37:39.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:39 smithi071 ceph-mon[33293]: pgmap v116: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:37:39.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:39 smithi165 ceph-mon[33235]: pgmap v116: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:37:41.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:41 smithi063 ceph-mon[29809]: pgmap v117: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-05-11T13:37:41.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:41 smithi071 ceph-mon[33293]: pgmap v117: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-05-11T13:37:41.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:41 smithi165 ceph-mon[33235]: pgmap v117: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-05-11T13:37:42.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:42 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:42.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:42 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:42.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:42 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:43.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:43 smithi063 ceph-mon[29809]: pgmap v118: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-05-11T13:37:43.440 INFO:teuthology.orchestra.run.smithi063.stderr:1400000 of 3200000 tuples (43%) done (elapsed 125.04 s, remaining 160.76 s) 2024-05-11T13:37:43.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:43 smithi071 ceph-mon[33293]: pgmap v118: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-05-11T13:37:43.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:43 smithi165 ceph-mon[33235]: pgmap v118: 129 pgs: 129 active+clean; 395 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-05-11T13:37:45.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:45 smithi071 ceph-mon[33293]: pgmap v119: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 29 op/s 2024-05-11T13:37:45.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:45 smithi165 ceph-mon[33235]: pgmap v119: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 29 op/s 2024-05-11T13:37:45.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:45 smithi063 ceph-mon[29809]: pgmap v119: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 29 op/s 2024-05-11T13:37:47.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:47 smithi165 ceph-mon[33235]: pgmap v120: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.9 MiB/s wr, 20 op/s 2024-05-11T13:37:47.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:47 smithi063 ceph-mon[29809]: pgmap v120: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.9 MiB/s wr, 20 op/s 2024-05-11T13:37:47.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:47 smithi071 ceph-mon[33293]: pgmap v120: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.9 MiB/s wr, 20 op/s 2024-05-11T13:37:47.799 INFO:teuthology.orchestra.run.smithi063.stderr:1500000 of 3200000 tuples (46%) done (elapsed 129.40 s, remaining 146.65 s) 2024-05-11T13:37:48.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:48 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:48.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:48 smithi165 ceph-mon[33235]: pgmap v121: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 30 op/s 2024-05-11T13:37:48.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:48 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:48.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:48 smithi063 ceph-mon[29809]: pgmap v121: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 30 op/s 2024-05-11T13:37:48.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:48 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:48.699 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:48 smithi071 ceph-mon[33293]: pgmap v121: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 30 op/s 2024-05-11T13:37:51.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:51 smithi063 ceph-mon[29809]: pgmap v122: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.2 MiB/s wr, 20 op/s 2024-05-11T13:37:51.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:51 smithi071 ceph-mon[33293]: pgmap v122: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.2 MiB/s wr, 20 op/s 2024-05-11T13:37:51.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:51 smithi165 ceph-mon[33235]: pgmap v122: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.2 MiB/s wr, 20 op/s 2024-05-11T13:37:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:1600000 of 3200000 tuples (50%) done (elapsed 134.64 s, remaining 134.64 s) 2024-05-11T13:37:53.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:53 smithi063 ceph-mon[29809]: pgmap v123: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.2 MiB/s wr, 20 op/s 2024-05-11T13:37:53.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:53 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:53.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:53 smithi071 ceph-mon[33293]: pgmap v123: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.2 MiB/s wr, 20 op/s 2024-05-11T13:37:53.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:53 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:53.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:53 smithi165 ceph-mon[33235]: pgmap v123: 129 pgs: 129 active+clean; 441 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.2 MiB/s wr, 20 op/s 2024-05-11T13:37:53.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:53 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:37:55.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:55 smithi063 ceph-mon[29809]: pgmap v124: 129 pgs: 129 active+clean; 469 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 29 op/s 2024-05-11T13:37:55.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:55 smithi071 ceph-mon[33293]: pgmap v124: 129 pgs: 129 active+clean; 469 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 29 op/s 2024-05-11T13:37:55.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:55 smithi165 ceph-mon[33235]: pgmap v124: 129 pgs: 129 active+clean; 469 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 29 op/s 2024-05-11T13:37:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:1700000 of 3200000 tuples (53%) done (elapsed 138.64 s, remaining 122.33 s) 2024-05-11T13:37:57.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:57 smithi063 ceph-mon[29809]: pgmap v125: 129 pgs: 129 active+clean; 469 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.1 MiB/s wr, 19 op/s 2024-05-11T13:37:57.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:57 smithi071 ceph-mon[33293]: pgmap v125: 129 pgs: 129 active+clean; 469 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.1 MiB/s wr, 19 op/s 2024-05-11T13:37:57.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:57 smithi165 ceph-mon[33235]: pgmap v125: 129 pgs: 129 active+clean; 469 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.1 MiB/s wr, 19 op/s 2024-05-11T13:37:59.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:37:59 smithi063 ceph-mon[29809]: pgmap v126: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-11T13:37:59.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:37:59 smithi071 ceph-mon[33293]: pgmap v126: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-11T13:37:59.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:37:59 smithi165 ceph-mon[33235]: pgmap v126: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-11T13:38:01.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:01 smithi071 ceph-mon[33293]: pgmap v127: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:38:01.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:01 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:01.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:01 smithi165 ceph-mon[33235]: pgmap v127: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:38:01.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:01 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:01.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:01 smithi063 ceph-mon[29809]: pgmap v127: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:38:01.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:01 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:02.140 INFO:teuthology.orchestra.run.smithi063.stderr:1800000 of 3200000 tuples (56%) done (elapsed 143.74 s, remaining 111.80 s) 2024-05-11T13:38:03.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:03 smithi071 ceph-mon[33293]: pgmap v128: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:38:03.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:03 smithi165 ceph-mon[33235]: pgmap v128: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:38:03.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:03 smithi063 ceph-mon[29809]: pgmap v128: 129 pgs: 129 active+clean; 495 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:38:05.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:05 smithi165 ceph-mon[33235]: pgmap v129: 129 pgs: 129 active+clean; 522 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 41 op/s 2024-05-11T13:38:05.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:05 smithi063 ceph-mon[29809]: pgmap v129: 129 pgs: 129 active+clean; 522 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 41 op/s 2024-05-11T13:38:05.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:05 smithi071 ceph-mon[33293]: pgmap v129: 129 pgs: 129 active+clean; 522 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 41 op/s 2024-05-11T13:38:06.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:06 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:06.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:06 smithi165 ceph-mon[33235]: pgmap v130: 129 pgs: 129 active+clean; 522 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 32 op/s 2024-05-11T13:38:06.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:06 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:06.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:06 smithi063 ceph-mon[29809]: pgmap v130: 129 pgs: 129 active+clean; 522 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 32 op/s 2024-05-11T13:38:06.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:06 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:06.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:06 smithi071 ceph-mon[33293]: pgmap v130: 129 pgs: 129 active+clean; 522 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 32 op/s 2024-05-11T13:38:07.345 INFO:teuthology.orchestra.run.smithi063.stderr:1900000 of 3200000 tuples (59%) done (elapsed 148.94 s, remaining 101.91 s) 2024-05-11T13:38:07.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:07 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:07.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:07 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:07.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:07 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:08.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:38:08.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:08 smithi165 ceph-mon[33235]: pgmap v131: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 41 op/s 2024-05-11T13:38:08.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:38:08.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:08 smithi063 ceph-mon[29809]: pgmap v131: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 41 op/s 2024-05-11T13:38:08.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:38:08.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:08 smithi071 ceph-mon[33293]: pgmap v131: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 41 op/s 2024-05-11T13:38:09.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:09 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:38:09.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:09 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:38:09.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:09 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:38:09.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:09 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:38:09.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:09 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:38:09.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:09 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:38:09.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:09 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:38:09.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:09 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:38:09.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:09 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:38:10.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:10 smithi165 ceph-mon[33235]: pgmap v132: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 33 op/s 2024-05-11T13:38:10.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:10 smithi063 ceph-mon[29809]: pgmap v132: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 33 op/s 2024-05-11T13:38:10.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:10 smithi071 ceph-mon[33293]: pgmap v132: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 33 op/s 2024-05-11T13:38:11.066 INFO:teuthology.orchestra.run.smithi063.stderr:2000000 of 3200000 tuples (62%) done (elapsed 152.66 s, remaining 91.60 s) 2024-05-11T13:38:13.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:13 smithi063 ceph-mon[29809]: pgmap v133: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 33 op/s 2024-05-11T13:38:13.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:13 smithi071 ceph-mon[33293]: pgmap v133: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 33 op/s 2024-05-11T13:38:13.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:13 smithi165 ceph-mon[33235]: pgmap v133: 129 pgs: 129 active+clean; 564 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 33 op/s 2024-05-11T13:38:15.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:15 smithi063 ceph-mon[29809]: pgmap v134: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 41 op/s 2024-05-11T13:38:15.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:15 smithi071 ceph-mon[33293]: pgmap v134: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 41 op/s 2024-05-11T13:38:15.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:15 smithi165 ceph-mon[33235]: pgmap v134: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 41 op/s 2024-05-11T13:38:15.997 INFO:teuthology.orchestra.run.smithi063.stderr:2100000 of 3200000 tuples (65%) done (elapsed 157.60 s, remaining 82.55 s) 2024-05-11T13:38:16.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:16 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:16.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:16 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:16.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:16 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:38:17.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:17 smithi071 ceph-mon[33293]: pgmap v135: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-05-11T13:38:17.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:17 smithi165 ceph-mon[33235]: pgmap v135: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-05-11T13:38:17.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:17 smithi063 ceph-mon[29809]: pgmap v135: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-05-11T13:38:19.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:19 smithi071 ceph-mon[33293]: pgmap v136: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:19.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:19 smithi165 ceph-mon[33235]: pgmap v136: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:19.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:19 smithi063 ceph-mon[29809]: pgmap v136: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:21.219 INFO:teuthology.orchestra.run.smithi063.stderr:2200000 of 3200000 tuples (68%) done (elapsed 162.82 s, remaining 74.01 s) 2024-05-11T13:38:21.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:21 smithi071 ceph-mon[33293]: pgmap v137: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:38:21.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:21 smithi165 ceph-mon[33235]: pgmap v137: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:38:21.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:21 smithi063 ceph-mon[29809]: pgmap v137: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:38:23.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:23 smithi165 ceph-mon[33235]: pgmap v138: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:38:23.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:23 smithi063 ceph-mon[29809]: pgmap v138: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:38:23.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:23 smithi071 ceph-mon[33293]: pgmap v138: 129 pgs: 129 active+clean; 608 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 16 op/s 2024-05-11T13:38:25.525 INFO:teuthology.orchestra.run.smithi063.stderr:2300000 of 3200000 tuples (71%) done (elapsed 167.12 s, remaining 65.40 s) 2024-05-11T13:38:25.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:25 smithi165 ceph-mon[33235]: pgmap v139: 129 pgs: 129 active+clean; 640 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:25.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:25 smithi063 ceph-mon[29809]: pgmap v139: 129 pgs: 129 active+clean; 640 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:25.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:25 smithi071 ceph-mon[33293]: pgmap v139: 129 pgs: 129 active+clean; 640 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:27.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:27 smithi165 ceph-mon[33235]: pgmap v140: 129 pgs: 129 active+clean; 640 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 16 op/s 2024-05-11T13:38:27.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:27 smithi063 ceph-mon[29809]: pgmap v140: 129 pgs: 129 active+clean; 640 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 16 op/s 2024-05-11T13:38:27.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:27 smithi071 ceph-mon[33293]: pgmap v140: 129 pgs: 129 active+clean; 640 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 16 op/s 2024-05-11T13:38:28.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:28 smithi165 ceph-mon[33235]: pgmap v141: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:28.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:28 smithi063 ceph-mon[29809]: pgmap v141: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:28.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:28 smithi071 ceph-mon[33293]: pgmap v141: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2400000 of 3200000 tuples (75%) done (elapsed 172.48 s, remaining 57.49 s) 2024-05-11T13:38:31.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:31 smithi063 ceph-mon[29809]: pgmap v142: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-05-11T13:38:31.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:31 smithi071 ceph-mon[33293]: pgmap v142: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-05-11T13:38:31.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:31 smithi165 ceph-mon[33235]: pgmap v142: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-05-11T13:38:33.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:33 smithi063 ceph-mon[29809]: pgmap v143: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-05-11T13:38:33.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:33 smithi071 ceph-mon[33293]: pgmap v143: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-05-11T13:38:33.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:33 smithi165 ceph-mon[33235]: pgmap v143: 129 pgs: 129 active+clean; 670 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-05-11T13:38:35.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:35 smithi063 ceph-mon[29809]: pgmap v144: 129 pgs: 129 active+clean; 681 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:35.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:35 smithi071 ceph-mon[33293]: pgmap v144: 129 pgs: 129 active+clean; 681 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:35.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:35 smithi165 ceph-mon[33235]: pgmap v144: 129 pgs: 129 active+clean; 681 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-11T13:38:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2500000 of 3200000 tuples (78%) done (elapsed 177.74 s, remaining 49.77 s) 2024-05-11T13:38:37.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:37 smithi063 ceph-mon[29809]: pgmap v145: 129 pgs: 129 active+clean; 681 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 16 op/s 2024-05-11T13:38:37.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:37 smithi071 ceph-mon[33293]: pgmap v145: 129 pgs: 129 active+clean; 681 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 16 op/s 2024-05-11T13:38:37.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:37 smithi165 ceph-mon[33235]: pgmap v145: 129 pgs: 129 active+clean; 681 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 16 op/s 2024-05-11T13:38:39.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:39 smithi071 ceph-mon[33293]: pgmap v146: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-11T13:38:39.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:39 smithi165 ceph-mon[33235]: pgmap v146: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-11T13:38:39.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:39 smithi063 ceph-mon[29809]: pgmap v146: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-11T13:38:40.128 INFO:teuthology.orchestra.run.smithi063.stderr:2600000 of 3200000 tuples (81%) done (elapsed 181.73 s, remaining 41.94 s) 2024-05-11T13:38:41.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:41 smithi071 ceph-mon[33293]: pgmap v147: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 17 op/s 2024-05-11T13:38:41.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:41 smithi165 ceph-mon[33235]: pgmap v147: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 17 op/s 2024-05-11T13:38:41.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:41 smithi063 ceph-mon[29809]: pgmap v147: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 17 op/s 2024-05-11T13:38:43.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:43 smithi071 ceph-mon[33293]: pgmap v148: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 17 op/s 2024-05-11T13:38:43.481 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:43 smithi063 ceph-mon[29809]: pgmap v148: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 17 op/s 2024-05-11T13:38:43.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:43 smithi165 ceph-mon[33235]: pgmap v148: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 17 op/s 2024-05-11T13:38:45.447 INFO:teuthology.orchestra.run.smithi063.stderr:2700000 of 3200000 tuples (84%) done (elapsed 187.05 s, remaining 34.64 s) 2024-05-11T13:38:45.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:45 smithi165 ceph-mon[33235]: pgmap v149: 129 pgs: 129 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-11T13:38:45.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:45 smithi063 ceph-mon[29809]: pgmap v149: 129 pgs: 129 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-11T13:38:45.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:45 smithi071 ceph-mon[33293]: pgmap v149: 129 pgs: 129 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-11T13:38:47.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:47 smithi165 ceph-mon[33235]: pgmap v150: 129 pgs: 129 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.7 MiB/s wr, 19 op/s 2024-05-11T13:38:47.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:47 smithi063 ceph-mon[29809]: pgmap v150: 129 pgs: 129 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.7 MiB/s wr, 19 op/s 2024-05-11T13:38:47.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:47 smithi071 ceph-mon[33293]: pgmap v150: 129 pgs: 129 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.7 MiB/s wr, 19 op/s 2024-05-11T13:38:48.951 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-11T13:38:48.952 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:38:49.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:49 smithi063 ceph-mon[29809]: pgmap v151: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 28 op/s 2024-05-11T13:38:49.479 INFO:teuthology.orchestra.run.smithi063.stderr:2800000 of 3200000 tuples (87%) done (elapsed 191.08 s, remaining 27.30 s) 2024-05-11T13:38:49.501 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:38:49.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:49 smithi165 ceph-mon[33235]: pgmap v151: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 28 op/s 2024-05-11T13:38:49.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:49 smithi071 ceph-mon[33293]: pgmap v151: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 28 op/s 2024-05-11T13:38:51.554 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:51 smithi063 ceph-mon[29809]: pgmap v152: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 18 op/s 2024-05-11T13:38:51.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:51 smithi165 ceph-mon[33235]: pgmap v152: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 18 op/s 2024-05-11T13:38:51.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:51 smithi071 ceph-mon[33293]: pgmap v152: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 18 op/s 2024-05-11T13:38:52.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 -- 172.21.15.63:0/1453529192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc076210 msgr2=0x7fc7dc113f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:52.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 --2- 172.21.15.63:0/1453529192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc076210 0x7fc7dc113f50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009990 tx=0x7fc7c8009ca0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 -- 172.21.15.63:0/1453529192 shutdown_connections 2024-05-11T13:38:52.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 --2- 172.21.15.63:0/1453529192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7dc114490 0x7fc7dc116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 --2- 172.21.15.63:0/1453529192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7dc0758f0 0x7fc7dc075cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 --2- 172.21.15.63:0/1453529192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc076210 0x7fc7dc113f50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 -- 172.21.15.63:0/1453529192 >> 172.21.15.63:0/1453529192 conn(0x7fc7dc0fefd0 msgr2=0x7fc7dc1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:52.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.005+0000 7fc7e2800700 1 -- 172.21.15.63:0/1453529192 shutdown_connections 2024-05-11T13:38:52.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.006+0000 7fc7e2800700 1 -- 172.21.15.63:0/1453529192 wait complete. 2024-05-11T13:38:52.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 Processor -- start 2024-05-11T13:38:52.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 -- start start 2024-05-11T13:38:52.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc0758f0 0x7fc7dc19ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:52.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7dc076210 0x7fc7dc1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:52.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7dc114490 0x7fc7dc1a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:52.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc119b60 con 0x7fc7dc0758f0 2024-05-11T13:38:52.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1199e0 con 0x7fc7dc076210 2024-05-11T13:38:52.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.007+0000 7fc7e2800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc119ce0 con 0x7fc7dc114490 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.008+0000 7fc7e1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc0758f0 0x7fc7dc19ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.008+0000 7fc7e0a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7dc076210 0x7fc7dc1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.008+0000 7fc7e1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc0758f0 0x7fc7dc19ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53302/0 (socket says 172.21.15.63:53302) 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.008+0000 7fc7e1400700 1 -- 172.21.15.63:0/4047991850 learned_addr learned my addr 172.21.15.63:0/4047991850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.008+0000 7fc7e1e00700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7dc114490 0x7fc7dc1a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7e1400700 1 -- 172.21.15.63:0/4047991850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7dc114490 msgr2=0x7fc7dc1a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7e1400700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7dc114490 0x7fc7dc1a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7e1400700 1 -- 172.21.15.63:0/4047991850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7dc076210 msgr2=0x7fc7dc1a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7e1400700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7dc076210 0x7fc7dc1a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7e1400700 1 -- 172.21.15.63:0/4047991850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc1a4fc0 con 0x7fc7dc0758f0 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7e1e00700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7dc114490 0x7fc7dc1a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7e1400700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc0758f0 0x7fc7dc19ffb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000c9b0 tx=0x7fc7d000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.009+0000 7fc7da000700 1 -- 172.21.15.63:0/4047991850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0007b00 con 0x7fc7dc0758f0 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.010+0000 7fc7da000700 1 -- 172.21.15.63:0/4047991850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7d0007ca0 con 0x7fc7dc0758f0 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.010+0000 7fc7da000700 1 -- 172.21.15.63:0/4047991850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0018660 con 0x7fc7dc0758f0 2024-05-11T13:38:52.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.010+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc1a52b0 con 0x7fc7dc0758f0 2024-05-11T13:38:52.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.010+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc1acb30 con 0x7fc7dc0758f0 2024-05-11T13:38:52.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.012+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc005330 con 0x7fc7dc0758f0 2024-05-11T13:38:52.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.015+0000 7fc7da000700 1 -- 172.21.15.63:0/4047991850 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc7d000f450 con 0x7fc7dc0758f0 2024-05-11T13:38:52.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.017+0000 7fc7da000700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc7c40b29f0 0x7fc7c40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:52.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.018+0000 7fc7da000700 1 -- 172.21.15.63:0/4047991850 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fc7d00e2940 con 0x7fc7dc0758f0 2024-05-11T13:38:52.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.018+0000 7fc7e0a00700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc7c40b29f0 0x7fc7c40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:52.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.019+0000 7fc7e0a00700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc7c40b29f0 0x7fc7c40b4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8003760 tx=0x7fc7c800b540 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:52.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.024+0000 7fc7da000700 1 -- 172.21.15.63:0/4047991850 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7d0014030 con 0x7fc7dc0758f0 2024-05-11T13:38:52.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.389+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7bc0058f0 con 0x7fc7dc0758f0 2024-05-11T13:38:52.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.392+0000 7fc7da000700 1 -- 172.21.15.63:0/4047991850 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11942 (secure 0 0 0) 0x7fc7d00a9f30 con 0x7fc7dc0758f0 2024-05-11T13:38:52.388 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:38:52.388 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:38:14.230701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:38:52.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc7c40b29f0 msgr2=0x7fc7c40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:52.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc7c40b29f0 0x7fc7c40b4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8003760 tx=0x7fc7c800b540 comp rx=0 tx=0).stop 2024-05-11T13:38:52.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc0758f0 msgr2=0x7fc7dc19ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc0758f0 0x7fc7dc19ffb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000c9b0 tx=0x7fc7d000ccc0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 shutdown_connections 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc7c40b29f0 0x7fc7c40b4eb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7dc114490 0x7fc7dc1a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7dc0758f0 0x7fc7dc19ffb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 --2- 172.21.15.63:0/4047991850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7dc076210 0x7fc7dc1a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 >> 172.21.15.63:0/4047991850 conn(0x7fc7dc0fefd0 msgr2=0x7fc7dc0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:52.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.396+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 shutdown_connections 2024-05-11T13:38:52.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:52.397+0000 7fc7e2800700 1 -- 172.21.15.63:0/4047991850 wait complete. 2024-05-11T13:38:52.393 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 48 2024-05-11T13:38:52.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:52 smithi165 ceph-mon[33235]: pgmap v153: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 18 op/s 2024-05-11T13:38:52.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:52 smithi063 ceph-mon[29809]: pgmap v153: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 18 op/s 2024-05-11T13:38:52.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:52 smithi071 ceph-mon[33293]: pgmap v153: 129 pgs: 129 active+clean; 753 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 18 op/s 2024-05-11T13:38:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2900000 of 3200000 tuples (90%) done (elapsed 194.49 s, remaining 20.12 s) 2024-05-11T13:38:53.228 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-05-11T13:38:53.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-05-11T13:38:53.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:53 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/4047991850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:38:53.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:53 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/4047991850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:38:53.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/4047991850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:38:53.734 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:38:54.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:54 smithi165 ceph-mon[33235]: pgmap v154: 129 pgs: 129 active+clean; 789 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-05-11T13:38:54.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:54 smithi063 ceph-mon[29809]: pgmap v154: 129 pgs: 129 active+clean; 789 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-05-11T13:38:54.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:54 smithi071 ceph-mon[33293]: pgmap v154: 129 pgs: 129 active+clean; 789 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-05-11T13:38:56.137 INFO:teuthology.orchestra.run.smithi063.stderr:3000000 of 3200000 tuples (93%) done (elapsed 197.74 s, remaining 13.18 s) 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 -- 172.21.15.63:0/1121738335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1058f0 msgr2=0x7feeec105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 --2- 172.21.15.63:0/1121738335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1058f0 0x7feeec105cd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feedc009990 tx=0x7feedc009ca0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 -- 172.21.15.63:0/1121738335 shutdown_connections 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 --2- 172.21.15.63:0/1121738335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeec1078c0 0x7feeec112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 --2- 172.21.15.63:0/1121738335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1058f0 0x7feeec105cd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 --2- 172.21.15.63:0/1121738335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feeec106210 0x7feeec107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 -- 172.21.15.63:0/1121738335 >> 172.21.15.63:0/1121738335 conn(0x7feeec101060 msgr2=0x7feeec103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.156+0000 7feeeb200700 1 -- 172.21.15.63:0/1121738335 shutdown_connections 2024-05-11T13:38:56.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.157+0000 7feeeb200700 1 -- 172.21.15.63:0/1121738335 wait complete. 2024-05-11T13:38:56.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 Processor -- start 2024-05-11T13:38:56.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 -- start start 2024-05-11T13:38:56.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feeec1058f0 0x7feeec1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeec106210 0x7feeec1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1078c0 0x7feeec19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1157f0 con 0x7feeec1078c0 2024-05-11T13:38:56.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feeec115670 con 0x7feeec106210 2024-05-11T13:38:56.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.158+0000 7feeeb200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feeec115970 con 0x7feeec1058f0 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.159+0000 7feeea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1078c0 0x7feeec19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.159+0000 7feee1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeec106210 0x7feeec1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.159+0000 7feeea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1078c0 0x7feeec19b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53322/0 (socket says 172.21.15.63:53322) 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.159+0000 7feee9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feeec1058f0 0x7feeec1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.159+0000 7feeea800700 1 -- 172.21.15.63:0/3226729882 learned_addr learned my addr 172.21.15.63:0/3226729882 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feeea800700 1 -- 172.21.15.63:0/3226729882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feeec1058f0 msgr2=0x7feeec1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feeea800700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feeec1058f0 0x7feeec1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feeea800700 1 -- 172.21.15.63:0/3226729882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeec106210 msgr2=0x7feeec1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feeea800700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeec106210 0x7feeec1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feeea800700 1 -- 172.21.15.63:0/3226729882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec19c150 con 0x7feeec1078c0 2024-05-11T13:38:56.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feeea800700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1078c0 0x7feeec19b9f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feee4007d70 tx=0x7feee400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:56.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feee3400700 1 -- 172.21.15.63:0/3226729882 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4016020 con 0x7feeec1078c0 2024-05-11T13:38:56.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feee3400700 1 -- 172.21.15.63:0/3226729882 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feee400ac90 con 0x7feeec1078c0 2024-05-11T13:38:56.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.160+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec19c440 con 0x7feeec1078c0 2024-05-11T13:38:56.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.161+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec1a86b0 con 0x7feeec1078c0 2024-05-11T13:38:56.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.161+0000 7feee3400700 1 -- 172.21.15.63:0/3226729882 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40115e0 con 0x7feeec1078c0 2024-05-11T13:38:56.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.165+0000 7feee3400700 1 -- 172.21.15.63:0/3226729882 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7feee4019020 con 0x7feeec1078c0 2024-05-11T13:38:56.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.166+0000 7feefba44b80 1 Processor -- start 2024-05-11T13:38:56.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.167+0000 7feefba44b80 1 -- start start 2024-05-11T13:38:56.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.167+0000 7feefba44b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e460e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.167+0000 7feefba44b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4608d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.167+0000 7feefba44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4609250 0x5566e46152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.167+0000 7feefba44b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566e455abb0 con 0x5566e460e530 2024-05-11T13:38:56.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.167+0000 7feefba44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566e44f88f0 con 0x5566e4609250 2024-05-11T13:38:56.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.167+0000 7feefba44b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566e4556b70 con 0x5566e4608890 2024-05-11T13:38:56.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e460e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e460e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53326/0 (socket says 172.21.15.63:53326) 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 -- 172.21.15.63:0/321865928 learned_addr learned my addr 172.21.15.63:0/321865928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee9e00700 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4608d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 -- 172.21.15.63:0/321865928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 msgr2=0x5566e4608d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4608d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 -- 172.21.15.63:0/321865928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4609250 msgr2=0x5566e46152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4609250 0x5566e46152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee1400700 1 -- 172.21.15.63:0/321865928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e44a7a60 con 0x5566e460e530 2024-05-11T13:38:56.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.168+0000 7feee3400700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7feecc0b2b20 0x7feecc0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.169+0000 7feee3400700 1 -- 172.21.15.63:0/3226729882 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7feee40e2690 con 0x7feeec1078c0 2024-05-11T13:38:56.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.169+0000 7feee9e00700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7feecc0b2b20 0x7feecc0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.169+0000 7feee1400700 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e460e910 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feed4009b10 tx=0x7feed4009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944ef3e845a4de61 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:56.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.169+0000 7feedb400700 1 -- 172.21.15.63:0/321865928 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4017070 con 0x5566e460e530 2024-05-11T13:38:56.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.170+0000 7feedb400700 1 -- 172.21.15.63:0/321865928 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed400ed60 con 0x5566e460e530 2024-05-11T13:38:56.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.170+0000 7feedb400700 1 -- 172.21.15.63:0/321865928 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4013db0 con 0x5566e460e530 2024-05-11T13:38:56.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 -- 172.21.15.63:0/321865928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 msgr2=0x5566e460e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feee9e00700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7feecc0b2b20 0x7feecc0b4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feedc003760 tx=0x7feedc00b540 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:56.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e460e910 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feed4009b10 tx=0x7feed4009e20 comp rx=0 tx=0).stop 2024-05-11T13:38:56.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 -- 172.21.15.63:0/321865928 shutdown_connections 2024-05-11T13:38:56.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4609250 0x5566e46152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e460e910 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 --2- 172.21.15.63:0/321865928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4608d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 -- 172.21.15.63:0/321865928 >> 172.21.15.63:0/321865928 conn(0x5566e4536cf0 msgr2=0x5566e4603510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:56.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 -- 172.21.15.63:0/321865928 shutdown_connections 2024-05-11T13:38:56.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.171+0000 7feefba44b80 1 -- 172.21.15.63:0/321865928 wait complete. 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.172+0000 7feefba44b80 1 -- start start 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.172+0000 7feefba44b80 1 Processor -- start 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feefba44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e4614fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feefba44b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4609250 0x5566e4612350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feefba44b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e4612890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feefba44b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566e4615890 con 0x5566e460e530 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feefba44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566e44c0cb0 con 0x5566e4608890 2024-05-11T13:38:56.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feefba44b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566e4615a10 con 0x5566e4609250 2024-05-11T13:38:56.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feefba44b80 10 client.0 _ll_register_callbacks cb 0x5566e44ee510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:38:56.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feeea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e4612890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feeea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e4612890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53328/0 (socket says 172.21.15.63:53328) 2024-05-11T13:38:56.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feeea800700 1 -- 172.21.15.63:0/970698596 learned_addr learned my addr 172.21.15.63:0/970698596 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.173+0000 7feee9e00700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e4614fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feee1400700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4609250 0x5566e4612350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feeea800700 1 -- 172.21.15.63:0/970698596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4609250 msgr2=0x5566e4612350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feeea800700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4609250 0x5566e4612350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feeea800700 1 -- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 msgr2=0x5566e4614fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feeea800700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e4614fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feeea800700 1 -- 172.21.15.63:0/970698596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e45205b0 con 0x5566e460e530 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feee9e00700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e4614fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feee1400700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4609250 0x5566e4612350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feeea800700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e4612890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7feee40a9620 tx=0x7feee4013f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dd5d008095e8e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.174+0000 7feed3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.175+0000 7feed3e00700 1 -- 172.21.15.63:0/970698596 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40a9d90 con 0x5566e460e530 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.175+0000 7feed3e00700 1 -- 172.21.15.63:0/970698596 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7feee40910f0 con 0x5566e460e530 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.175+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed400ed60 con 0x5566e460e530 2024-05-11T13:38:56.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.175+0000 7feed3e00700 1 -- 172.21.15.63:0/970698596 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feee40a9690 con 0x5566e460e530 2024-05-11T13:38:56.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.175+0000 7feefba44b80 10 client.15678 fetch_fsmap learned FSMap version 48 2024-05-11T13:38:56.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.175+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5566e46201c0 con 0x5566e460e530 2024-05-11T13:38:56.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feed3e00700 1 -- 172.21.15.63:0/970698596 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed400ed60 con 0x5566e460e530 2024-05-11T13:38:56.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feed3e00700 1 -- 172.21.15.63:0/970698596 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7feee40a98f0 con 0x5566e460e530 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feefba44b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 48 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feefba44b80 10 client.15678 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feefba44b80 20 client.15678 populate_metadata read hostname 'smithi063' 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feefba44b80 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4622800 0x5566e4624c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feefba44b80 4 client.15678 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.176+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566e45d51f0 con 0x5566e4622800 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.177+0000 7feee9e00700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4622800 0x5566e4624c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.178+0000 7feee9e00700 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4622800 0x5566e4624c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e288753bcf731033 server_cookie=62621f6c26730661 in_seq=0 out_seq=0 2024-05-11T13:38:56.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.178+0000 7feed3e00700 10 client.15678 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:38:56.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feed3e00700 1 -- 172.21.15.63:0/970698596 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566e45d51f0 con 0x5566e4622800 2024-05-11T13:38:56.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feed3e00700 10 client.15678 handle_command_reply: tid=0 2024-05-11T13:38:56.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feefba44b80 1 client.15678 shutdown 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 msgr2=0x5566e4612890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feefba44b80 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e4612890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7feee40a9620 tx=0x7feee4013f30 comp rx=0 tx=0).stop 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 shutdown_connections 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feefba44b80 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e460e530 0x5566e4612890 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.180+0000 7feefba44b80 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4622800 0x5566e4624c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.181+0000 7feefba44b80 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e4614fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.181+0000 7feefba44b80 1 --2- 172.21.15.63:0/970698596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4609250 0x5566e4612350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.181+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 >> 172.21.15.63:0/970698596 conn(0x5566e4536cf0 msgr2=0x5566e4607880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:56.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.181+0000 7feed3e00700 0 client.15678 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:38:56.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.181+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 shutdown_connections 2024-05-11T13:38:56.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.181+0000 7feefba44b80 1 -- 172.21.15.63:0/970698596 wait complete. 2024-05-11T13:38:56.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.181+0000 7feefba44b80 20 client.15678 trim_cache size 0 max 16384 2024-05-11T13:38:56.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.207+0000 7feefba44b80 1 Processor -- start 2024-05-11T13:38:56.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.208+0000 7feefba44b80 1 -- start start 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.208+0000 7feefba44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e461e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.208+0000 7feefba44b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e4610c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.208+0000 7feefba44b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e460e530 0x5566e4611180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.208+0000 7feefba44b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feec800ed20 con 0x5566e4609250 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.208+0000 7feefba44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feec8010920 con 0x5566e4608890 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.208+0000 7feefba44b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feec800f3f0 con 0x5566e460e530 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e4610c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e4610c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53338/0 (socket says 172.21.15.63:53338) 2024-05-11T13:38:56.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 -- 172.21.15.63:0/1475224959 learned_addr learned my addr 172.21.15.63:0/1475224959 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee9e00700 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e461e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feeea800700 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e460e530 0x5566e4611180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 -- 172.21.15.63:0/1475224959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e460e530 msgr2=0x5566e4611180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e460e530 0x5566e4611180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 -- 172.21.15.63:0/1475224959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 msgr2=0x5566e461e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e461e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.209+0000 7feee1400700 1 -- 172.21.15.63:0/1475224959 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e4609030 con 0x5566e4609250 2024-05-11T13:38:56.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.210+0000 7feee1400700 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e4610c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feed4009ae0 tx=0x7feed4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195e7ee5b040bc29 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:56.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.210+0000 7feedaa00700 1 -- 172.21.15.63:0/1475224959 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4029070 con 0x5566e4609250 2024-05-11T13:38:56.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.210+0000 7feedaa00700 1 -- 172.21.15.63:0/1475224959 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed400ed60 con 0x5566e4609250 2024-05-11T13:38:56.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.210+0000 7feedaa00700 1 -- 172.21.15.63:0/1475224959 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4017040 con 0x5566e4609250 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.211+0000 7feefba44b80 1 -- 172.21.15.63:0/1475224959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 msgr2=0x5566e4610c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.211+0000 7feefba44b80 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e4610c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feed4009ae0 tx=0x7feed4009360 comp rx=0 tx=0).stop 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.211+0000 7feefba44b80 1 -- 172.21.15.63:0/1475224959 shutdown_connections 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.211+0000 7feefba44b80 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e460e530 0x5566e4611180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.211+0000 7feefba44b80 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e4608890 0x5566e461e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.211+0000 7feefba44b80 1 --2- 172.21.15.63:0/1475224959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e4610c40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.211+0000 7feefba44b80 1 -- 172.21.15.63:0/1475224959 >> 172.21.15.63:0/1475224959 conn(0x5566e4536cf0 msgr2=0x5566e460db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.212+0000 7feefba44b80 1 -- 172.21.15.63:0/1475224959 shutdown_connections 2024-05-11T13:38:56.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.212+0000 7feefba44b80 1 -- 172.21.15.63:0/1475224959 wait complete. 2024-05-11T13:38:56.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.212+0000 7feefba44b80 1 -- start start 2024-05-11T13:38:56.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.212+0000 7feefba44b80 1 Processor -- start 2024-05-11T13:38:56.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.213+0000 7feefba44b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4617c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.213+0000 7feefba44b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e46181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.213+0000 7feefba44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e460e530 0x5566e4618710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.213+0000 7feefba44b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feec8010250 con 0x5566e4609250 2024-05-11T13:38:56.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.213+0000 7feefba44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feec800fac0 con 0x5566e460e530 2024-05-11T13:38:56.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.213+0000 7feefba44b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feec800ccc0 con 0x5566e4608890 2024-05-11T13:38:56.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.213+0000 7feefba44b80 10 client.0 _ll_register_callbacks cb 0x5566e44ee510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e46181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e46181d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53344/0 (socket says 172.21.15.63:53344) 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 -- 172.21.15.63:0/230792201 learned_addr learned my addr 172.21.15.63:0/230792201 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee9e00700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4617c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feeea800700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e460e530 0x5566e4618710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 -- 172.21.15.63:0/230792201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 msgr2=0x5566e4617c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4617c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 -- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e460e530 msgr2=0x5566e4618710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:38:56.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e460e530 0x5566e4618710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee1400700 1 -- 172.21.15.63:0/230792201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e4695940 con 0x5566e4609250 2024-05-11T13:38:56.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.214+0000 7feee9e00700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4617c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:38:56.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.215+0000 7feee1400700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e46181d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feed4027040 tx=0x7feed40136a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed8749f6e40185e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:38:56.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.215+0000 7feed3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:38:56.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.215+0000 7feed3e00700 1 -- 172.21.15.63:0/230792201 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4032060 con 0x5566e4609250 2024-05-11T13:38:56.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.215+0000 7feed3e00700 1 -- 172.21.15.63:0/230792201 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7feed4013aa0 con 0x5566e4609250 2024-05-11T13:38:56.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.216+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed400ed60 con 0x5566e4609250 2024-05-11T13:38:56.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.216+0000 7feed3e00700 1 -- 172.21.15.63:0/230792201 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feed4029070 con 0x5566e4609250 2024-05-11T13:38:56.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.216+0000 7feefba44b80 10 client.15684 fetch_fsmap learned FSMap version 48 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.216+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5566e4696b80 con 0x5566e4609250 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.216+0000 7feed3e00700 1 -- 172.21.15.63:0/230792201 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed400ed60 con 0x5566e4609250 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.217+0000 7feed3e00700 1 -- 172.21.15.63:0/230792201 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7feed4006450 con 0x5566e4609250 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.217+0000 7feefba44b80 10 client.15684 fetch_fsmap finished waiting for FSMap version 48 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.217+0000 7feefba44b80 10 client.15684 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.217+0000 7feefba44b80 20 client.15684 populate_metadata read hostname 'smithi063' 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.217+0000 7feefba44b80 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4696fd0 0x5566e46993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.217+0000 7feefba44b80 4 client.15684 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-05-11T13:38:56.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.217+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x5566e4568590 con 0x5566e4696fd0 2024-05-11T13:38:56.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.218+0000 7feee9e00700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4696fd0 0x5566e46993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:38:56.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.218+0000 7feee9e00700 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4696fd0 0x5566e46993f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b94f004efe0b0dc server_cookie=449f275f47284b72 in_seq=0 out_seq=0 2024-05-11T13:38:56.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.218+0000 7feed3e00700 10 client.15684 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:38:56.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.493+0000 7feed3e00700 1 -- 172.21.15.63:0/230792201 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5566e4568590 con 0x5566e4696fd0 2024-05-11T13:38:56.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.493+0000 7feed3e00700 10 client.15684 handle_command_reply: tid=0 2024-05-11T13:38:56.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.493+0000 7feefba44b80 1 client.15684 shutdown 2024-05-11T13:38:56.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.493+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 msgr2=0x5566e46181d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.493+0000 7feefba44b80 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e46181d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feed4027040 tx=0x7feed40136a0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 shutdown_connections 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5566e4696fd0 0x5566e46993f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e460e530 0x5566e4618710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566e4608890 0x5566e4617c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 --2- 172.21.15.63:0/230792201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566e4609250 0x5566e46181d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feed3e00700 0 client.15684 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 >> 172.21.15.63:0/230792201 conn(0x5566e4536cf0 msgr2=0x5566e4603010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 shutdown_connections 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.494+0000 7feefba44b80 1 -- 172.21.15.63:0/230792201 wait complete. 2024-05-11T13:38:56.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.495+0000 7feefba44b80 20 client.15684 trim_cache size 0 max 16384 2024-05-11T13:38:56.491 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:38:56.491 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 48, 2024-05-11T13:38:56.491 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:38:56.491 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 14, 2024-05-11T13:38:56.491 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:38:56.491 INFO:teuthology.orchestra.run.smithi063.stdout: "92508e09": { 2024-05-11T13:38:56.491 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 14, 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 70.0, 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:38:56.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7feecc0b2b20 msgr2=0x7feecc0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7feecc0b2b20 0x7feecc0b4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feedc003760 tx=0x7feedc00b540 comp rx=0 tx=0).stop 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1078c0 msgr2=0x7feeec19b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:38:56.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1078c0 0x7feeec19b9f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feee4007d70 tx=0x7feee400a340 comp rx=0 tx=0).stop 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 shutdown_connections 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7feecc0b2b20 0x7feecc0b4fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeec1078c0 0x7feeec19b9f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feeec1058f0 0x7feeec1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 --2- 172.21.15.63:0/3226729882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeec106210 0x7feeec1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 >> 172.21.15.63:0/3226729882 conn(0x7feeec101060 msgr2=0x7feeec102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 shutdown_connections 2024-05-11T13:38:56.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:38:56.496+0000 7feeeb200700 1 -- 172.21.15.63:0/3226729882 wait complete. 2024-05-11T13:38:57.418 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:57 smithi063 ceph-mon[29809]: pgmap v155: 129 pgs: 129 active+clean; 789 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.8 MiB/s wr, 18 op/s 2024-05-11T13:38:57.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:57 smithi071 ceph-mon[33293]: pgmap v155: 129 pgs: 129 active+clean; 789 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.8 MiB/s wr, 18 op/s 2024-05-11T13:38:57.494 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '92508e09', quiesce took 8.5 seconds. Will release after: 10 2024-05-11T13:38:57.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:57 smithi165 ceph-mon[33235]: pgmap v155: 129 pgs: 129 active+clean; 789 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.8 MiB/s wr, 18 op/s 2024-05-11T13:38:59.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:38:59 smithi063 ceph-mon[29809]: pgmap v156: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-05-11T13:38:59.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:38:59 smithi071 ceph-mon[33293]: pgmap v156: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-05-11T13:38:59.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:38:59 smithi165 ceph-mon[33235]: pgmap v156: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-05-11T13:39:01.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:01 smithi063 ceph-mon[29809]: pgmap v157: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:39:01.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:01 smithi071 ceph-mon[33293]: pgmap v157: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:39:01.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:01 smithi165 ceph-mon[33235]: pgmap v157: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:39:03.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:03 smithi071 ceph-mon[33293]: pgmap v158: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:39:03.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:03 smithi165 ceph-mon[33235]: pgmap v158: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:39:03.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:03 smithi063 ceph-mon[29809]: pgmap v158: 129 pgs: 129 active+clean; 817 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:39:05.448 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:05 smithi071 ceph-mon[33293]: pgmap v159: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 37 op/s 2024-05-11T13:39:05.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:05 smithi165 ceph-mon[33235]: pgmap v159: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 37 op/s 2024-05-11T13:39:05.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:05 smithi063 ceph-mon[29809]: pgmap v159: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 37 op/s 2024-05-11T13:39:07.495 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:39:07.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:07 smithi165 ceph-mon[33235]: pgmap v160: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-11T13:39:07.584 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:07 smithi063 ceph-mon[29809]: pgmap v160: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-11T13:39:07.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:07 smithi071 ceph-mon[33293]: pgmap v160: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-11T13:39:07.963 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:39:09.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:09 smithi165 ceph-mon[33235]: pgmap v161: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 33 op/s 2024-05-11T13:39:09.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:09 smithi063 ceph-mon[29809]: pgmap v161: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 33 op/s 2024-05-11T13:39:09.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:09 smithi071 ceph-mon[33293]: pgmap v161: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 33 op/s 2024-05-11T13:39:10.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.251+0000 7f604e600700 1 -- 172.21.15.63:0/517064858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048108510 msgr2=0x7f60481088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:10.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.251+0000 7f604e600700 1 --2- 172.21.15.63:0/517064858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048108510 0x7f60481088f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6038009990 tx=0x7f6038009ca0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.252+0000 7f604e600700 1 -- 172.21.15.63:0/517064858 shutdown_connections 2024-05-11T13:39:10.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.252+0000 7f604e600700 1 --2- 172.21.15.63:0/517064858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6048103fa0 0x7f6048110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.252+0000 7f604e600700 1 --2- 172.21.15.63:0/517064858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048108510 0x7f60481088f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.252+0000 7f604e600700 1 --2- 172.21.15.63:0/517064858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6048103600 0x7f6048103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.252+0000 7f604e600700 1 -- 172.21.15.63:0/517064858 >> 172.21.15.63:0/517064858 conn(0x7f604807a6b0 msgr2=0x7f604807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:10.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.252+0000 7f604e600700 1 -- 172.21.15.63:0/517064858 shutdown_connections 2024-05-11T13:39:10.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.252+0000 7f604e600700 1 -- 172.21.15.63:0/517064858 wait complete. 2024-05-11T13:39:10.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.253+0000 7f604e600700 1 Processor -- start 2024-05-11T13:39:10.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.253+0000 7f604e600700 1 -- start start 2024-05-11T13:39:10.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f604e600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6048103600 0x7f604819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:10.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f604e600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048103fa0 0x7f604819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:10.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f604e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6048108510 0x7f60481a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:10.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f604e600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6048113630 con 0x7f6048103fa0 2024-05-11T13:39:10.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f604e600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60481134b0 con 0x7f6048108510 2024-05-11T13:39:10.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f604e600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60481137b0 con 0x7f6048103600 2024-05-11T13:39:10.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f6046a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048103fa0 0x7f604819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:10.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f6047400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6048103600 0x7f604819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:10.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f6046a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048103fa0 0x7f604819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46454/0 (socket says 172.21.15.63:46454) 2024-05-11T13:39:10.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f6046a00700 1 -- 172.21.15.63:0/99524262 learned_addr learned my addr 172.21.15.63:0/99524262 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:10.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.254+0000 7f6047e00700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6048108510 0x7f60481a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f6046a00700 1 -- 172.21.15.63:0/99524262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6048103600 msgr2=0x7f604819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f6046a00700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6048103600 0x7f604819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f6046a00700 1 -- 172.21.15.63:0/99524262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6048108510 msgr2=0x7f60481a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f6046a00700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6048108510 0x7f60481a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f6046a00700 1 -- 172.21.15.63:0/99524262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60481a0d90 con 0x7f6048103fa0 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f6047400700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6048103600 0x7f604819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f6046a00700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048103fa0 0x7f604819c2c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f603400d980 tx=0x7f603400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f603fe00700 1 -- 172.21.15.63:0/99524262 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034016070 con 0x7f6048103fa0 2024-05-11T13:39:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f603fe00700 1 -- 172.21.15.63:0/99524262 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6034004830 con 0x7f6048103fa0 2024-05-11T13:39:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f603fe00700 1 -- 172.21.15.63:0/99524262 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034005440 con 0x7f6048103fa0 2024-05-11T13:39:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.255+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60481a1080 con 0x7f6048103fa0 2024-05-11T13:39:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.256+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60481a8900 con 0x7f6048103fa0 2024-05-11T13:39:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.257+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604806a240 con 0x7f6048103fa0 2024-05-11T13:39:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.259+0000 7f603fe00700 1 -- 172.21.15.63:0/99524262 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f60340055e0 con 0x7f6048103fa0 2024-05-11T13:39:10.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.261+0000 7f603fe00700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f602c0b29f0 0x7f602c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:10.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.261+0000 7f603fe00700 1 -- 172.21.15.63:0/99524262 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f60340e2d10 con 0x7f6048103fa0 2024-05-11T13:39:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.262+0000 7f6047400700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f602c0b29f0 0x7f602c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.262+0000 7f6047400700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f602c0b29f0 0x7f602c0b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6038003760 tx=0x7f603800b540 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:10.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.267+0000 7f603fe00700 1 -- 172.21.15.63:0/99524262 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f603401f020 con 0x7f6048103fa0 2024-05-11T13:39:10.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:10 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:39:10.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.560+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60481a8b00 con 0x7f6048103fa0 2024-05-11T13:39:10.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.561+0000 7f603fe00700 1 -- 172.21.15.63:0/99524262 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11942 (secure 0 0 0) 0x7f60340aa300 con 0x7f6048103fa0 2024-05-11T13:39:10.556 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:39:10.556 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:38:14.230701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:39:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f602c0b29f0 msgr2=0x7f602c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f602c0b29f0 0x7f602c0b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6038003760 tx=0x7f603800b540 comp rx=0 tx=0).stop 2024-05-11T13:39:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048103fa0 msgr2=0x7f604819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048103fa0 0x7f604819c2c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f603400d980 tx=0x7f603400dc90 comp rx=0 tx=0).stop 2024-05-11T13:39:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 shutdown_connections 2024-05-11T13:39:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f602c0b29f0 0x7f602c0b4eb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6048108510 0x7f60481a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6048103600 0x7f604819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 --2- 172.21.15.63:0/99524262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6048103fa0 0x7f604819c2c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 >> 172.21.15.63:0/99524262 conn(0x7f604807a6b0 msgr2=0x7f6048101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 shutdown_connections 2024-05-11T13:39:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:10.565+0000 7f604e600700 1 -- 172.21.15.63:0/99524262 wait complete. 2024-05-11T13:39:10.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:10 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:39:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 48 2024-05-11T13:39:10.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:10 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:39:11.422 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 92508e09 --release --await' 2024-05-11T13:39:11.422 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 92508e09 --release --await 2024-05-11T13:39:11.492 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:11 smithi063 ceph-mon[29809]: pgmap v162: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 405 KiB/s wr, 24 op/s 2024-05-11T13:39:11.492 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:11 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:11.492 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:11 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:11.492 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:11 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/99524262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:39:11.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:11 smithi165 ceph-mon[33235]: pgmap v162: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 405 KiB/s wr, 24 op/s 2024-05-11T13:39:11.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:11 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:11.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:11 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:11.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:11 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/99524262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:39:11.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:11 smithi071 ceph-mon[33293]: pgmap v162: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 405 KiB/s wr, 24 op/s 2024-05-11T13:39:11.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:11 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:11.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:11 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:11.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:11 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/99524262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:39:11.908 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:39:12.392 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:12 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:39:12.393 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:12 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:39:12.393 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:12 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:12.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:12 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:39:12.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:12 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:39:12.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:12 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:12.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:12 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:39:12.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:12 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:39:12.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:12 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:39:13.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:13 smithi165 ceph-mon[33235]: pgmap v163: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 405 KiB/s wr, 24 op/s 2024-05-11T13:39:13.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:13 smithi063 ceph-mon[29809]: pgmap v163: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 405 KiB/s wr, 24 op/s 2024-05-11T13:39:13.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:13 smithi071 ceph-mon[33293]: pgmap v163: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 405 KiB/s wr, 24 op/s 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 -- 172.21.15.63:0/3270917344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf44075780 msgr2=0x7fcf44075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3270917344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf44075780 0x7fcf44075b60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009990 tx=0x7fcf34009ca0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 -- 172.21.15.63:0/3270917344 shutdown_connections 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3270917344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf4410ff50 0x7fcf44112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3270917344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf44075780 0x7fcf44075b60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3270917344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf440760a0 0x7fcf4410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 -- 172.21.15.63:0/3270917344 >> 172.21.15.63:0/3270917344 conn(0x7fcf4407a6b0 msgr2=0x7fcf4407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 -- 172.21.15.63:0/3270917344 shutdown_connections 2024-05-11T13:39:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.315+0000 7fcf4a200700 1 -- 172.21.15.63:0/3270917344 wait complete. 2024-05-11T13:39:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 Processor -- start 2024-05-11T13:39:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 -- start start 2024-05-11T13:39:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf44075780 0x7fcf4419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf440760a0 0x7fcf4419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf4410ff50 0x7fcf441a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441157f0 con 0x7fcf4410ff50 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44115670 con 0x7fcf440760a0 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.317+0000 7fcf4a200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44115970 con 0x7fcf44075780 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf4410ff50 0x7fcf441a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf4410ff50 0x7fcf441a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46476/0 (socket says 172.21.15.63:46476) 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 -- 172.21.15.63:0/3358709632 learned_addr learned my addr 172.21.15.63:0/3358709632 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43400700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf44075780 0x7fcf4419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf440760a0 0x7fcf4419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 -- 172.21.15.63:0/3358709632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf44075780 msgr2=0x7fcf4419be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf44075780 0x7fcf4419be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 -- 172.21.15.63:0/3358709632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf440760a0 msgr2=0x7fcf4419c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf440760a0 0x7fcf4419c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43e00700 1 -- 172.21.15.63:0/3358709632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf441a0e60 con 0x7fcf4410ff50 2024-05-11T13:39:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf43400700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf44075780 0x7fcf4419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:39:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.318+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf440760a0 0x7fcf4419c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:39:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.319+0000 7fcf43e00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf4410ff50 0x7fcf441a0720 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c007ee0 tx=0x7fcf3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.319+0000 7fcf3be00700 1 -- 172.21.15.63:0/3358709632 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00dbd0 con 0x7fcf4410ff50 2024-05-11T13:39:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.319+0000 7fcf3be00700 1 -- 172.21.15.63:0/3358709632 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf3c00dd70 con 0x7fcf4410ff50 2024-05-11T13:39:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.319+0000 7fcf3be00700 1 -- 172.21.15.63:0/3358709632 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0058d0 con 0x7fcf4410ff50 2024-05-11T13:39:14.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.319+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf441a1150 con 0x7fcf4410ff50 2024-05-11T13:39:14.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.319+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a89d0 con 0x7fcf4410ff50 2024-05-11T13:39:14.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.323+0000 7fcf3be00700 1 -- 172.21.15.63:0/3358709632 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcf3c01a020 con 0x7fcf4410ff50 2024-05-11T13:39:14.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 Processor -- start 2024-05-11T13:39:14.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 -- start start 2024-05-11T13:39:14.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82c9a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82c99ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82c9aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c82c8f0b20 con 0x55c82c99ef60 2024-05-11T13:39:14.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c82c88e860 con 0x55c82c9a3fd0 2024-05-11T13:39:14.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.324+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c82c8ecae0 con 0x55c82c99e5a0 2024-05-11T13:39:14.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf42a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82c9aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf3be00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcf280b2a70 0x7fcf280b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf42a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82c9aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46480/0 (socket says 172.21.15.63:46480) 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf42a00700 1 -- 172.21.15.63:0/3362768080 learned_addr learned my addr 172.21.15.63:0/3362768080 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf43400700 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82c99ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82c9a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf3be00700 1 -- 172.21.15.63:0/3358709632 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fcf3c0e21a0 con 0x7fcf4410ff50 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.325+0000 7fcf43e00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcf280b2a70 0x7fcf280b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf42a00700 1 -- 172.21.15.63:0/3362768080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 msgr2=0x55c82c99ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82c99ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf42a00700 1 -- 172.21.15.63:0/3362768080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 msgr2=0x55c82c9a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82c9a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf42a00700 1 -- 172.21.15.63:0/3362768080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82c83d9d0 con 0x55c82c99ef60 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82c9aad90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcf30014aa0 tx=0x7fcf3000c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b108571c712538e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf43e00700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcf280b2a70 0x7fcf280b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0df560 tx=0x7fcf3c004ef0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf38c00700 1 -- 172.21.15.63:0/3362768080 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf300140d0 con 0x55c82c99ef60 2024-05-11T13:39:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf38c00700 1 -- 172.21.15.63:0/3362768080 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf30014270 con 0x55c82c99ef60 2024-05-11T13:39:14.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.326+0000 7fcf38c00700 1 -- 172.21.15.63:0/3362768080 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30005580 con 0x55c82c99ef60 2024-05-11T13:39:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.327+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3362768080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 msgr2=0x55c82c9aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.327+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82c9aad90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcf30014aa0 tx=0x7fcf3000c5c0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.327+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3362768080 shutdown_connections 2024-05-11T13:39:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.327+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82c9aad90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.327+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82c9a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.327+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3362768080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82c99ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.327+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3362768080 >> 172.21.15.63:0/3362768080 conn(0x55c82c8ccc60 msgr2=0x55c82c998e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.328+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3362768080 shutdown_connections 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.328+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3362768080 wait complete. 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.328+0000 7fcf54a99b80 1 -- start start 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.328+0000 7fcf54a99b80 1 Processor -- start 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9aa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99ef60 0x55c82c9a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82c9a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c82c9ab330 con 0x55c82c99e5a0 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c82c856c20 con 0x55c82c9a3fd0 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c82c9ab4b0 con 0x55c82c99ef60 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf54a99b80 10 client.0 _ll_register_callbacks cb 0x55c82c897d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:39:14.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf42a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf42a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9aa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46486/0 (socket says 172.21.15.63:46486) 2024-05-11T13:39:14.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.329+0000 7fcf42a00700 1 -- 172.21.15.63:0/3244269467 learned_addr learned my addr 172.21.15.63:0/3244269467 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf43400700 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99ef60 0x55c82c9a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf42a00700 1 -- 172.21.15.63:0/3244269467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99ef60 msgr2=0x55c82c9a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99ef60 0x55c82c9a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf42a00700 1 -- 172.21.15.63:0/3244269467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 msgr2=0x55c82c9a8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82c9a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf42a00700 1 -- 172.21.15.63:0/3244269467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82c8b6520 con 0x55c82c99e5a0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf42a00700 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9aa910 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcf30014100 tx=0x7fcf300141e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c408c5d12b45f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.330+0000 7fcf2d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.331+0000 7fcf2d600700 1 -- 172.21.15.63:0/3244269467 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30027020 con 0x55c82c99e5a0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.331+0000 7fcf2d600700 1 -- 172.21.15.63:0/3244269467 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fcf30004b60 con 0x55c82c99e5a0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.331+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf30014270 con 0x55c82c99e5a0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.331+0000 7fcf2d600700 1 -- 172.21.15.63:0/3244269467 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcf30004d60 con 0x55c82c99e5a0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.331+0000 7fcf54a99b80 10 client.15702 fetch_fsmap learned FSMap version 48 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.331+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c82c9b5b30 con 0x55c82c99e5a0 2024-05-11T13:39:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.331+0000 7fcf2d600700 1 -- 172.21.15.63:0/3244269467 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf30014270 con 0x55c82c99e5a0 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf2d600700 1 -- 172.21.15.63:0/3244269467 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fcf30012610 con 0x55c82c99e5a0 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf54a99b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 48 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf54a99b80 10 client.15702 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf54a99b80 20 client.15702 populate_metadata read hostname 'smithi063' 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82c9b80b0 0x55c82c9ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf54a99b80 4 client.15702 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c82c96b2b0 con 0x55c82c9b80b0 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.332+0000 7fcf43400700 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82c9b80b0 0x55c82c9ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.333+0000 7fcf43400700 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82c9b80b0 0x55c82c9ba4a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a3465ef8374a1c server_cookie=d38e62f003ccd2de in_seq=0 out_seq=0 2024-05-11T13:39:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.333+0000 7fcf2d600700 10 client.15702 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.335+0000 7fcf2d600700 1 -- 172.21.15.63:0/3244269467 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c82c96b2b0 con 0x55c82c9b80b0 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.335+0000 7fcf2d600700 10 client.15702 handle_command_reply: tid=0 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.335+0000 7fcf54a99b80 1 client.15702 shutdown 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 msgr2=0x55c82c9aa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9aa910 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcf30014100 tx=0x7fcf300141e0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 shutdown_connections 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82c9b80b0 0x55c82c9ba4a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82c9a8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9aa910 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/3244269467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99ef60 0x55c82c9a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf2d600700 0 client.15702 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 >> 172.21.15.63:0/3244269467 conn(0x55c82c8ccc60 msgr2=0x55c82c993960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 shutdown_connections 2024-05-11T13:39:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 1 -- 172.21.15.63:0/3244269467 wait complete. 2024-05-11T13:39:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.336+0000 7fcf54a99b80 20 client.15702 trim_cache size 0 max 16384 2024-05-11T13:39:14.333 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:14 smithi063 ceph-mon[29809]: pgmap v164: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 406 KiB/s wr, 28 op/s 2024-05-11T13:39:14.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.367+0000 7fcf54a99b80 1 Processor -- start 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf54a99b80 1 -- start start 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c99ef60 0x55c82c9bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c9a3fd0 0x55c82c9bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441b17f0 con 0x55c82c99e5a0 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441b33f0 con 0x55c82c99ef60 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441b1ec0 con 0x55c82c9a3fd0 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.368+0000 7fcf42a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9937e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46498/0 (socket says 172.21.15.63:46498) 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 -- 172.21.15.63:0/1836530477 learned_addr learned my addr 172.21.15.63:0/1836530477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf43400700 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c99ef60 0x55c82c9bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 -- 172.21.15.63:0/1836530477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c9a3fd0 msgr2=0x55c82c9bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c9a3fd0 0x55c82c9bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 -- 172.21.15.63:0/1836530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c99ef60 msgr2=0x55c82c9bd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c99ef60 0x55c82c9bd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 -- 172.21.15.63:0/1836530477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82c9bdf50 con 0x55c82c99e5a0 2024-05-11T13:39:14.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.369+0000 7fcf42a00700 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9937e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcf30005a80 tx=0x7fcf3000ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1504b38dfb3de842 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:14.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.370+0000 7fcf2fe00700 1 -- 172.21.15.63:0/1836530477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3003c070 con 0x55c82c99e5a0 2024-05-11T13:39:14.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.370+0000 7fcf2fe00700 1 -- 172.21.15.63:0/1836530477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf30005680 con 0x55c82c99e5a0 2024-05-11T13:39:14.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.370+0000 7fcf2fe00700 1 -- 172.21.15.63:0/1836530477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30027070 con 0x55c82c99e5a0 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 -- 172.21.15.63:0/1836530477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 msgr2=0x55c82c9937e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9937e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcf30005a80 tx=0x7fcf3000ce90 comp rx=0 tx=0).stop 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 -- 172.21.15.63:0/1836530477 shutdown_connections 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c9a3fd0 0x55c82c9bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99e5a0 0x55c82c9937e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/1836530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c99ef60 0x55c82c9bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 -- 172.21.15.63:0/1836530477 >> 172.21.15.63:0/1836530477 conn(0x55c82c8ccc60 msgr2=0x55c82c9930d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 -- 172.21.15.63:0/1836530477 shutdown_connections 2024-05-11T13:39:14.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 -- 172.21.15.63:0/1836530477 wait complete. 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.371+0000 7fcf54a99b80 1 -- start start 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 1 Processor -- start 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82ca1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82ca1c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82ca16560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441b2d20 con 0x55c82c99ef60 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441b2590 con 0x55c82c9a3fd0 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441af790 con 0x55c82c99e5a0 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.372+0000 7fcf54a99b80 10 client.0 _ll_register_callbacks cb 0x55c82c897d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82ca1c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82ca1c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46508/0 (socket says 172.21.15.63:46508) 2024-05-11T13:39:14.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 -- 172.21.15.63:0/4234223298 learned_addr learned my addr 172.21.15.63:0/4234223298 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:14.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf42a00700 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82ca1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf42a00700 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82ca16560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 -- 172.21.15.63:0/4234223298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 msgr2=0x55c82ca1bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82ca1bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 -- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 msgr2=0x55c82ca16560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:14.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82ca16560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 -- 172.21.15.63:0/4234223298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82ca16c60 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf43400700 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82ca1c270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcf34000c00 tx=0x7fcf3400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348cb2132fcdf8f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.373+0000 7fcf2d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.374+0000 7fcf2d600700 1 -- 172.21.15.63:0/4234223298 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400bc00 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.374+0000 7fcf2d600700 1 -- 172.21.15.63:0/4234223298 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7fcf3400bda0 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.374+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf30005680 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.374+0000 7fcf2d600700 1 -- 172.21.15.63:0/4234223298 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcf34039050 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.374+0000 7fcf54a99b80 10 client.15708 fetch_fsmap learned FSMap version 48 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.374+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c82ca235a0 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.374+0000 7fcf2d600700 1 -- 172.21.15.63:0/4234223298 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf30005680 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf2d600700 1 -- 172.21.15.63:0/4234223298 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fcf34003bf0 con 0x55c82c99ef60 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf54a99b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 48 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf54a99b80 10 client.15708 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf54a99b80 20 client.15708 populate_metadata read hostname 'smithi063' 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82ca23c80 0x55c82ca260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:14.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf54a99b80 4 client.15708 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "92508e09", "release": true, "await": true}] 2024-05-11T13:39:14.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "92508e09", "release": true, "await": true}) -- 0x55c82c8fe500 con 0x55c82ca23c80 2024-05-11T13:39:14.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.375+0000 7fcf42a00700 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82ca23c80 0x55c82ca260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:14.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.376+0000 7fcf42a00700 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82ca23c80 0x55c82ca260a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=945f218b91ac788f server_cookie=7b2bdb220bd1846a in_seq=0 out_seq=0 2024-05-11T13:39:14.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.376+0000 7fcf2d600700 10 client.15708 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.434+0000 7fcf2d600700 1 -- 172.21.15.63:0/4234223298 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c82c8fe500 con 0x55c82ca23c80 2024-05-11T13:39:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.434+0000 7fcf2d600700 10 client.15708 handle_command_reply: tid=0 2024-05-11T13:39:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.434+0000 7fcf54a99b80 1 client.15708 shutdown 2024-05-11T13:39:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.435+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 msgr2=0x55c82ca1c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.435+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82ca1c270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcf34000c00 tx=0x7fcf3400b7d0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.436+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 shutdown_connections 2024-05-11T13:39:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.436+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55c82ca23c80 0x55c82ca260a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.436+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c82c9a3fd0 0x55c82ca16560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.436+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c82c99e5a0 0x55c82ca1bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.436+0000 7fcf54a99b80 1 --2- 172.21.15.63:0/4234223298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c82c99ef60 0x55c82ca1c270 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.436+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 >> 172.21.15.63:0/4234223298 conn(0x55c82c8ccc60 msgr2=0x55c82c993960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.436+0000 7fcf2d600700 0 client.15708 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.437+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 shutdown_connections 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.437+0000 7fcf54a99b80 1 -- 172.21.15.63:0/4234223298 wait complete. 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.437+0000 7fcf54a99b80 20 client.15708 trim_cache size 0 max 16384 2024-05-11T13:39:14.432 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 48, 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 16, 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "92508e09": { 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 16, 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:39:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 70.0, 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:14.434 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:39:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.440+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcf280b2a70 msgr2=0x7fcf280b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.440+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcf280b2a70 0x7fcf280b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0df560 tx=0x7fcf3c004ef0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.440+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf4410ff50 msgr2=0x7fcf441a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.440+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf4410ff50 0x7fcf441a0720 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c007ee0 tx=0x7fcf3c00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.442+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 shutdown_connections 2024-05-11T13:39:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.442+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fcf280b2a70 0x7fcf280b4f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.442+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf4410ff50 0x7fcf441a0720 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.442+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf44075780 0x7fcf4419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.442+0000 7fcf4a200700 1 --2- 172.21.15.63:0/3358709632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf440760a0 0x7fcf4419c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.442+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 >> 172.21.15.63:0/3358709632 conn(0x7fcf4407a6b0 msgr2=0x7fcf44101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.443+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 shutdown_connections 2024-05-11T13:39:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:14.443+0000 7fcf4a200700 1 -- 172.21.15.63:0/3358709632 wait complete. 2024-05-11T13:39:14.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:14 smithi165 ceph-mon[33235]: pgmap v164: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 406 KiB/s wr, 28 op/s 2024-05-11T13:39:14.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:14 smithi071 ceph-mon[33293]: pgmap v164: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 406 KiB/s wr, 28 op/s 2024-05-11T13:39:15.433 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '92508e09', seconds elapsed: 26.5 2024-05-11T13:39:17.409 INFO:teuthology.orchestra.run.smithi063.stderr:3100000 of 3200000 tuples (96%) done (elapsed 219.01 s, remaining 7.06 s) 2024-05-11T13:39:17.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:17 smithi063 ceph-mon[29809]: pgmap v165: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-05-11T13:39:17.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:17 smithi071 ceph-mon[33293]: pgmap v165: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-05-11T13:39:17.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:17 smithi165 ceph-mon[33235]: pgmap v165: 129 pgs: 129 active+clean; 821 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-05-11T13:39:19.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:19 smithi071 ceph-mon[33293]: pgmap v166: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.2 MiB/s wr, 15 op/s 2024-05-11T13:39:19.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:19 smithi165 ceph-mon[33235]: pgmap v166: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.2 MiB/s wr, 15 op/s 2024-05-11T13:39:19.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:19 smithi063 ceph-mon[29809]: pgmap v166: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.2 MiB/s wr, 15 op/s 2024-05-11T13:39:21.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:21 smithi071 ceph-mon[33293]: pgmap v167: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 MiB/s wr, 11 op/s 2024-05-11T13:39:21.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:21 smithi165 ceph-mon[33235]: pgmap v167: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 MiB/s wr, 11 op/s 2024-05-11T13:39:21.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:21 smithi063 ceph-mon[29809]: pgmap v167: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 MiB/s wr, 11 op/s 2024-05-11T13:39:22.546 INFO:teuthology.orchestra.run.smithi063.stderr:3200000 of 3200000 tuples (100%) done (elapsed 224.15 s, remaining 0.00 s) 2024-05-11T13:39:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:vacuuming... 2024-05-11T13:39:23.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:23 smithi165 ceph-mon[33235]: pgmap v168: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 MiB/s wr, 11 op/s 2024-05-11T13:39:23.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:23 smithi063 ceph-mon[29809]: pgmap v168: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 MiB/s wr, 11 op/s 2024-05-11T13:39:23.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:23 smithi071 ceph-mon[33293]: pgmap v168: 129 pgs: 129 active+clean; 860 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 MiB/s wr, 11 op/s 2024-05-11T13:39:25.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:25 smithi165 ceph-mon[33235]: pgmap v169: 129 pgs: 129 active+clean; 887 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 20 op/s 2024-05-11T13:39:25.645 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:25 smithi063 ceph-mon[29809]: pgmap v169: 129 pgs: 129 active+clean; 887 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 20 op/s 2024-05-11T13:39:25.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:25 smithi071 ceph-mon[33293]: pgmap v169: 129 pgs: 129 active+clean; 887 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 20 op/s 2024-05-11T13:39:27.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:27 smithi165 ceph-mon[33235]: pgmap v170: 129 pgs: 129 active+clean; 887 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 16 op/s 2024-05-11T13:39:27.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:27 smithi063 ceph-mon[29809]: pgmap v170: 129 pgs: 129 active+clean; 887 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 16 op/s 2024-05-11T13:39:27.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:27 smithi071 ceph-mon[33293]: pgmap v170: 129 pgs: 129 active+clean; 887 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 16 op/s 2024-05-11T13:39:29.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:29 smithi165 ceph-mon[33235]: pgmap v171: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 29 op/s 2024-05-11T13:39:29.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:29 smithi063 ceph-mon[29809]: pgmap v171: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 29 op/s 2024-05-11T13:39:29.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:29 smithi071 ceph-mon[33293]: pgmap v171: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 29 op/s 2024-05-11T13:39:31.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:31 smithi071 ceph-mon[33293]: pgmap v172: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 22 op/s 2024-05-11T13:39:31.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:31 smithi165 ceph-mon[33235]: pgmap v172: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 22 op/s 2024-05-11T13:39:31.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:31 smithi063 ceph-mon[29809]: pgmap v172: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 22 op/s 2024-05-11T13:39:32.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:32 smithi165 ceph-mon[33235]: pgmap v173: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 22 op/s 2024-05-11T13:39:32.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:32 smithi063 ceph-mon[29809]: pgmap v173: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 22 op/s 2024-05-11T13:39:32.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:32 smithi071 ceph-mon[33293]: pgmap v173: 129 pgs: 129 active+clean; 919 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 22 op/s 2024-05-11T13:39:34.006 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-11T13:39:34.006 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.85 2024-05-11T13:39:34.006 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:39:34.534 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:39:35.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:35 smithi063 ceph-mon[29809]: pgmap v174: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 31 op/s 2024-05-11T13:39:35.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:35 smithi071 ceph-mon[33293]: pgmap v174: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 31 op/s 2024-05-11T13:39:35.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:35 smithi165 ceph-mon[33235]: pgmap v174: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 31 op/s 2024-05-11T13:39:37.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:37 smithi071 ceph-mon[33293]: pgmap v175: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 21 op/s 2024-05-11T13:39:37.549 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:37 smithi063 ceph-mon[29809]: pgmap v175: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 21 op/s 2024-05-11T13:39:37.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:37 smithi165 ceph-mon[33235]: pgmap v175: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 21 op/s 2024-05-11T13:39:37.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.613+0000 7f00ddc00700 1 -- 172.21.15.63:0/1120028097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 msgr2=0x7f00d8103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.613+0000 7f00ddc00700 1 --2- 172.21.15.63:0/1120028097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d8103a60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009990 tx=0x7f00c4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.613+0000 7f00ddc00700 1 -- 172.21.15.63:0/1120028097 shutdown_connections 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.613+0000 7f00ddc00700 1 --2- 172.21.15.63:0/1120028097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d8103fa0 0x7f00d8110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.613+0000 7f00ddc00700 1 --2- 172.21.15.63:0/1120028097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00d8108510 0x7f00d81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.613+0000 7f00ddc00700 1 --2- 172.21.15.63:0/1120028097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d8103a60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.614+0000 7f00ddc00700 1 -- 172.21.15.63:0/1120028097 >> 172.21.15.63:0/1120028097 conn(0x7f00d807a6b0 msgr2=0x7f00d807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.614+0000 7f00ddc00700 1 -- 172.21.15.63:0/1120028097 shutdown_connections 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.614+0000 7f00ddc00700 1 -- 172.21.15.63:0/1120028097 wait complete. 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.615+0000 7f00ddc00700 1 Processor -- start 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.615+0000 7f00ddc00700 1 -- start start 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.615+0000 7f00ddc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:37.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00ddc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00d8103fa0 0x7f00d819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00ddc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d8108510 0x7f00d8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00ddc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8113640 con 0x7f00d8103600 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00ddc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81134c0 con 0x7f00d8108510 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00ddc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81137c0 con 0x7f00d8103fa0 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32948/0 (socket says 172.21.15.63:32948) 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 -- 172.21.15.63:0/3318123612 learned_addr learned my addr 172.21.15.63:0/3318123612 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d7400700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d8108510 0x7f00d8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 -- 172.21.15.63:0/3318123612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00d8103fa0 msgr2=0x7f00d819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00d8103fa0 0x7f00d819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 -- 172.21.15.63:0/3318123612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d8108510 msgr2=0x7f00d8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d8108510 0x7f00d8199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.616+0000 7f00d6a00700 1 -- 172.21.15.63:0/3318123612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d819a060 con 0x7f00d8103600 2024-05-11T13:39:37.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.617+0000 7f00d6a00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d819f740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d980 tx=0x7f00c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.617+0000 7f00cf400700 1 -- 172.21.15.63:0/3318123612 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8017030 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.617+0000 7f00cf400700 1 -- 172.21.15.63:0/3318123612 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c80044b0 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.617+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d819a350 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.617+0000 7f00cf400700 1 -- 172.21.15.63:0/3318123612 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8003970 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.617+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d81a6610 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.619+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d806a240 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.621+0000 7f00cf400700 1 -- 172.21.15.63:0/3318123612 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f00c801a070 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.623+0000 7f00cf400700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f00bc0b29f0 0x7f00bc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.624+0000 7f00cf400700 1 -- 172.21.15.63:0/3318123612 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f00c80e23a0 con 0x7f00d8103600 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.624+0000 7f00d6000700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f00bc0b29f0 0x7f00bc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.625+0000 7f00d6000700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f00bc0b29f0 0x7f00bc0b4eb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00c4003760 tx=0x7f00c4005fb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:37.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:37.629+0000 7f00cf400700 1 -- 172.21.15.63:0/3318123612 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00c80a9990 con 0x7f00d8103600 2024-05-11T13:39:38.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.055+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00d802e570 con 0x7f00d8103600 2024-05-11T13:39:38.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.057+0000 7f00cf400700 1 -- 172.21.15.63:0/3318123612 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11942 (secure 0 0 0) 0x7f00c80a9290 con 0x7f00d8103600 2024-05-11T13:39:38.051 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:39:38.051 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:38:14.230701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:39:38.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.061+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f00bc0b29f0 msgr2=0x7f00bc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:38.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.061+0000 7f00ddc00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f00bc0b29f0 0x7f00bc0b4eb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00c4003760 tx=0x7f00c4005fb0 comp rx=0 tx=0).stop 2024-05-11T13:39:38.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.061+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 msgr2=0x7f00d819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:38.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.061+0000 7f00ddc00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d819f740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d980 tx=0x7f00c800dd40 comp rx=0 tx=0).stop 2024-05-11T13:39:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.062+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 shutdown_connections 2024-05-11T13:39:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.062+0000 7f00ddc00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f00bc0b29f0 0x7f00bc0b4eb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.062+0000 7f00ddc00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d8108510 0x7f00d8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.062+0000 7f00ddc00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8103600 0x7f00d819f740 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.062+0000 7f00ddc00700 1 --2- 172.21.15.63:0/3318123612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00d8103fa0 0x7f00d819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.062+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 >> 172.21.15.63:0/3318123612 conn(0x7f00d807a6b0 msgr2=0x7f00d8101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:38.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.064+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 shutdown_connections 2024-05-11T13:39:38.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:38.064+0000 7f00ddc00700 1 -- 172.21.15.63:0/3318123612 wait complete. 2024-05-11T13:39:38.062 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 48 2024-05-11T13:39:38.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:38 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3318123612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:39:38.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:38 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3318123612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:39:38.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:38 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3318123612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:39:38.979 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 80.85 --await-for 42.0' 2024-05-11T13:39:38.979 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 80.85 --await-for 42.0 2024-05-11T13:39:39.443 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:39:39.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:39 smithi165 ceph-mon[33235]: pgmap v176: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 30 op/s 2024-05-11T13:39:39.595 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:39 smithi063 ceph-mon[29809]: pgmap v176: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 30 op/s 2024-05-11T13:39:39.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:39 smithi071 ceph-mon[33293]: pgmap v176: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 30 op/s 2024-05-11T13:39:41.485 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:41 smithi063 ceph-mon[29809]: pgmap v177: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:39:41.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:41 smithi165 ceph-mon[33235]: pgmap v177: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:39:41.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:41 smithi071 ceph-mon[33293]: pgmap v177: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:39:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 -- 172.21.15.63:0/1570839102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c108700 msgr2=0x7f5a7c1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 --2- 172.21.15.63:0/1570839102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c108700 0x7f5a7c1102d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b0d0 tx=0x7f5a7400b3e0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 -- 172.21.15.63:0/1570839102 shutdown_connections 2024-05-11T13:39:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 --2- 172.21.15.63:0/1570839102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c108700 0x7f5a7c1102d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 --2- 172.21.15.63:0/1570839102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a7c103620 0x7f5a7c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 --2- 172.21.15.63:0/1570839102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c103fd0 0x7f5a7c108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 -- 172.21.15.63:0/1570839102 >> 172.21.15.63:0/1570839102 conn(0x7f5a7c0feef0 msgr2=0x7f5a7c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 -- 172.21.15.63:0/1570839102 shutdown_connections 2024-05-11T13:39:41.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.895+0000 7f5a82600700 1 -- 172.21.15.63:0/1570839102 wait complete. 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.896+0000 7f5a82600700 1 Processor -- start 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.897+0000 7f5a82600700 1 -- start start 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.897+0000 7f5a82600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c103620 0x7f5a7c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.897+0000 7f5a82600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c103fd0 0x7f5a7c1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.897+0000 7f5a82600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a7c108700 0x7f5a7c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.897+0000 7f5a82600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1134c0 con 0x7f5a7c103620 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.897+0000 7f5a82600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c113340 con 0x7f5a7c103fd0 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.897+0000 7f5a82600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c113640 con 0x7f5a7c108700 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c103620 0x7f5a7c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c103620 0x7f5a7c1a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58670/0 (socket says 172.21.15.63:58670) 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 -- 172.21.15.63:0/964621379 learned_addr learned my addr 172.21.15.63:0/964621379 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a7be00700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c103fd0 0x7f5a7c1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81c00700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a7c108700 0x7f5a7c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 -- 172.21.15.63:0/964621379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a7c108700 msgr2=0x7f5a7c19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:41.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a7c108700 0x7f5a7c19b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 -- 172.21.15.63:0/964621379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c103fd0 msgr2=0x7f5a7c1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c103fd0 0x7f5a7c1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.898+0000 7f5a81200700 1 -- 172.21.15.63:0/964621379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c19c080 con 0x7f5a7c103620 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.899+0000 7f5a81200700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c103620 0x7f5a7c1a1760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000d980 tx=0x7f5a7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.899+0000 7f5a79600700 1 -- 172.21.15.63:0/964621379 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70017070 con 0x7f5a7c103620 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.899+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7c19c370 con 0x7f5a7c103620 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.899+0000 7f5a79600700 1 -- 172.21.15.63:0/964621379 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a70005020 con 0x7f5a7c103620 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.899+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7c1a8630 con 0x7f5a7c103620 2024-05-11T13:39:41.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.900+0000 7f5a79600700 1 -- 172.21.15.63:0/964621379 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70009ab0 con 0x7f5a7c103620 2024-05-11T13:39:41.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 Processor -- start 2024-05-11T13:39:41.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 -- start start 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d42910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d3cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562282c8ebb0 con 0x562282d42530 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562282c2c8f0 con 0x562282d3c890 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.905+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562282c8ab70 con 0x562282d3d250 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.906+0000 7f5a79600700 1 -- 172.21.15.63:0/964621379 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5a700051d0 con 0x7f5a7c103620 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.906+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d42910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.906+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d42910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58686/0 (socket says 172.21.15.63:58686) 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.906+0000 7f5a81c00700 1 -- 172.21.15.63:0/3280162729 learned_addr learned my addr 172.21.15.63:0/3280162729 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.906+0000 7f5a7be00700 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.907+0000 7f5a81c00700 1 -- 172.21.15.63:0/3280162729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 msgr2=0x562282d492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.907+0000 7f5a81c00700 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.907+0000 7f5a81c00700 1 -- 172.21.15.63:0/3280162729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 msgr2=0x562282d3cd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T13:39:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.907+0000 7f5a81c00700 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d3cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.907+0000 7f5a81c00700 1 -- 172.21.15.63:0/3280162729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562282bdba60 con 0x562282d42530 2024-05-11T13:39:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.907+0000 7f5a81c00700 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d42910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009d40 tx=0x7f5a74009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc720dab56002f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.908+0000 7f5a6e000700 1 -- 172.21.15.63:0/3280162729 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74020030 con 0x562282d42530 2024-05-11T13:39:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.908+0000 7f5a6e000700 1 -- 172.21.15.63:0/3280162729 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a74004a50 con 0x562282d42530 2024-05-11T13:39:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.908+0000 7f5a6e000700 1 -- 172.21.15.63:0/3280162729 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74007bf0 con 0x562282d42530 2024-05-11T13:39:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.908+0000 7f5a79600700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5a640b2b20 0x7f5a640b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a79600700 1 -- 172.21.15.63:0/964621379 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f5a700e2910 con 0x7f5a7c103620 2024-05-11T13:39:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3280162729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 msgr2=0x562282d42910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d42910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009d40 tx=0x7f5a74009d70 comp rx=0 tx=0).stop 2024-05-11T13:39:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3280162729 shutdown_connections 2024-05-11T13:39:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d42910 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3280162729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d3cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a81200700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5a640b2b20 0x7f5a640b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3280162729 >> 172.21.15.63:0/3280162729 conn(0x562282c6acf0 msgr2=0x562282d37510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:41.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.909+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3280162729 shutdown_connections 2024-05-11T13:39:41.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.910+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3280162729 wait complete. 2024-05-11T13:39:41.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.910+0000 7f5a8e629b80 1 -- start start 2024-05-11T13:39:41.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.910+0000 7f5a81200700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5a640b2b20 0x7f5a640b4fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a700a5880 tx=0x7f5a700e2b60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:41.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.910+0000 7f5a8e629b80 1 Processor -- start 2024-05-11T13:39:41.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.911+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d48fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.911+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.911+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.911+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562282d49830 con 0x562282d42530 2024-05-11T13:39:41.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.911+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562282bf4cb0 con 0x562282d3c890 2024-05-11T13:39:41.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.911+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562282d499b0 con 0x562282d3d250 2024-05-11T13:39:41.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.911+0000 7f5a8e629b80 10 client.0 _ll_register_callbacks cb 0x562282c22510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:39:41.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.912+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d48fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.912+0000 7f5a7be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.912+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d48fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:54984/0 (socket says 172.21.15.63:54984) 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.912+0000 7f5a81c00700 1 -- 172.21.15.63:0/401523214 learned_addr learned my addr 172.21.15.63:0/401523214 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.913+0000 7f5a81c00700 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.913+0000 7f5a81c00700 1 -- 172.21.15.63:0/401523214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 msgr2=0x562282d46360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.913+0000 7f5a81c00700 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d46360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.913+0000 7f5a81c00700 1 -- 172.21.15.63:0/401523214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 msgr2=0x562282d468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.913+0000 7f5a81c00700 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.913+0000 7f5a81c00700 1 -- 172.21.15.63:0/401523214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562282c545b0 con 0x562282d3c890 2024-05-11T13:39:41.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.914+0000 7f5a81c00700 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d48fc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a74004e80 tx=0x7f5a74007790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddf4f5ac02897be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:41.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.914+0000 7f5a62a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:39:41.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.914+0000 7f5a62a00700 1 -- 172.21.15.63:0/401523214 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74020040 con 0x562282d3c890 2024-05-11T13:39:41.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.914+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a74004a50 con 0x562282d3c890 2024-05-11T13:39:41.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.914+0000 7f5a62a00700 1 -- 172.21.15.63:0/401523214 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f5a74009300 con 0x562282d3c890 2024-05-11T13:39:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.915+0000 7f5a62a00700 1 -- 172.21.15.63:0/401523214 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5a74039050 con 0x562282d3c890 2024-05-11T13:39:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.915+0000 7f5a8e629b80 10 client.35333 fetch_fsmap learned FSMap version 48 2024-05-11T13:39:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.915+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562282d5c8d0 con 0x562282d3c890 2024-05-11T13:39:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.915+0000 7f5a62a00700 1 -- 172.21.15.63:0/401523214 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a74004a50 con 0x562282d3c890 2024-05-11T13:39:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.916+0000 7f5a62a00700 1 -- 172.21.15.63:0/401523214 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f5a74011070 con 0x562282d3c890 2024-05-11T13:39:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.916+0000 7f5a8e629b80 10 client.35333 fetch_fsmap finished waiting for FSMap version 48 2024-05-11T13:39:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.916+0000 7f5a8e629b80 10 client.35333 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:39:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.916+0000 7f5a8e629b80 20 client.35333 populate_metadata read hostname 'smithi063' 2024-05-11T13:39:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.916+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282d5eda0 0x562282d61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.916+0000 7f5a8e629b80 4 client.35333 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:39:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.916+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562282d091f0 con 0x562282d5eda0 2024-05-11T13:39:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.917+0000 7f5a7be00700 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282d5eda0 0x562282d61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.918+0000 7f5a7be00700 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282d5eda0 0x562282d61190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2f4a384eaa319f server_cookie=d3926b4d153ed4c4 in_seq=0 out_seq=0 2024-05-11T13:39:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.918+0000 7f5a62a00700 10 client.35333 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a62a00700 1 -- 172.21.15.63:0/401523214 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562282d091f0 con 0x562282d5eda0 2024-05-11T13:39:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a62a00700 10 client.35333 handle_command_reply: tid=0 2024-05-11T13:39:41.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 client.35333 shutdown 2024-05-11T13:39:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 msgr2=0x562282d48fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d48fc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a74004e80 tx=0x7f5a74007790 comp rx=0 tx=0).stop 2024-05-11T13:39:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 shutdown_connections 2024-05-11T13:39:41.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282d5eda0 0x562282d61190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d48fc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/401523214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 >> 172.21.15.63:0/401523214 conn(0x562282c6acf0 msgr2=0x562282d3b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.920+0000 7f5a62a00700 0 client.35333 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.921+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 shutdown_connections 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.921+0000 7f5a8e629b80 1 -- 172.21.15.63:0/401523214 wait complete. 2024-05-11T13:39:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.921+0000 7f5a8e629b80 20 client.35333 trim_cache size 0 max 16384 2024-05-11T13:39:41.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.947+0000 7f5a8e629b80 1 Processor -- start 2024-05-11T13:39:41.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.947+0000 7f5a8e629b80 1 -- start start 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3c890 0x562282d37680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3d250 0x562282d3c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1ad040 con 0x562282d42530 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1aec40 con 0x562282d3d250 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1ad710 con 0x562282d3c890 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58710/0 (socket says 172.21.15.63:58710) 2024-05-11T13:39:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a81c00700 1 -- 172.21.15.63:0/3710100035 learned_addr learned my addr 172.21.15.63:0/3710100035 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:41.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a7be00700 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3d250 0x562282d3c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.948+0000 7f5a81c00700 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3c890 0x562282d37680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a81c00700 1 -- 172.21.15.63:0/3710100035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3c890 msgr2=0x562282d37680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:41.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a81c00700 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3c890 0x562282d37680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a81c00700 1 -- 172.21.15.63:0/3710100035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3d250 msgr2=0x562282d3c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:41.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a81c00700 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3d250 0x562282d3c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a81c00700 1 -- 172.21.15.63:0/3710100035 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562282d49330 con 0x562282d42530 2024-05-11T13:39:41.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a81c00700 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d406b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b820 tx=0x7f5a740099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5824e00a5aa0bc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:41.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a6d600700 1 -- 172.21.15.63:0/3710100035 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74020070 con 0x562282d42530 2024-05-11T13:39:41.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.949+0000 7f5a6d600700 1 -- 172.21.15.63:0/3710100035 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a74004a50 con 0x562282d42530 2024-05-11T13:39:41.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.950+0000 7f5a6d600700 1 -- 172.21.15.63:0/3710100035 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740092e0 con 0x562282d42530 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.950+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3710100035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 msgr2=0x562282d406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.950+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d406b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b820 tx=0x7f5a740099e0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.950+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3710100035 shutdown_connections 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.951+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d406b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.951+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3c890 0x562282d37680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.951+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/3710100035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3d250 0x562282d3c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.951+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3710100035 >> 172.21.15.63:0/3710100035 conn(0x562282c6acf0 msgr2=0x562282d3ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:41.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.951+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3710100035 shutdown_connections 2024-05-11T13:39:41.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.951+0000 7f5a8e629b80 1 -- 172.21.15.63:0/3710100035 wait complete. 2024-05-11T13:39:41.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.951+0000 7f5a8e629b80 1 -- start start 2024-05-11T13:39:41.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 1 Processor -- start 2024-05-11T13:39:41.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d62080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d62b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1ae570 con 0x562282d42530 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1adde0 con 0x562282d3c890 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1aafe0 con 0x562282d3d250 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.952+0000 7f5a8e629b80 10 client.0 _ll_register_callbacks cb 0x562282c22510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d62b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d62b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58726/0 (socket says 172.21.15.63:58726) 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 -- 172.21.15.63:0/2811706193 learned_addr learned my addr 172.21.15.63:0/2811706193 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a7be00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 -- 172.21.15.63:0/2811706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 msgr2=0x562282d625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:41.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 -- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 msgr2=0x562282d62080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:39:41.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d62080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:41.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a81c00700 1 -- 172.21.15.63:0/2811706193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562282dc85a0 con 0x562282d42530 2024-05-11T13:39:41.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.953+0000 7f5a7be00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:39:41.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.954+0000 7f5a81c00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d62b20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5a74036040 tx=0x7f5a7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f0cc5df230eb51 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:39:41.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.954+0000 7f5a62a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:39:41.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.954+0000 7f5a62a00700 1 -- 172.21.15.63:0/2811706193 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74020040 con 0x562282d42530 2024-05-11T13:39:41.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.954+0000 7f5a62a00700 1 -- 172.21.15.63:0/2811706193 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f5a7401c8a0 con 0x562282d42530 2024-05-11T13:39:41.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.954+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a74004a50 con 0x562282d42530 2024-05-11T13:39:41.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.955+0000 7f5a62a00700 1 -- 172.21.15.63:0/2811706193 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5a74049050 con 0x562282d42530 2024-05-11T13:39:41.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.955+0000 7f5a8e629b80 10 client.15732 fetch_fsmap learned FSMap version 48 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.955+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562282dc9880 con 0x562282d42530 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.955+0000 7f5a62a00700 1 -- 172.21.15.63:0/2811706193 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a74004a50 con 0x562282d42530 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.955+0000 7f5a62a00700 1 -- 172.21.15.63:0/2811706193 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f5a74011040 con 0x562282d42530 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.956+0000 7f5a8e629b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 48 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.956+0000 7f5a8e629b80 10 client.15732 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.956+0000 7f5a8e629b80 20 client.15732 populate_metadata read hostname 'smithi063' 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.956+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282dc9ee0 0x562282dcc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.956+0000 7f5a8e629b80 4 client.15732 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.85, "await_for": 42.0}] 2024-05-11T13:39:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.956+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.85, "await_for": 42.0}) -- 0x562282c9c590 con 0x562282dc9ee0 2024-05-11T13:39:41.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.956+0000 7f5a7be00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282dc9ee0 0x562282dcc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:39:41.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.957+0000 7f5a7be00700 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282dc9ee0 0x562282dcc300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc6b51ebe225bee server_cookie=eb37a0917543255 in_seq=0 out_seq=0 2024-05-11T13:39:41.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:41.957+0000 7f5a62a00700 10 client.15732 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:42.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a62a00700 1 -- 172.21.15.63:0/2811706193 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562282c9c590 con 0x562282dc9ee0 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a62a00700 10 client.15732 handle_command_reply: tid=0 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 client.15732 shutdown 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 msgr2=0x562282d62b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d62b20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5a74036040 tx=0x7f5a7400b040 comp rx=0 tx=0).stop 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 shutdown_connections 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562282d42530 0x562282d62b20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562282dc9ee0 0x562282dcc300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562282d3c890 0x562282d62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 --2- 172.21.15.63:0/2811706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562282d3d250 0x562282d625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 >> 172.21.15.63:0/2811706193 conn(0x562282c6acf0 msgr2=0x562282d40830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:42.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.331+0000 7f5a62a00700 0 client.15732 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:39:42.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.332+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 shutdown_connections 2024-05-11T13:39:42.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.332+0000 7f5a8e629b80 1 -- 172.21.15.63:0/2811706193 wait complete. 2024-05-11T13:39:42.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.332+0000 7f5a8e629b80 20 client.15732 trim_cache size 0 max 16384 2024-05-11T13:39:42.327 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:39:42.327 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 48, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 18, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "268fae8f": { 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 18, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 80.8, 2024-05-11T13:39:42.328 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:39:42.329 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:39:42.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5a640b2b20 msgr2=0x7f5a640b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:42.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5a640b2b20 0x7f5a640b4fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a700a5880 tx=0x7f5a700e2b60 comp rx=0 tx=0).stop 2024-05-11T13:39:42.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c103620 msgr2=0x7f5a7c1a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c103620 0x7f5a7c1a1760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000d980 tx=0x7f5a7000dc90 comp rx=0 tx=0).stop 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 shutdown_connections 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5a640b2b20 0x7f5a640b4fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a7c108700 0x7f5a7c19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a7c103620 0x7f5a7c1a1760 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 --2- 172.21.15.63:0/964621379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c103fd0 0x7f5a7c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 >> 172.21.15.63:0/964621379 conn(0x7f5a7c0feef0 msgr2=0x7f5a7c10ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.334+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 shutdown_connections 2024-05-11T13:39:42.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:39:42.335+0000 7f5a82600700 1 -- 172.21.15.63:0/964621379 wait complete. 2024-05-11T13:39:43.301 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '268fae8f', quiesce took 9.3 seconds. Will release after: 20.85 2024-05-11T13:39:43.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:43 smithi165 ceph-mon[33235]: pgmap v178: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:39:43.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:43 smithi063 ceph-mon[29809]: pgmap v178: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:39:43.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:43 smithi071 ceph-mon[33293]: pgmap v178: 129 pgs: 129 active+clean; 1020 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-11T13:39:45.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:45 smithi165 ceph-mon[33235]: pgmap v179: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 29 op/s 2024-05-11T13:39:45.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:45 smithi063 ceph-mon[29809]: pgmap v179: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 29 op/s 2024-05-11T13:39:45.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:45 smithi071 ceph-mon[33293]: pgmap v179: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 29 op/s 2024-05-11T13:39:46.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:46 smithi165 ceph-mon[33235]: pgmap v180: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-05-11T13:39:46.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:46 smithi063 ceph-mon[29809]: pgmap v180: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-05-11T13:39:46.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:46 smithi071 ceph-mon[33293]: pgmap v180: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-05-11T13:39:49.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:49 smithi071 ceph-mon[33293]: pgmap v181: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-11T13:39:49.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:49 smithi165 ceph-mon[33235]: pgmap v181: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-11T13:39:49.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:49 smithi063 ceph-mon[29809]: pgmap v181: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-11T13:39:51.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:51 smithi165 ceph-mon[33235]: pgmap v182: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-11T13:39:51.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:51 smithi063 ceph-mon[29809]: pgmap v182: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-11T13:39:51.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:51 smithi071 ceph-mon[33293]: pgmap v182: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-11T13:39:53.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:53 smithi165 ceph-mon[33235]: pgmap v183: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-11T13:39:53.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:53 smithi063 ceph-mon[29809]: pgmap v183: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-11T13:39:53.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:53 smithi071 ceph-mon[33293]: pgmap v183: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-11T13:39:55.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:55 smithi165 ceph-mon[33235]: pgmap v184: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 20 op/s 2024-05-11T13:39:55.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:55 smithi063 ceph-mon[29809]: pgmap v184: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 20 op/s 2024-05-11T13:39:55.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:55 smithi071 ceph-mon[33293]: pgmap v184: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 20 op/s 2024-05-11T13:39:57.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:57 smithi165 ceph-mon[33235]: pgmap v185: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 352 KiB/s wr, 9 op/s 2024-05-11T13:39:57.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:57 smithi063 ceph-mon[29809]: pgmap v185: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 352 KiB/s wr, 9 op/s 2024-05-11T13:39:57.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:57 smithi071 ceph-mon[33293]: pgmap v185: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 352 KiB/s wr, 9 op/s 2024-05-11T13:39:59.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:39:59 smithi165 ceph-mon[33235]: pgmap v186: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 353 KiB/s wr, 14 op/s 2024-05-11T13:39:59.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:39:59 smithi063 ceph-mon[29809]: pgmap v186: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 353 KiB/s wr, 14 op/s 2024-05-11T13:39:59.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:39:59 smithi071 ceph-mon[33293]: pgmap v186: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 353 KiB/s wr, 14 op/s 2024-05-11T13:40:00.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:00 smithi165 ceph-mon[33235]: overall HEALTH_OK 2024-05-11T13:40:00.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:00 smithi063 ceph-mon[29809]: overall HEALTH_OK 2024-05-11T13:40:00.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:00 smithi071 ceph-mon[33293]: overall HEALTH_OK 2024-05-11T13:40:01.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:01 smithi165 ceph-mon[33235]: pgmap v187: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2024-05-11T13:40:01.599 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:01 smithi071 ceph-mon[33293]: pgmap v187: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2024-05-11T13:40:01.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:01 smithi063 ceph-mon[29809]: pgmap v187: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2024-05-11T13:40:02.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:02 smithi165 ceph-mon[33235]: pgmap v188: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2024-05-11T13:40:02.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:02 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:40:02.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:02 smithi063 ceph-mon[29809]: pgmap v188: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2024-05-11T13:40:02.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:02 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:40:02.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:02 smithi071 ceph-mon[33293]: pgmap v188: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2024-05-11T13:40:02.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:02 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:40:04.152 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:40:04.602 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:40:05.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:05 smithi063 ceph-mon[29809]: pgmap v189: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-11T13:40:05.498 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:05 smithi165 ceph-mon[33235]: pgmap v189: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-11T13:40:05.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:05 smithi071 ceph-mon[33293]: pgmap v189: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.960+0000 7f1b04a00700 1 -- 172.21.15.63:0/2128760630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b0010b660 msgr2=0x7f1b0010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.960+0000 7f1b04a00700 1 --2- 172.21.15.63:0/2128760630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b0010b660 0x7f1b0010ba40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009990 tx=0x7f1af0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.961+0000 7f1b04a00700 1 -- 172.21.15.63:0/2128760630 shutdown_connections 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.961+0000 7f1b04a00700 1 --2- 172.21.15.63:0/2128760630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b00106020 0x7f1b00112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.961+0000 7f1b04a00700 1 --2- 172.21.15.63:0/2128760630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b0010b660 0x7f1b0010ba40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.961+0000 7f1b04a00700 1 --2- 172.21.15.63:0/2128760630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b00105660 0x7f1b00105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.961+0000 7f1b04a00700 1 -- 172.21.15.63:0/2128760630 >> 172.21.15.63:0/2128760630 conn(0x7f1b00101130 msgr2=0x7f1b00103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.961+0000 7f1b04a00700 1 -- 172.21.15.63:0/2128760630 shutdown_connections 2024-05-11T13:40:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.961+0000 7f1b04a00700 1 -- 172.21.15.63:0/2128760630 wait complete. 2024-05-11T13:40:06.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.962+0000 7f1b04a00700 1 Processor -- start 2024-05-11T13:40:06.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1b04a00700 1 -- start start 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1b04a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b00105660 0x7f1b001a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1b04a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b00106020 0x7f1b001a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1b04a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b0010b660 0x7f1b0019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1b04a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00115710 con 0x7f1b00105660 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1b04a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00115590 con 0x7f1b0010b660 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1b04a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00115890 con 0x7f1b00106020 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.963+0000 7f1afe800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b00105660 0x7f1b001a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b00105660 0x7f1b001a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47398/0 (socket says 172.21.15.63:47398) 2024-05-11T13:40:06.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 -- 172.21.15.63:0/897679661 learned_addr learned my addr 172.21.15.63:0/897679661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:06.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afde00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b00106020 0x7f1b001a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 -- 172.21.15.63:0/897679661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b00106020 msgr2=0x7f1b001a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b00106020 0x7f1b001a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 -- 172.21.15.63:0/897679661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b0010b660 msgr2=0x7f1b0019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1aff200700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b0010b660 0x7f1b0019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b0010b660 0x7f1b0019ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 -- 172.21.15.63:0/897679661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0019c1b0 con 0x7f1b00105660 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.964+0000 7f1afe800700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b00105660 0x7f1b001a1890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1af0006010 tx=0x7f1af000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.965+0000 7f1af7400700 1 -- 172.21.15.63:0/897679661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0027070 con 0x7f1b00105660 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.965+0000 7f1af7400700 1 -- 172.21.15.63:0/897679661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1af0022470 con 0x7f1b00105660 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.965+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0019c440 con 0x7f1b00105660 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.965+0000 7f1af7400700 1 -- 172.21.15.63:0/897679661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0017650 con 0x7f1b00105660 2024-05-11T13:40:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.965+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001a8760 con 0x7f1b00105660 2024-05-11T13:40:06.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.966+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0006a240 con 0x7f1b00105660 2024-05-11T13:40:06.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.969+0000 7f1af7400700 1 -- 172.21.15.63:0/897679661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1af0009d10 con 0x7f1b00105660 2024-05-11T13:40:06.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.970+0000 7f1af7400700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ae40b2a70 0x7f1ae40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:06.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.970+0000 7f1af7400700 1 -- 172.21.15.63:0/897679661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9437+0+0 (secure 0 0 0) 0x7f1af00e3930 con 0x7f1b00105660 2024-05-11T13:40:06.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.970+0000 7f1afde00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ae40b2a70 0x7f1ae40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:06.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.971+0000 7f1afde00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ae40b2a70 0x7f1ae40b4f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005950 tx=0x7f1aec0058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:06.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:06.973+0000 7f1af7400700 1 -- 172.21.15.63:0/897679661 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1af0013070 con 0x7f1b00105660 2024-05-11T13:40:07.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.288+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b0019cf80 con 0x7f1b00105660 2024-05-11T13:40:07.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.289+0000 7f1af7400700 1 -- 172.21.15.63:0/897679661 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11936 (secure 0 0 0) 0x7f1af00aafb0 con 0x7f1b00105660 2024-05-11T13:40:07.284 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:40:07.284 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:40:01.464213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:40:07.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ae40b2a70 msgr2=0x7f1ae40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:07.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ae40b2a70 0x7f1ae40b4f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005950 tx=0x7f1aec0058e0 comp rx=0 tx=0).stop 2024-05-11T13:40:07.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b00105660 msgr2=0x7f1b001a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:07.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b00105660 0x7f1b001a1890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1af0006010 tx=0x7f1af000bfc0 comp rx=0 tx=0).stop 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 shutdown_connections 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ae40b2a70 0x7f1ae40b4f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b0010b660 0x7f1b0019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b00105660 0x7f1b001a1890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 --2- 172.21.15.63:0/897679661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b00106020 0x7f1b001a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 >> 172.21.15.63:0/897679661 conn(0x7f1b00101130 msgr2=0x7f1b00101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 shutdown_connections 2024-05-11T13:40:07.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:07.293+0000 7f1b04a00700 1 -- 172.21.15.63:0/897679661 wait complete. 2024-05-11T13:40:07.289 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 49 2024-05-11T13:40:07.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:07 smithi165 ceph-mon[33235]: pgmap v190: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-05-11T13:40:07.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:07 smithi063 ceph-mon[29809]: pgmap v190: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-05-11T13:40:07.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:07 smithi071 ceph-mon[33293]: pgmap v190: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-05-11T13:40:08.302 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:08 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/897679661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:08.351 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 268fae8f --release --await' 2024-05-11T13:40:08.351 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 268fae8f --release --await 2024-05-11T13:40:08.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:08 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/897679661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:08.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:08 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/897679661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:08.786 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:40:09.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:09 smithi165 ceph-mon[33235]: pgmap v191: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-05-11T13:40:09.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:09 smithi063 ceph-mon[29809]: pgmap v191: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-05-11T13:40:09.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:09 smithi071 ceph-mon[33293]: pgmap v191: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-05-11T13:40:10.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:10 smithi165 ceph-mon[33235]: osdmap e92: 12 total, 12 up, 12 in 2024-05-11T13:40:10.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:10 smithi165 ceph-mon[33235]: pgmap v193: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-11T13:40:10.646 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:10 smithi063 ceph-mon[29809]: osdmap e92: 12 total, 12 up, 12 in 2024-05-11T13:40:10.646 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:10 smithi063 ceph-mon[29809]: pgmap v193: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-11T13:40:10.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:10 smithi071 ceph-mon[33293]: osdmap e92: 12 total, 12 up, 12 in 2024-05-11T13:40:10.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:10 smithi071 ceph-mon[33293]: pgmap v193: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 -- 172.21.15.63:0/1972666285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb410b790 msgr2=0x7f0eb410bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 --2- 172.21.15.63:0/1972666285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb410b790 0x7f0eb410bb70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009990 tx=0x7f0ea4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 -- 172.21.15.63:0/1972666285 shutdown_connections 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 --2- 172.21.15.63:0/1972666285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb4106150 0x7f0eb4112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 --2- 172.21.15.63:0/1972666285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb410b790 0x7f0eb410bb70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 --2- 172.21.15.63:0/1972666285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb4105790 0x7f0eb4105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 -- 172.21.15.63:0/1972666285 >> 172.21.15.63:0/1972666285 conn(0x7f0eb4101280 msgr2=0x7f0eb41036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 -- 172.21.15.63:0/1972666285 shutdown_connections 2024-05-11T13:40:11.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.248+0000 7f0ebae00700 1 -- 172.21.15.63:0/1972666285 wait complete. 2024-05-11T13:40:11.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 Processor -- start 2024-05-11T13:40:11.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 -- start start 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb4105790 0x7f0eb419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb4106150 0x7f0eb419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb410b790 0x7f0eb41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4115890 con 0x7f0eb4106150 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4115710 con 0x7f0eb410b790 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.250+0000 7f0ebae00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4115a10 con 0x7f0eb4105790 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb4106150 0x7f0eb419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb4106150 0x7f0eb419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35386/0 (socket says 172.21.15.63:35386) 2024-05-11T13:40:11.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 -- 172.21.15.63:0/3374699658 learned_addr learned my addr 172.21.15.63:0/3374699658 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:11.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb410b790 0x7f0eb41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb4105790 0x7f0eb419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 -- 172.21.15.63:0/3374699658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb4105790 msgr2=0x7f0eb419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb4105790 0x7f0eb419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 -- 172.21.15.63:0/3374699658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb410b790 msgr2=0x7f0eb41a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb410b790 0x7f0eb41a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.251+0000 7f0eb3400700 1 -- 172.21.15.63:0/3374699658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb41a0d70 con 0x7f0eb4106150 2024-05-11T13:40:11.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.252+0000 7f0eb3400700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb4106150 0x7f0eb419c2a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000d950 tx=0x7f0ea000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:11.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.252+0000 7f0eb0c00700 1 -- 172.21.15.63:0/3374699658 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0016070 con 0x7f0eb4106150 2024-05-11T13:40:11.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.252+0000 7f0eb0c00700 1 -- 172.21.15.63:0/3374699658 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea0010460 con 0x7f0eb4106150 2024-05-11T13:40:11.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.252+0000 7f0eb0c00700 1 -- 172.21.15.63:0/3374699658 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000f620 con 0x7f0eb4106150 2024-05-11T13:40:11.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.252+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb41a1060 con 0x7f0eb4106150 2024-05-11T13:40:11.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.252+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb41a88e0 con 0x7f0eb4106150 2024-05-11T13:40:11.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.258+0000 7f0eb0c00700 1 -- 172.21.15.63:0/3374699658 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0ea0019070 con 0x7f0eb4106150 2024-05-11T13:40:11.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.258+0000 7f0ec5712b80 1 Processor -- start 2024-05-11T13:40:11.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.258+0000 7f0ec5712b80 1 -- start start 2024-05-11T13:40:11.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564e4e218b20 con 0x564e4e2c6f60 2024-05-11T13:40:11.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564e4e1b6860 con 0x564e4e2c65a0 2024-05-11T13:40:11.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e4e214ae0 con 0x564e4e2cbfd0 2024-05-11T13:40:11.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0eb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0eb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2d2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35400/0 (socket says 172.21.15.63:35400) 2024-05-11T13:40:11.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0eb3400700 1 -- 172.21.15.63:0/1378769896 learned_addr learned my addr 172.21.15.63:0/1378769896 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:11.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.259+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb0c00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0e980b2a70 0x7f0e980b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb0c00700 1 -- 172.21.15.63:0/3374699658 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9613+0+0 (secure 0 0 0) 0x7f0ea00e2690 con 0x7f0eb4106150 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0e980b2a70 0x7f0e980b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb3e00700 1 -- 172.21.15.63:0/1378769896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 msgr2=0x564e4e2cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb3e00700 1 -- 172.21.15.63:0/1378769896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 msgr2=0x564e4e2d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.260+0000 7f0eb3e00700 1 -- 172.21.15.63:0/1378769896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e4e1659d0 con 0x564e4e2c65a0 2024-05-11T13:40:11.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.261+0000 7f0eb3400700 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:40:11.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.261+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2c6a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c00 tx=0x7f0ea4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8cd29cae7b3c06e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:11.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.261+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0e980b2a70 0x7f0e980b4f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007760 tx=0x7f0eac004b40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:11.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.262+0000 7f0ea9600700 1 -- 172.21.15.63:0/1378769896 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4027070 con 0x564e4e2c65a0 2024-05-11T13:40:11.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.262+0000 7f0ea9600700 1 -- 172.21.15.63:0/1378769896 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea40037c0 con 0x564e4e2c65a0 2024-05-11T13:40:11.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.262+0000 7f0ea9600700 1 -- 172.21.15.63:0/1378769896 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400b9f0 con 0x564e4e2c65a0 2024-05-11T13:40:11.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 -- 172.21.15.63:0/1378769896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 msgr2=0x564e4e2c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2c6a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c00 tx=0x7f0ea4005e80 comp rx=0 tx=0).stop 2024-05-11T13:40:11.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 -- 172.21.15.63:0/1378769896 shutdown_connections 2024-05-11T13:40:11.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/1378769896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2c6a20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 -- 172.21.15.63:0/1378769896 >> 172.21.15.63:0/1378769896 conn(0x564e4e1f4c60 msgr2=0x564e4e2c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:11.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 -- 172.21.15.63:0/1378769896 shutdown_connections 2024-05-11T13:40:11.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.263+0000 7f0ec5712b80 1 -- 172.21.15.63:0/1378769896 wait complete. 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 -- start start 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 Processor -- start 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564e4e2d32d0 con 0x564e4e2c6f60 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564e4e17ec20 con 0x564e4e2c65a0 2024-05-11T13:40:11.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e4e2d3450 con 0x564e4e2cbfd0 2024-05-11T13:40:11.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.264+0000 7f0ec5712b80 10 client.0 _ll_register_callbacks cb 0x564e4e1bfd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:40:11.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35414/0 (socket says 172.21.15.63:35414) 2024-05-11T13:40:11.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 -- 172.21.15.63:0/129369882 learned_addr learned my addr 172.21.15.63:0/129369882 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 -- 172.21.15.63:0/129369882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 msgr2=0x564e4e2d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 -- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 msgr2=0x564e4e2d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3400700 1 -- 172.21.15.63:0/129369882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4e1de520 con 0x564e4e2c6f60 2024-05-11T13:40:11.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.265+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0eb3400700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2cfd60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005950 tx=0x7f0ea0010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d9bbf88a2341ed server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0e9e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0e9e000700 1 -- 172.21.15.63:0/129369882 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0020030 con 0x564e4e2c6f60 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0e9e000700 1 -- 172.21.15.63:0/129369882 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9613+0+0 (secure 0 0 0) 0x7f0ea00a9730 con 0x564e4e2c6f60 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea40037c0 con 0x564e4e2c6f60 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0e9e000700 1 -- 172.21.15.63:0/129369882 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0ea00a8c40 con 0x564e4e2c6f60 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0ec5712b80 10 client.15750 fetch_fsmap learned FSMap version 49 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.266+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564e4e2ddb10 con 0x564e4e2c6f60 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0e9e000700 1 -- 172.21.15.63:0/129369882 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea40037c0 con 0x564e4e2c6f60 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0e9e000700 1 -- 172.21.15.63:0/129369882 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f0ea00b1ea0 con 0x564e4e2c6f60 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0ec5712b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 49 2024-05-11T13:40:11.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0ec5712b80 10 client.15750 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:40:11.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0ec5712b80 20 client.15750 populate_metadata read hostname 'smithi063' 2024-05-11T13:40:11.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e2e0150 0x564e4e2e2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0ec5712b80 4 client.15750 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:40:11.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.267+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e4e2932b0 con 0x564e4e2e0150 2024-05-11T13:40:11.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.268+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e2e0150 0x564e4e2e2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.269+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e2e0150 0x564e4e2e2540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9b92abc1098a70 server_cookie=a407080203240c05 in_seq=0 out_seq=0 2024-05-11T13:40:11.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.269+0000 7f0e9e000700 10 client.15750 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:11.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.270+0000 7f0e9e000700 1 -- 172.21.15.63:0/129369882 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e4e2932b0 con 0x564e4e2e0150 2024-05-11T13:40:11.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.270+0000 7f0e9e000700 10 client.15750 handle_command_reply: tid=0 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.270+0000 7f0ec5712b80 1 client.15750 shutdown 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 msgr2=0x564e4e2cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2cfd60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005950 tx=0x7f0ea0010890 comp rx=0 tx=0).stop 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 shutdown_connections 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e2e0150 0x564e4e2e2540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/129369882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2c6f60 0x564e4e2cfd60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 >> 172.21.15.63:0/129369882 conn(0x564e4e1f4c60 msgr2=0x564e4e2bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0e9e000700 0 client.15750 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 shutdown_connections 2024-05-11T13:40:11.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 1 -- 172.21.15.63:0/129369882 wait complete. 2024-05-11T13:40:11.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.271+0000 7f0ec5712b80 20 client.15750 trim_cache size 0 max 16384 2024-05-11T13:40:11.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 Processor -- start 2024-05-11T13:40:11.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 -- start start 2024-05-11T13:40:11.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c65a0 0x564e4e2dc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c6f60 0x564e4e2bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41b18d0 con 0x564e4e2cbfd0 2024-05-11T13:40:11.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41b3440 con 0x564e4e2c6f60 2024-05-11T13:40:11.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.294+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41b1fa0 con 0x564e4e2c65a0 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2bb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35420/0 (socket says 172.21.15.63:35420) 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 -- 172.21.15.63:0/3372613483 learned_addr learned my addr 172.21.15.63:0/3372613483 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c65a0 0x564e4e2dc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb3400700 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c6f60 0x564e4e2bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 -- 172.21.15.63:0/3372613483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c65a0 msgr2=0x564e4e2dc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c65a0 0x564e4e2dc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 -- 172.21.15.63:0/3372613483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c6f60 msgr2=0x564e4e2bb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c6f60 0x564e4e2bb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb8e00700 1 -- 172.21.15.63:0/3372613483 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e4e2bda90 con 0x564e4e2cbfd0 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.295+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c65a0 0x564e4e2dc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.296+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2bb610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009fd0 tx=0x7f0eac009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dda524eb9722d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.296+0000 7f0ea8c00700 1 -- 172.21.15.63:0/3372613483 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac012040 con 0x564e4e2cbfd0 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.296+0000 7f0ea8c00700 1 -- 172.21.15.63:0/3372613483 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eac025880 con 0x564e4e2cbfd0 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.296+0000 7f0ea8c00700 1 -- 172.21.15.63:0/3372613483 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0246c0 con 0x564e4e2cbfd0 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 -- 172.21.15.63:0/3372613483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 msgr2=0x564e4e2bb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2bb610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009fd0 tx=0x7f0eac009300 comp rx=0 tx=0).stop 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 -- 172.21.15.63:0/3372613483 shutdown_connections 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2bb610 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c65a0 0x564e4e2dc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/3372613483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c6f60 0x564e4e2bb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 -- 172.21.15.63:0/3372613483 >> 172.21.15.63:0/3372613483 conn(0x564e4e1f4c60 msgr2=0x564e4e2caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:11.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 -- 172.21.15.63:0/3372613483 shutdown_connections 2024-05-11T13:40:11.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 -- 172.21.15.63:0/3372613483 wait complete. 2024-05-11T13:40:11.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.297+0000 7f0ec5712b80 1 -- start start 2024-05-11T13:40:11.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 1 Processor -- start 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c6f60 0x564e4e2d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41b2d70 con 0x564e4e2cbfd0 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41b2670 con 0x564e4e2c65a0 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41af890 con 0x564e4e2c6f60 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0ec5712b80 10 client.0 _ll_register_callbacks cb 0x564e4e1bfd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.298+0000 7f0eb8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35424/0 (socket says 172.21.15.63:35424) 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 -- 172.21.15.63:0/2421176975 learned_addr learned my addr 172.21.15.63:0/2421176975 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb3400700 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c6f60 0x564e4e2d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 -- 172.21.15.63:0/2421176975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c6f60 msgr2=0x564e4e2d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c6f60 0x564e4e2d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 -- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 msgr2=0x564e4e2d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 -- 172.21.15.63:0/2421176975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4e34a9f0 con 0x564e4e2cbfd0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0eb8e00700 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d40e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0eac019b70 tx=0x7f0eac025a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5588ba5db0f5c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.299+0000 7f0e9e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0e9e000700 1 -- 172.21.15.63:0/2421176975 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac02f410 con 0x564e4e2cbfd0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0e9e000700 1 -- 172.21.15.63:0/2421176975 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9613+0+0 (secure 0 0 0) 0x7f0eac02f5b0 con 0x564e4e2cbfd0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eac025880 con 0x564e4e2cbfd0 2024-05-11T13:40:11.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0e9e000700 1 -- 172.21.15.63:0/2421176975 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0eac03a050 con 0x564e4e2cbfd0 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0ec5712b80 10 client.15756 fetch_fsmap learned FSMap version 49 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564e4e34bd30 con 0x564e4e2cbfd0 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0e9e000700 1 -- 172.21.15.63:0/2421176975 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eac025880 con 0x564e4e2cbfd0 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0e9e000700 1 -- 172.21.15.63:0/2421176975 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f0eac024070 con 0x564e4e2cbfd0 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0ec5712b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 49 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.300+0000 7f0ec5712b80 10 client.15756 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.301+0000 7f0ec5712b80 20 client.15756 populate_metadata read hostname 'smithi063' 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.301+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e34c380 0x564e4e34e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.301+0000 7f0ec5712b80 4 client.15756 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "268fae8f", "release": true, "await": true}] 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.301+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "268fae8f", "release": true, "await": true}) -- 0x564e4e226500 con 0x564e4e34c380 2024-05-11T13:40:11.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.301+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e34c380 0x564e4e34e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.302+0000 7f0eb3e00700 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e34c380 0x564e4e34e770 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d33b654e742a05d server_cookie=fb947aa0b3afa9bc in_seq=0 out_seq=0 2024-05-11T13:40:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.302+0000 7f0e9e000700 10 client.15756 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:11.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.357+0000 7f0e9e000700 1 -- 172.21.15.63:0/2421176975 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564e4e226500 con 0x564e4e34c380 2024-05-11T13:40:11.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.357+0000 7f0e9e000700 10 client.15756 handle_command_reply: tid=0 2024-05-11T13:40:11.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.357+0000 7f0ec5712b80 1 client.15756 shutdown 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 msgr2=0x564e4e2d40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d40e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0eac019b70 tx=0x7f0eac025a30 comp rx=0 tx=0).stop 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 shutdown_connections 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564e4e2cbfd0 0x564e4e2d40e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564e4e34c380 0x564e4e34e770 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e4e2c65a0 0x564e4e2d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 --2- 172.21.15.63:0/2421176975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e4e2c6f60 0x564e4e2d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0e9e000700 0 client.15756 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:11.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.358+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 >> 172.21.15.63:0/2421176975 conn(0x564e4e1f4c60 msgr2=0x564e4e2c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:11.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.359+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 shutdown_connections 2024-05-11T13:40:11.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.359+0000 7f0ec5712b80 1 -- 172.21.15.63:0/2421176975 wait complete. 2024-05-11T13:40:11.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.360+0000 7f0ec5712b80 20 client.15756 trim_cache size 0 max 16384 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 49, 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 20, 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout: "268fae8f": { 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 20, 2024-05-11T13:40:11.355 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 80.8, 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:40:11.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:11.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:11.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:11.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:11.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:11.357 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:40:11.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.361+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0e980b2a70 msgr2=0x7f0e980b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.361+0000 7f0ebae00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0e980b2a70 0x7f0e980b4f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007760 tx=0x7f0eac004b40 comp rx=0 tx=0).stop 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.361+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb4106150 msgr2=0x7f0eb419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.361+0000 7f0ebae00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb4106150 0x7f0eb419c2a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000d950 tx=0x7f0ea000dd10 comp rx=0 tx=0).stop 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 shutdown_connections 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0e980b2a70 0x7f0e980b4f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb410b790 0x7f0eb41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb4105790 0x7f0eb419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 --2- 172.21.15.63:0/3374699658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eb4106150 0x7f0eb419c2a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 >> 172.21.15.63:0/3374699658 conn(0x7f0eb4101280 msgr2=0x7f0eb4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 shutdown_connections 2024-05-11T13:40:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:11.362+0000 7f0ebae00700 1 -- 172.21.15.63:0/3374699658 wait complete. 2024-05-11T13:40:12.283 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '268fae8f', seconds elapsed: 38.3 2024-05-11T13:40:12.283 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.85 2024-05-11T13:40:12.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:40:12.824 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:40:13.227 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:13 smithi063 ceph-mon[29809]: pgmap v194: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-11T13:40:13.227 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:13 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:40:13.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:13 smithi165 ceph-mon[33235]: pgmap v194: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-11T13:40:13.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:13 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:40:13.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:13 smithi071 ceph-mon[33293]: pgmap v194: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-11T13:40:13.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:13 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:40:14.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:14 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:14 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:14 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:14 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.601 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:14 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.601 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:14 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.601 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:14 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.601 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:14 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:14 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:14 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:14 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:14.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:14 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:15.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 -- 172.21.15.63:0/3084936019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0570075780 msgr2=0x7f0570075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:15.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 --2- 172.21.15.63:0/3084936019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0570075780 0x7f0570075b60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0560009990 tx=0x7f0560009ca0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 -- 172.21.15.63:0/3084936019 shutdown_connections 2024-05-11T13:40:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 --2- 172.21.15.63:0/3084936019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f057010ff80 0x7f0570112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 --2- 172.21.15.63:0/3084936019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0570075780 0x7f0570075b60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 --2- 172.21.15.63:0/3084936019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05700760a0 0x7f057010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 -- 172.21.15.63:0/3084936019 >> 172.21.15.63:0/3084936019 conn(0x7f057007a700 msgr2=0x7f057007ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 -- 172.21.15.63:0/3084936019 shutdown_connections 2024-05-11T13:40:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.190+0000 7f0576a00700 1 -- 172.21.15.63:0/3084936019 wait complete. 2024-05-11T13:40:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.192+0000 7f0576a00700 1 Processor -- start 2024-05-11T13:40:15.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.192+0000 7f0576a00700 1 -- start start 2024-05-11T13:40:15.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f0576a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0570075780 0x7f05701a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:15.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f0576a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05700760a0 0x7f05701a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:15.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f0576a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f057010ff80 0x7f057019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:15.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f0576a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0570115800 con 0x7f05700760a0 2024-05-11T13:40:15.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f0576a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0570115680 con 0x7f0570075780 2024-05-11T13:40:15.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f0576a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0570115980 con 0x7f057010ff80 2024-05-11T13:40:15.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f056f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05700760a0 0x7f05701a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:15.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f056f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05700760a0 0x7f05701a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35448/0 (socket says 172.21.15.63:35448) 2024-05-11T13:40:15.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f056f400700 1 -- 172.21.15.63:0/640639236 learned_addr learned my addr 172.21.15.63:0/640639236 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:15.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.193+0000 7f056fe00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0570075780 0x7f05701a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:15.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.194+0000 7f0574a00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f057010ff80 0x7f057019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:15.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.194+0000 7f056f400700 1 -- 172.21.15.63:0/640639236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f057010ff80 msgr2=0x7f057019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:15.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.194+0000 7f056f400700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f057010ff80 0x7f057019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.194+0000 7f056f400700 1 -- 172.21.15.63:0/640639236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0570075780 msgr2=0x7f05701a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:15.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.194+0000 7f056f400700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0570075780 0x7f05701a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.194+0000 7f056f400700 1 -- 172.21.15.63:0/640639236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057019c260 con 0x7f05700760a0 2024-05-11T13:40:15.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.195+0000 7f056f400700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05700760a0 0x7f05701a1e80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f055c00d980 tx=0x7f055c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:15.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.195+0000 7f056cc00700 1 -- 172.21.15.63:0/640639236 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c017030 con 0x7f05700760a0 2024-05-11T13:40:15.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.195+0000 7f056cc00700 1 -- 172.21.15.63:0/640639236 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f055c0044b0 con 0x7f05700760a0 2024-05-11T13:40:15.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.195+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057019c550 con 0x7f05700760a0 2024-05-11T13:40:15.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.195+0000 7f056cc00700 1 -- 172.21.15.63:0/640639236 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0038e0 con 0x7f05700760a0 2024-05-11T13:40:15.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.195+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05701a8810 con 0x7f05700760a0 2024-05-11T13:40:15.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.197+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0550005330 con 0x7f05700760a0 2024-05-11T13:40:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.202+0000 7f056cc00700 1 -- 172.21.15.63:0/640639236 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f055c01a070 con 0x7f05700760a0 2024-05-11T13:40:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.205+0000 7f056cc00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f05540b2a70 0x7f05540b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.205+0000 7f056cc00700 1 -- 172.21.15.63:0/640639236 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7f055c0e2290 con 0x7f05700760a0 2024-05-11T13:40:15.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.205+0000 7f056fe00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f05540b2a70 0x7f05540b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.207+0000 7f056fe00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f05540b2a70 0x7f05540b4f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f056000b5c0 tx=0x7f0560005960 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.211+0000 7f056cc00700 1 -- 172.21.15.63:0/640639236 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f055c0a9820 con 0x7f05700760a0 2024-05-11T13:40:15.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:15 smithi063 ceph-mon[29809]: pgmap v195: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-05-11T13:40:15.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:15 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:40:15.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:15 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:40:15.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:15 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:15.428 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:15 smithi063 ceph-mon[29809]: osdmap e93: 12 total, 12 up, 12 in 2024-05-11T13:40:15.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.560+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05500058f0 con 0x7f05700760a0 2024-05-11T13:40:15.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.561+0000 7f056cc00700 1 -- 172.21.15.63:0/640639236 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11936 (secure 0 0 0) 0x7f055c0a9120 con 0x7f05700760a0 2024-05-11T13:40:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:40:15.556 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:40:01.464213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:40:15.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.564+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f05540b2a70 msgr2=0x7f05540b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:15.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.564+0000 7f0576a00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f05540b2a70 0x7f05540b4f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f056000b5c0 tx=0x7f0560005960 comp rx=0 tx=0).stop 2024-05-11T13:40:15.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.564+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05700760a0 msgr2=0x7f05701a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:15.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:15 smithi165 ceph-mon[33235]: pgmap v195: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-05-11T13:40:15.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:15 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:40:15.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:15 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:40:15.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:15 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:15.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:15 smithi165 ceph-mon[33235]: osdmap e93: 12 total, 12 up, 12 in 2024-05-11T13:40:15.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.564+0000 7f0576a00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05700760a0 0x7f05701a1e80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f055c00d980 tx=0x7f055c00dc90 comp rx=0 tx=0).stop 2024-05-11T13:40:15.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 shutdown_connections 2024-05-11T13:40:15.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f05540b2a70 0x7f05540b4f30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f057010ff80 0x7f057019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0570075780 0x7f05701a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 --2- 172.21.15.63:0/640639236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05700760a0 0x7f05701a1e80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 >> 172.21.15.63:0/640639236 conn(0x7f057007a700 msgr2=0x7f0570100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 shutdown_connections 2024-05-11T13:40:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:15.565+0000 7f0576a00700 1 -- 172.21.15.63:0/640639236 wait complete. 2024-05-11T13:40:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 49 2024-05-11T13:40:15.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:15 smithi071 ceph-mon[33293]: pgmap v195: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-05-11T13:40:15.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:15 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:40:15.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:15 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:40:15.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:15 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:40:15.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:15 smithi071 ceph-mon[33293]: osdmap e93: 12 total, 12 up, 12 in 2024-05-11T13:40:16.457 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 80.85 --await-for 42.0' 2024-05-11T13:40:16.457 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 80.85 --await-for 42.0 2024-05-11T13:40:16.522 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:16 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/640639236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:16.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:16 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/640639236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:16.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:16 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/640639236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:16.946 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:40:17.351 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:17 smithi063 ceph-mon[29809]: pgmap v197: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-05-11T13:40:17.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:17 smithi165 ceph-mon[33235]: pgmap v197: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-05-11T13:40:17.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:17 smithi071 ceph-mon[33293]: pgmap v197: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-05-11T13:40:18.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:18 smithi165 ceph-mon[33235]: pgmap v198: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:40:18.667 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:18 smithi063 ceph-mon[29809]: pgmap v198: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:40:18.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:18 smithi071 ceph-mon[33293]: pgmap v198: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 -- 172.21.15.63:0/4198509398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814106140 msgr2=0x7f3814112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 --2- 172.21.15.63:0/4198509398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814106140 0x7f3814112650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b0a0 tx=0x7f380c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 -- 172.21.15.63:0/4198509398 shutdown_connections 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 --2- 172.21.15.63:0/4198509398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814106140 0x7f3814112650 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 --2- 172.21.15.63:0/4198509398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f381410b780 0x7f381410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 --2- 172.21.15.63:0/4198509398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3814105780 0x7f3814105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 -- 172.21.15.63:0/4198509398 >> 172.21.15.63:0/4198509398 conn(0x7f3814101270 msgr2=0x7f3814103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 -- 172.21.15.63:0/4198509398 shutdown_connections 2024-05-11T13:40:19.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.351+0000 7f3819000700 1 -- 172.21.15.63:0/4198509398 wait complete. 2024-05-11T13:40:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.354+0000 7f3819000700 1 Processor -- start 2024-05-11T13:40:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.354+0000 7f3819000700 1 -- start start 2024-05-11T13:40:19.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3819000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814105780 0x7f38141a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3819000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3814106140 0x7f38141a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3819000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f381410b780 0x7f381419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3819000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38141158c0 con 0x7f3814105780 2024-05-11T13:40:19.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3819000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3814115740 con 0x7f3814106140 2024-05-11T13:40:19.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3819000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3814115a40 con 0x7f381410b780 2024-05-11T13:40:19.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3811e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814105780 0x7f38141a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3811e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814105780 0x7f38141a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35460/0 (socket says 172.21.15.63:35460) 2024-05-11T13:40:19.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3811e00700 1 -- 172.21.15.63:0/842336955 learned_addr learned my addr 172.21.15.63:0/842336955 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:19.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3811400700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3814106140 0x7f38141a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.355+0000 7f3812800700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f381410b780 0x7f381419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f3811e00700 1 -- 172.21.15.63:0/842336955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f381410b780 msgr2=0x7f381419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f3811e00700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f381410b780 0x7f381419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f3811e00700 1 -- 172.21.15.63:0/842336955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3814106140 msgr2=0x7f38141a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f3811e00700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3814106140 0x7f38141a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f3811e00700 1 -- 172.21.15.63:0/842336955 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381419c240 con 0x7f3814105780 2024-05-11T13:40:19.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f3811e00700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814105780 0x7f38141a18d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9a0 tx=0x7f380400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:19.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f380aa00700 1 -- 172.21.15.63:0/842336955 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380400cbd0 con 0x7f3814105780 2024-05-11T13:40:19.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f380aa00700 1 -- 172.21.15.63:0/842336955 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f380400cd70 con 0x7f3814105780 2024-05-11T13:40:19.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.356+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381419c530 con 0x7f3814105780 2024-05-11T13:40:19.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.357+0000 7f380aa00700 1 -- 172.21.15.63:0/842336955 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804018680 con 0x7f3814105780 2024-05-11T13:40:19.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.357+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141a87a0 con 0x7f3814105780 2024-05-11T13:40:19.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.361+0000 7f380aa00700 1 -- 172.21.15.63:0/842336955 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3804010950 con 0x7f3814105780 2024-05-11T13:40:19.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.362+0000 7f382392cb80 1 Processor -- start 2024-05-11T13:40:19.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.362+0000 7f382392cb80 1 -- start start 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f380aa00700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f37f80b2aa0 0x7f37f80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f382392cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f382392cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f382392cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f380aa00700 1 -- 172.21.15.63:0/842336955 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7f38040e2d40 con 0x7f3814105780 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f382392cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a613310bb0 con 0x55a6133beff0 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f382392cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a6132ae8f0 con 0x55a6133be630 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f382392cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a61330cb70 con 0x55a6133c4060 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f3812800700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f37f80b2aa0 0x7f37f80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.363+0000 7f3812800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3811e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3812800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35464/0 (socket says 172.21.15.63:35464) 2024-05-11T13:40:19.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3812800700 1 -- 172.21.15.63:0/2658229859 learned_addr learned my addr 172.21.15.63:0/2658229859 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:19.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3811400700 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3812800700 1 -- 172.21.15.63:0/2658229859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 msgr2=0x55a6133c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3812800700 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133c4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3812800700 1 -- 172.21.15.63:0/2658229859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 msgr2=0x55a6133beab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3812800700 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133beab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3812800700 1 -- 172.21.15.63:0/2658229859 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a61325da60 con 0x55a6133beff0 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3811e00700 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.364+0000 7f3811400700 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.365+0000 7f3812800700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f37f80b2aa0 0x7f37f80b4f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e60 tx=0x7f380c0092e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.365+0000 7f3812800700 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cae20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b0a0 tx=0x7f380c025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a56d4a7cbed364c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.365+0000 7f37ff400700 1 -- 172.21.15.63:0/2658229859 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c004d40 con 0x55a6133beff0 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.365+0000 7f37ff400700 1 -- 172.21.15.63:0/2658229859 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f380c009b80 con 0x55a6133beff0 2024-05-11T13:40:19.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.365+0000 7f37ff400700 1 -- 172.21.15.63:0/2658229859 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c020860 con 0x55a6133beff0 2024-05-11T13:40:19.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 -- 172.21.15.63:0/2658229859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 msgr2=0x55a6133cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cae20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b0a0 tx=0x7f380c025000 comp rx=0 tx=0).stop 2024-05-11T13:40:19.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 -- 172.21.15.63:0/2658229859 shutdown_connections 2024-05-11T13:40:19.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cae20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 --2- 172.21.15.63:0/2658229859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 -- 172.21.15.63:0/2658229859 >> 172.21.15.63:0/2658229859 conn(0x55a6132eccf0 msgr2=0x55a6133b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:19.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.366+0000 7f382392cb80 1 -- 172.21.15.63:0/2658229859 shutdown_connections 2024-05-11T13:40:19.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.367+0000 7f382392cb80 1 -- 172.21.15.63:0/2658229859 wait complete. 2024-05-11T13:40:19.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.367+0000 7f382392cb80 1 -- start start 2024-05-11T13:40:19.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.367+0000 7f382392cb80 1 Processor -- start 2024-05-11T13:40:19.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.368+0000 7f382392cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.368+0000 7f382392cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.368+0000 7f382392cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 0x55a6133c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.368+0000 7f382392cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a6133cb3c0 con 0x55a6133beff0 2024-05-11T13:40:19.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.368+0000 7f382392cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a613276cb0 con 0x55a6133c4060 2024-05-11T13:40:19.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.368+0000 7f382392cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a6133cb540 con 0x55a6133be630 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.368+0000 7f382392cb80 10 client.0 _ll_register_callbacks cb 0x55a6132b7e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35474/0 (socket says 172.21.15.63:35474) 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 -- 172.21.15.63:0/1217498893 learned_addr learned my addr 172.21.15.63:0/1217498893 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811e00700 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 -- 172.21.15.63:0/1217498893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 msgr2=0x55a6133cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 -- 172.21.15.63:0/1217498893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 msgr2=0x55a6133c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 0x55a6133c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.369+0000 7f3811400700 1 -- 172.21.15.63:0/1217498893 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6132d65b0 con 0x55a6133beff0 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.370+0000 7f3811400700 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133c7eb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3800003f50 tx=0x7f3800003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca0ca59ce085419 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.370+0000 7f37f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.370+0000 7f37f3e00700 1 -- 172.21.15.63:0/1217498893 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380001b670 con 0x55a6133beff0 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.371+0000 7f37f3e00700 1 -- 172.21.15.63:0/1217498893 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7f380001b810 con 0x55a6133beff0 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.371+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f380c009b80 con 0x55a6133beff0 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.371+0000 7f37f3e00700 1 -- 172.21.15.63:0/1217498893 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3800024050 con 0x55a6133beff0 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.371+0000 7f382392cb80 10 client.15774 fetch_fsmap learned FSMap version 49 2024-05-11T13:40:19.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.371+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a6133d5c70 con 0x55a6133beff0 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.371+0000 7f37f3e00700 1 -- 172.21.15.63:0/1217498893 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f380c009b80 con 0x55a6133beff0 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.371+0000 7f37f3e00700 1 -- 172.21.15.63:0/1217498893 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f3800012aa0 con 0x55a6133beff0 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.372+0000 7f382392cb80 10 client.15774 fetch_fsmap finished waiting for FSMap version 49 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.372+0000 7f382392cb80 10 client.15774 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.372+0000 7f382392cb80 20 client.15774 populate_metadata read hostname 'smithi063' 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.372+0000 7f382392cb80 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a6133d8300 0x55a6133da730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.372+0000 7f382392cb80 4 client.15774 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:40:19.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.372+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a61338b340 con 0x55a6133d8300 2024-05-11T13:40:19.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.372+0000 7f3811e00700 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a6133d8300 0x55a6133da730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.373+0000 7f3811e00700 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a6133d8300 0x55a6133da730 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882f1c49791a20fb server_cookie=1e61b32146e84d00 in_seq=0 out_seq=0 2024-05-11T13:40:19.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.373+0000 7f37f3e00700 10 client.15774 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:19.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.375+0000 7f37f3e00700 1 -- 172.21.15.63:0/1217498893 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a61338b340 con 0x55a6133d8300 2024-05-11T13:40:19.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.375+0000 7f37f3e00700 10 client.15774 handle_command_reply: tid=0 2024-05-11T13:40:19.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.375+0000 7f382392cb80 1 client.15774 shutdown 2024-05-11T13:40:19.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.375+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 msgr2=0x55a6133c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.375+0000 7f382392cb80 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133c7eb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3800003f50 tx=0x7f3800003f80 comp rx=0 tx=0).stop 2024-05-11T13:40:19.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 shutdown_connections 2024-05-11T13:40:19.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a6133d8300 0x55a6133da730 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 0x55a6133c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 --2- 172.21.15.63:0/1217498893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133c7eb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 >> 172.21.15.63:0/1217498893 conn(0x55a6132eccf0 msgr2=0x55a6133b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:19.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f37f3e00700 0 client.15774 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:19.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 shutdown_connections 2024-05-11T13:40:19.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 1 -- 172.21.15.63:0/1217498893 wait complete. 2024-05-11T13:40:19.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.376+0000 7f382392cb80 20 client.15774 trim_cache size 0 max 16384 2024-05-11T13:40:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.400+0000 7f382392cb80 1 Processor -- start 2024-05-11T13:40:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f382392cb80 1 -- start start 2024-05-11T13:40:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f382392cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f382392cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f382392cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133e0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f382392cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b17d0 con 0x55a6133beff0 2024-05-11T13:40:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f382392cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b3340 con 0x55a6133be630 2024-05-11T13:40:19.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f382392cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b1ea0 con 0x55a6133c4060 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f3811400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f3811400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35484/0 (socket says 172.21.15.63:35484) 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.401+0000 7f3811400700 1 -- 172.21.15.63:0/3353845405 learned_addr learned my addr 172.21.15.63:0/3353845405 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811e00700 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3812800700 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811400700 1 -- 172.21.15.63:0/3353845405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 msgr2=0x55a6133e0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811400700 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133e0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811400700 1 -- 172.21.15.63:0/3353845405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 msgr2=0x55a6133c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811400700 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811400700 1 -- 172.21.15.63:0/3353845405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6133b5b20 con 0x55a6133beff0 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811e00700 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3812800700 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:40:19.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.402+0000 7f3811400700 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133d4410 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3800001eb0 tx=0x7f3800005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165b61673942771 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:19.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.403+0000 7f37fea00700 1 -- 172.21.15.63:0/3353845405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800009e90 con 0x55a6133beff0 2024-05-11T13:40:19.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.403+0000 7f37fea00700 1 -- 172.21.15.63:0/3353845405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f380002c020 con 0x55a6133beff0 2024-05-11T13:40:19.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.403+0000 7f37fea00700 1 -- 172.21.15.63:0/3353845405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000127d0 con 0x55a6133beff0 2024-05-11T13:40:19.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 -- 172.21.15.63:0/3353845405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 msgr2=0x55a6133d4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133d4410 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3800001eb0 tx=0x7f3800005980 comp rx=0 tx=0).stop 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 -- 172.21.15.63:0/3353845405 shutdown_connections 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133c4060 0x55a6133e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133be630 0x55a6133c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 --2- 172.21.15.63:0/3353845405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133d4410 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 -- 172.21.15.63:0/3353845405 >> 172.21.15.63:0/3353845405 conn(0x55a6132eccf0 msgr2=0x55a6133bdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 -- 172.21.15.63:0/3353845405 shutdown_connections 2024-05-11T13:40:19.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.404+0000 7f382392cb80 1 -- 172.21.15.63:0/3353845405 wait complete. 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.405+0000 7f382392cb80 1 -- start start 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.405+0000 7f382392cb80 1 Processor -- start 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f382392cb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f382392cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f382392cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 0x55a6133cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f382392cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b2c70 con 0x55a6133beff0 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f382392cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b2570 con 0x55a6133c4060 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f382392cb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38141af790 con 0x55a6133be630 2024-05-11T13:40:19.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f382392cb80 10 client.0 _ll_register_callbacks cb 0x55a6132b7e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:40:19.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35500/0 (socket says 172.21.15.63:35500) 2024-05-11T13:40:19.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811400700 1 -- 172.21.15.63:0/3753096085 learned_addr learned my addr 172.21.15.63:0/3753096085 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811e00700 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3812800700 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 0x55a6133cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811400700 1 -- 172.21.15.63:0/3753096085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 msgr2=0x55a6133d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811400700 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811400700 1 -- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 msgr2=0x55a6133cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.406+0000 7f3811400700 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 0x55a6133cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.407+0000 7f3811400700 1 -- 172.21.15.63:0/3753096085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6133ce230 con 0x55a6133beff0 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.407+0000 7f3811400700 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cd760 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3800012f60 tx=0x7f3800011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e864cb36c12361c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.407+0000 7f37f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.407+0000 7f37f3e00700 1 -- 172.21.15.63:0/3753096085 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800035070 con 0x55a6133beff0 2024-05-11T13:40:19.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.407+0000 7f37f3e00700 1 -- 172.21.15.63:0/3753096085 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7f380002d040 con 0x55a6133beff0 2024-05-11T13:40:19.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.407+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f380002c020 con 0x55a6133beff0 2024-05-11T13:40:19.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.408+0000 7f37f3e00700 1 -- 172.21.15.63:0/3753096085 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f380003f050 con 0x55a6133beff0 2024-05-11T13:40:19.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.408+0000 7f382392cb80 10 client.15780 fetch_fsmap learned FSMap version 49 2024-05-11T13:40:19.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.408+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a613443910 con 0x55a6133beff0 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.408+0000 7f37f3e00700 1 -- 172.21.15.63:0/3753096085 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f380002c020 con 0x55a6133beff0 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.408+0000 7f37f3e00700 1 -- 172.21.15.63:0/3753096085 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f380001b590 con 0x55a6133beff0 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.409+0000 7f382392cb80 10 client.15780 fetch_fsmap finished waiting for FSMap version 49 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.409+0000 7f382392cb80 10 client.15780 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.409+0000 7f382392cb80 20 client.15780 populate_metadata read hostname 'smithi063' 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.409+0000 7f382392cb80 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a613443fc0 0x55a6134463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.409+0000 7f382392cb80 4 client.15780 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.85, "await_for": 42.0}] 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.409+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.85, "await_for": 42.0}) -- 0x55a61331e590 con 0x55a613443fc0 2024-05-11T13:40:19.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.409+0000 7f3811e00700 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a613443fc0 0x55a6134463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:19.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.410+0000 7f3811e00700 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a613443fc0 0x55a6134463e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e479b99c588b16 server_cookie=896e8edc4d12381b in_seq=0 out_seq=0 2024-05-11T13:40:19.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.410+0000 7f37f3e00700 10 client.15780 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:19.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.684+0000 7f37f3e00700 1 -- 172.21.15.63:0/3753096085 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55a61331e590 con 0x55a613443fc0 2024-05-11T13:40:19.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.684+0000 7f37f3e00700 10 client.15780 handle_command_reply: tid=0 2024-05-11T13:40:19.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.684+0000 7f382392cb80 1 client.15780 shutdown 2024-05-11T13:40:19.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 msgr2=0x55a6133cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cd760 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3800012f60 tx=0x7f3800011b80 comp rx=0 tx=0).stop 2024-05-11T13:40:19.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 shutdown_connections 2024-05-11T13:40:19.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55a613443fc0 0x55a6134463e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6133c4060 0x55a6133cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6133be630 0x55a6133d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 --2- 172.21.15.63:0/3753096085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6133beff0 0x55a6133cd760 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 >> 172.21.15.63:0/3753096085 conn(0x55a6132eccf0 msgr2=0x55a6133c2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:19.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.685+0000 7f37f3e00700 0 client.15780 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:19.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.686+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 shutdown_connections 2024-05-11T13:40:19.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.686+0000 7f382392cb80 1 -- 172.21.15.63:0/3753096085 wait complete. 2024-05-11T13:40:19.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.686+0000 7f382392cb80 20 client.15780 trim_cache size 0 max 16384 2024-05-11T13:40:19.681 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 49, 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 22, 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "d73159a2": { 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 22, 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:40:19.682 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 80.8, 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:19.683 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f37f80b2aa0 msgr2=0x7f37f80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f37f80b2aa0 0x7f37f80b4f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e60 tx=0x7f380c0092e0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814105780 msgr2=0x7f38141a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814105780 0x7f38141a18d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9a0 tx=0x7f380400ed60 comp rx=0 tx=0).stop 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 shutdown_connections 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f37f80b2aa0 0x7f37f80b4f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f381410b780 0x7f381419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3814105780 0x7f38141a18d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 --2- 172.21.15.63:0/842336955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3814106140 0x7f38141a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:19.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 >> 172.21.15.63:0/842336955 conn(0x7f3814101270 msgr2=0x7f3814101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:19.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 shutdown_connections 2024-05-11T13:40:19.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:19.688+0000 7f3819000700 1 -- 172.21.15.63:0/842336955 wait complete. 2024-05-11T13:40:20.529 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd73159a2', quiesce took 8.2 seconds. Will release after: 20.85 2024-05-11T13:40:21.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:21 smithi165 ceph-mon[33235]: pgmap v199: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:21.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:21 smithi063 ceph-mon[29809]: pgmap v199: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:21.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:21 smithi071 ceph-mon[33293]: pgmap v199: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:23.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:23 smithi165 ceph-mon[33235]: pgmap v200: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:23.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:23 smithi063 ceph-mon[29809]: pgmap v200: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:23.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:23 smithi071 ceph-mon[33293]: pgmap v200: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:25.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:25 smithi165 ceph-mon[33235]: pgmap v201: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:25.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:25 smithi063 ceph-mon[29809]: pgmap v201: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:25.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:25 smithi071 ceph-mon[33293]: pgmap v201: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-11T13:40:27.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:27 smithi165 ceph-mon[33235]: pgmap v202: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.9 MiB/s wr, 15 op/s 2024-05-11T13:40:27.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:27 smithi063 ceph-mon[29809]: pgmap v202: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.9 MiB/s wr, 15 op/s 2024-05-11T13:40:27.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:27 smithi071 ceph-mon[33293]: pgmap v202: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.9 MiB/s wr, 15 op/s 2024-05-11T13:40:28.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:28 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:40:28.560 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:28 smithi165 ceph-mon[33235]: pgmap v203: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 18 op/s 2024-05-11T13:40:28.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:28 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:40:28.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:28 smithi063 ceph-mon[29809]: pgmap v203: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 18 op/s 2024-05-11T13:40:28.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:28 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:40:28.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:28 smithi071 ceph-mon[33293]: pgmap v203: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 18 op/s 2024-05-11T13:40:31.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:31 smithi165 ceph-mon[33235]: pgmap v204: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 10 op/s 2024-05-11T13:40:31.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:31 smithi063 ceph-mon[29809]: pgmap v204: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 10 op/s 2024-05-11T13:40:31.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:31 smithi071 ceph-mon[33293]: pgmap v204: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 10 op/s 2024-05-11T13:40:33.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:33 smithi165 ceph-mon[33235]: pgmap v205: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 10 op/s 2024-05-11T13:40:33.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:33 smithi063 ceph-mon[29809]: pgmap v205: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 10 op/s 2024-05-11T13:40:33.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:33 smithi071 ceph-mon[33293]: pgmap v205: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 10 op/s 2024-05-11T13:40:35.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:35 smithi165 ceph-mon[33235]: pgmap v206: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.2 MiB/s wr, 15 op/s 2024-05-11T13:40:35.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:35 smithi063 ceph-mon[29809]: pgmap v206: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.2 MiB/s wr, 15 op/s 2024-05-11T13:40:35.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:35 smithi071 ceph-mon[33293]: pgmap v206: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.2 MiB/s wr, 15 op/s 2024-05-11T13:40:37.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:37 smithi165 ceph-mon[33235]: pgmap v207: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 525 KiB/s wr, 9 op/s 2024-05-11T13:40:37.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:37 smithi063 ceph-mon[29809]: pgmap v207: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 525 KiB/s wr, 9 op/s 2024-05-11T13:40:37.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:37 smithi071 ceph-mon[33293]: pgmap v207: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 525 KiB/s wr, 9 op/s 2024-05-11T13:40:39.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:39 smithi165 ceph-mon[33235]: pgmap v208: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 529 KiB/s wr, 14 op/s 2024-05-11T13:40:39.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:39 smithi063 ceph-mon[29809]: pgmap v208: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 529 KiB/s wr, 14 op/s 2024-05-11T13:40:39.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:39 smithi071 ceph-mon[33293]: pgmap v208: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 529 KiB/s wr, 14 op/s 2024-05-11T13:40:40.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:40 smithi165 ceph-mon[33235]: pgmap v209: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-11T13:40:40.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:40 smithi063 ceph-mon[29809]: pgmap v209: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-11T13:40:40.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:40 smithi071 ceph-mon[33293]: pgmap v209: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-11T13:40:41.381 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:40:41.882 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:40:43.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:43 smithi063 ceph-mon[29809]: pgmap v210: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-11T13:40:43.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:43 smithi165 ceph-mon[33235]: pgmap v210: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-11T13:40:43.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:43 smithi071 ceph-mon[33293]: pgmap v210: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 -- 172.21.15.63:0/2277751086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec06b230 msgr2=0x7f6eec06b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 --2- 172.21.15.63:0/2277751086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec06b230 0x7f6eec06b6b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009990 tx=0x7f6ed4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 -- 172.21.15.63:0/2277751086 shutdown_connections 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 --2- 172.21.15.63:0/2277751086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eec06bbf0 0x7f6eec1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 --2- 172.21.15.63:0/2277751086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eec1095c0 0x7f6eec1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 --2- 172.21.15.63:0/2277751086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec06b230 0x7f6eec06b6b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 -- 172.21.15.63:0/2277751086 >> 172.21.15.63:0/2277751086 conn(0x7f6eec079400 msgr2=0x7f6eec079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 -- 172.21.15.63:0/2277751086 shutdown_connections 2024-05-11T13:40:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.338+0000 7f6ef2200700 1 -- 172.21.15.63:0/2277751086 wait complete. 2024-05-11T13:40:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.339+0000 7f6ef2200700 1 Processor -- start 2024-05-11T13:40:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6ef2200700 1 -- start start 2024-05-11T13:40:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6ef2200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eec06b230 0x7f6eec19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6ef2200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eec06bbf0 0x7f6eec19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6ef2200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec1095c0 0x7f6eec1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6ef2200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec113580 con 0x7f6eec1095c0 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6ef2200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec113400 con 0x7f6eec06b230 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6ef2200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec113700 con 0x7f6eec06bbf0 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.340+0000 7f6eebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec1095c0 0x7f6eec1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec1095c0 0x7f6eec1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58462/0 (socket says 172.21.15.63:58462) 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 -- 172.21.15.63:0/1871355074 learned_addr learned my addr 172.21.15.63:0/1871355074 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6ee2a00700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eec06bbf0 0x7f6eec19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eeb400700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eec06b230 0x7f6eec19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 -- 172.21.15.63:0/1871355074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eec06bbf0 msgr2=0x7f6eec19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eec06bbf0 0x7f6eec19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 -- 172.21.15.63:0/1871355074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eec06b230 msgr2=0x7f6eec19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eec06b230 0x7f6eec19bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 -- 172.21.15.63:0/1871355074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec1a0e00 con 0x7f6eec1095c0 2024-05-11T13:40:44.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eeb400700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eec06b230 0x7f6eec19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:40:44.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.341+0000 7f6eebe00700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec1095c0 0x7f6eec1a06c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4007ee0 tx=0x7f6ee400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:44.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.342+0000 7f6ee8c00700 1 -- 172.21.15.63:0/1871355074 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400ddd0 con 0x7f6eec1095c0 2024-05-11T13:40:44.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.342+0000 7f6ee8c00700 1 -- 172.21.15.63:0/1871355074 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ee4004020 con 0x7f6eec1095c0 2024-05-11T13:40:44.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.342+0000 7f6ee8c00700 1 -- 172.21.15.63:0/1871355074 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4015660 con 0x7f6eec1095c0 2024-05-11T13:40:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.342+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eec1a10f0 con 0x7f6eec1095c0 2024-05-11T13:40:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.342+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eec1a8970 con 0x7f6eec1095c0 2024-05-11T13:40:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.343+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eec06a240 con 0x7f6eec1095c0 2024-05-11T13:40:44.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.346+0000 7f6ee8c00700 1 -- 172.21.15.63:0/1871355074 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6ee40041d0 con 0x7f6eec1095c0 2024-05-11T13:40:44.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.347+0000 7f6ee8c00700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6ed00b2a70 0x7f6ed00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:44.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.348+0000 7f6ee8c00700 1 -- 172.21.15.63:0/1871355074 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7f6ee40e2d40 con 0x7f6eec1095c0 2024-05-11T13:40:44.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.348+0000 7f6eeb400700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6ed00b2a70 0x7f6ed00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:44.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.349+0000 7f6eeb400700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6ed00b2a70 0x7f6ed00b4f30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00bb90 tx=0x7f6edc005d80 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:44.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.350+0000 7f6ee8c00700 1 -- 172.21.15.63:0/1871355074 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ee4010070 con 0x7f6eec1095c0 2024-05-11T13:40:44.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.656+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6eec1a8b40 con 0x7f6eec1095c0 2024-05-11T13:40:44.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.658+0000 7f6ee8c00700 1 -- 172.21.15.63:0/1871355074 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11935 (secure 0 0 0) 0x7f6ee40aa2d0 con 0x7f6eec1095c0 2024-05-11T13:40:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:40:44.651 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:40:26.465134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:40:44.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6ed00b2a70 msgr2=0x7f6ed00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:44.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6ed00b2a70 0x7f6ed00b4f30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00bb90 tx=0x7f6edc005d80 comp rx=0 tx=0).stop 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec1095c0 msgr2=0x7f6eec1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec1095c0 0x7f6eec1a06c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4007ee0 tx=0x7f6ee400d3b0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 shutdown_connections 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f6ed00b2a70 0x7f6ed00b4f30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6eec1095c0 0x7f6eec1a06c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eec06b230 0x7f6eec19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 --2- 172.21.15.63:0/1871355074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eec06bbf0 0x7f6eec19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.661+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 >> 172.21.15.63:0/1871355074 conn(0x7f6eec079400 msgr2=0x7f6eec10e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.662+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 shutdown_connections 2024-05-11T13:40:44.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:44.662+0000 7f6ef2200700 1 -- 172.21.15.63:0/1871355074 wait complete. 2024-05-11T13:40:44.656 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 50 2024-05-11T13:40:45.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:45 smithi063 ceph-mon[29809]: pgmap v211: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-05-11T13:40:45.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:45 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1871355074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:45.495 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id d73159a2 --release --await' 2024-05-11T13:40:45.495 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id d73159a2 --release --await 2024-05-11T13:40:45.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:45 smithi165 ceph-mon[33235]: pgmap v211: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-05-11T13:40:45.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:45 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1871355074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:45.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:45 smithi071 ceph-mon[33293]: pgmap v211: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-05-11T13:40:45.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:45 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1871355074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:40:45.942 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:40:47.343 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:47 smithi063 ceph-mon[29809]: pgmap v212: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-11T13:40:47.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:47 smithi165 ceph-mon[33235]: pgmap v212: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-11T13:40:47.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:47 smithi071 ceph-mon[33293]: pgmap v212: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 -- 172.21.15.63:0/1891084052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9901025e0 msgr2=0x7fc9901029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 --2- 172.21.15.63:0/1891084052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9901025e0 0x7fc9901029c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc9840099c0 tx=0x7fc984009cd0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 -- 172.21.15.63:0/1891084052 shutdown_connections 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 --2- 172.21.15.63:0/1891084052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9901076c0 0x7fc99010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 --2- 172.21.15.63:0/1891084052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9901025e0 0x7fc9901029c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 --2- 172.21.15.63:0/1891084052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc990102f90 0x7fc990106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 -- 172.21.15.63:0/1891084052 >> 172.21.15.63:0/1891084052 conn(0x7fc9900fde70 msgr2=0x7fc990100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.298+0000 7fc997200700 1 -- 172.21.15.63:0/1891084052 shutdown_connections 2024-05-11T13:40:48.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.299+0000 7fc997200700 1 -- 172.21.15.63:0/1891084052 wait complete. 2024-05-11T13:40:48.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.300+0000 7fc997200700 1 Processor -- start 2024-05-11T13:40:48.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.300+0000 7fc997200700 1 -- start start 2024-05-11T13:40:48.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc997200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9901025e0 0x7fc9901a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc997200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990102f90 0x7fc9901a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc997200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9901076c0 0x7fc99019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc997200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc990112430 con 0x7fc990102f90 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc997200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901122b0 con 0x7fc9901025e0 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc997200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901125b0 con 0x7fc9901076c0 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990102f90 0x7fc9901a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990102f90 0x7fc9901a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58472/0 (socket says 172.21.15.63:58472) 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc995400700 1 -- 172.21.15.63:0/271362505 learned_addr learned my addr 172.21.15.63:0/271362505 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.301+0000 7fc995e00700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9901025e0 0x7fc9901a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc996800700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9901076c0 0x7fc99019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc995400700 1 -- 172.21.15.63:0/271362505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9901076c0 msgr2=0x7fc99019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc995400700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9901076c0 0x7fc99019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc995400700 1 -- 172.21.15.63:0/271362505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9901025e0 msgr2=0x7fc9901a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc995400700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9901025e0 0x7fc9901a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc995400700 1 -- 172.21.15.63:0/271362505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99019c250 con 0x7fc990102f90 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc996800700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9901076c0 0x7fc99019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.302+0000 7fc995400700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990102f90 0x7fc9901a1e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d950 tx=0x7fc98000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.303+0000 7fc98aa00700 1 -- 172.21.15.63:0/271362505 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980016070 con 0x7fc990102f90 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.303+0000 7fc98aa00700 1 -- 172.21.15.63:0/271362505 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc980004830 con 0x7fc990102f90 2024-05-11T13:40:48.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.303+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99019c540 con 0x7fc990102f90 2024-05-11T13:40:48.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.303+0000 7fc98aa00700 1 -- 172.21.15.63:0/271362505 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800053e0 con 0x7fc990102f90 2024-05-11T13:40:48.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.303+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a8800 con 0x7fc990102f90 2024-05-11T13:40:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.306+0000 7fc98aa00700 1 -- 172.21.15.63:0/271362505 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc980005580 con 0x7fc990102f90 2024-05-11T13:40:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 Processor -- start 2024-05-11T13:40:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 -- start start 2024-05-11T13:40:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc98aa00700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc9780b29f0 0x7fc9780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc98aa00700 1 -- 172.21.15.63:0/271362505 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7fc9800e27f0 con 0x7fc990102f90 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5624e91fbb20 con 0x5624e92af990 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5624e9199860 con 0x5624e92af070 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.308+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5624e91f7ae0 con 0x5624e92a9bc0 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc996800700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc9780b29f0 0x7fc9780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92a95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58482/0 (socket says 172.21.15.63:58482) 2024-05-11T13:40:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 -- 172.21.15.63:0/3241508402 learned_addr learned my addr 172.21.15.63:0/3241508402 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995e00700 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc996800700 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 -- 172.21.15.63:0/3241508402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 msgr2=0x5624e92aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 -- 172.21.15.63:0/3241508402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 msgr2=0x5624e92af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.309+0000 7fc995400700 1 -- 172.21.15.63:0/3241508402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e91489d0 con 0x5624e92af990 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.310+0000 7fc996800700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc9780b29f0 0x7fc9780b4eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009710 tx=0x7fc98c009f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.310+0000 7fc995400700 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92a95a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc9800a5700 tx=0x7fc9800a9100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6f4487263c9e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.310+0000 7fc97f400700 1 -- 172.21.15.63:0/3241508402 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800ada30 con 0x5624e92af990 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.310+0000 7fc97f400700 1 -- 172.21.15.63:0/3241508402 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9800b24e0 con 0x5624e92af990 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.310+0000 7fc97f400700 1 -- 172.21.15.63:0/3241508402 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800a9b40 con 0x5624e92af990 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3241508402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 msgr2=0x5624e92a95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92a95a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc9800a5700 tx=0x7fc9800a9100 comp rx=0 tx=0).stop 2024-05-11T13:40:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3241508402 shutdown_connections 2024-05-11T13:40:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3241508402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92a95a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3241508402 >> 172.21.15.63:0/3241508402 conn(0x5624e91d7c60 msgr2=0x5624e92a4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3241508402 shutdown_connections 2024-05-11T13:40:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.311+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3241508402 wait complete. 2024-05-11T13:40:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.312+0000 7fc9a31c5b80 1 -- start start 2024-05-11T13:40:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.312+0000 7fc9a31c5b80 1 Processor -- start 2024-05-11T13:40:48.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af070 0x5624e92b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5624e92b64b0 con 0x5624e92af990 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5624e9161c20 con 0x5624e92a9bc0 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5624e92b6630 con 0x5624e92af070 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc9a31c5b80 10 client.0 _ll_register_callbacks cb 0x5624e91a2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc996800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc996800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58484/0 (socket says 172.21.15.63:58484) 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.313+0000 7fc996800700 1 -- 172.21.15.63:0/958367659 learned_addr learned my addr 172.21.15.63:0/958367659 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc995400700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af070 0x5624e92b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc995e00700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc996800700 1 -- 172.21.15.63:0/958367659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af070 msgr2=0x5624e92b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc996800700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af070 0x5624e92b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc996800700 1 -- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 msgr2=0x5624e92b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc996800700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc996800700 1 -- 172.21.15.63:0/958367659 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624e91c1520 con 0x5624e92af990 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc995400700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af070 0x5624e92b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc996800700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92b2f50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc98c01ce40 tx=0x7fc98c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571d50ce2ed46726 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.314+0000 7fc973e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.315+0000 7fc973e00700 1 -- 172.21.15.63:0/958367659 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c029060 con 0x5624e92af990 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.315+0000 7fc973e00700 1 -- 172.21.15.63:0/958367659 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7fc98c01c250 con 0x5624e92af990 2024-05-11T13:40:48.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.315+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9800b24e0 con 0x5624e92af990 2024-05-11T13:40:48.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.315+0000 7fc973e00700 1 -- 172.21.15.63:0/958367659 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc98c030050 con 0x5624e92af990 2024-05-11T13:40:48.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.315+0000 7fc9a31c5b80 10 client.15798 fetch_fsmap learned FSMap version 50 2024-05-11T13:40:48.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.315+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5624e92c9540 con 0x5624e92af990 2024-05-11T13:40:48.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc973e00700 1 -- 172.21.15.63:0/958367659 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9800b24e0 con 0x5624e92af990 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc973e00700 1 -- 172.21.15.63:0/958367659 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fc98c023030 con 0x5624e92af990 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc9a31c5b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 50 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc9a31c5b80 10 client.15798 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc9a31c5b80 20 client.15798 populate_metadata read hostname 'smithi063' 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e92cbb80 0x5624e92cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc9a31c5b80 4 client.15798 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.316+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624e92762b0 con 0x5624e92cbb80 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.317+0000 7fc995e00700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e92cbb80 0x5624e92cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.318+0000 7fc995e00700 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e92cbb80 0x5624e92cdf70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46d7283493f7cd7 server_cookie=a051e63cc08034a0 in_seq=0 out_seq=0 2024-05-11T13:40:48.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.318+0000 7fc973e00700 10 client.15798 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:48.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.319+0000 7fc973e00700 1 -- 172.21.15.63:0/958367659 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624e92762b0 con 0x5624e92cbb80 2024-05-11T13:40:48.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.319+0000 7fc973e00700 10 client.15798 handle_command_reply: tid=0 2024-05-11T13:40:48.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.319+0000 7fc9a31c5b80 1 client.15798 shutdown 2024-05-11T13:40:48.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 msgr2=0x5624e92b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92b2f50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc98c01ce40 tx=0x7fc98c004b40 comp rx=0 tx=0).stop 2024-05-11T13:40:48.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 shutdown_connections 2024-05-11T13:40:48.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92b2f50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e92cbb80 0x5624e92cdf70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/958367659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af070 0x5624e92b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 >> 172.21.15.63:0/958367659 conn(0x5624e91d7c60 msgr2=0x5624e92a8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc973e00700 0 client.15798 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 shutdown_connections 2024-05-11T13:40:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/958367659 wait complete. 2024-05-11T13:40:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.320+0000 7fc9a31c5b80 20 client.15798 trim_cache size 0 max 16384 2024-05-11T13:40:48.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.342+0000 7fc9a31c5b80 1 Processor -- start 2024-05-11T13:40:48.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc9a31c5b80 1 -- start start 2024-05-11T13:40:48.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901ad270 con 0x5624e92af990 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901aedb0 con 0x5624e92af070 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901ad940 con 0x5624e92a9bc0 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.343+0000 7fc996800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92d38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58490/0 (socket says 172.21.15.63:58490) 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 -- 172.21.15.63:0/3062128937 learned_addr learned my addr 172.21.15.63:0/3062128937 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:48.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc995400700 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc995e00700 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 -- 172.21.15.63:0/3062128937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 msgr2=0x5624e92b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 -- 172.21.15.63:0/3062128937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 msgr2=0x5624e92ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 -- 172.21.15.63:0/3062128937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e92b0f80 con 0x5624e92af990 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.344+0000 7fc996800700 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92d38d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0092a0 tx=0x7fc98c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb93724e8c20f9f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.345+0000 7fc97ea00700 1 -- 172.21.15.63:0/3062128937 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c029050 con 0x5624e92af990 2024-05-11T13:40:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.345+0000 7fc97ea00700 1 -- 172.21.15.63:0/3062128937 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98c013040 con 0x5624e92af990 2024-05-11T13:40:48.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.345+0000 7fc97ea00700 1 -- 172.21.15.63:0/3062128937 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c019e60 con 0x5624e92af990 2024-05-11T13:40:48.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3062128937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 msgr2=0x5624e92d38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92d38d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0092a0 tx=0x7fc98c0092d0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3062128937 shutdown_connections 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af990 0x5624e92d38d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92a9bc0 0x5624e92b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/3062128937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92af070 0x5624e92ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3062128937 >> 172.21.15.63:0/3062128937 conn(0x5624e91d7c60 msgr2=0x5624e92acaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3062128937 shutdown_connections 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/3062128937 wait complete. 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.346+0000 7fc9a31c5b80 1 -- start start 2024-05-11T13:40:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 1 Processor -- start 2024-05-11T13:40:48.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92c1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af070 0x5624e92c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af990 0x5624e92c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901ae6e0 con 0x5624e92af070 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901ae010 con 0x5624e92a9bc0 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901ab230 con 0x5624e92af990 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc9a31c5b80 10 client.0 _ll_register_callbacks cb 0x5624e91a2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af070 0x5624e92c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af070 0x5624e92c1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58504/0 (socket says 172.21.15.63:58504) 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.347+0000 7fc995400700 1 -- 172.21.15.63:0/1145487021 learned_addr learned my addr 172.21.15.63:0/1145487021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995e00700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc996800700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af990 0x5624e92c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995400700 1 -- 172.21.15.63:0/1145487021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af990 msgr2=0x5624e92c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995400700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af990 0x5624e92c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995400700 1 -- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 msgr2=0x5624e92c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:40:48.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995400700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92c1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995400700 1 -- 172.21.15.63:0/1145487021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624e93364c0 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc996800700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af990 0x5624e92c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995e00700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc995400700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af070 0x5624e92c1690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc98000dfd0 tx=0x7fc980004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22eb26e9ece3631c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc973e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.348+0000 7fc973e00700 1 -- 172.21.15.63:0/1145487021 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800b24e0 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc973e00700 1 -- 172.21.15.63:0/1145487021 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9525+0+0 (secure 0 0 0) 0x7fc9800a98e0 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c013040 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc973e00700 1 -- 172.21.15.63:0/1145487021 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc9800ada30 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 10 client.15804 fetch_fsmap learned FSMap version 50 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5624e9337810 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc973e00700 1 -- 172.21.15.63:0/1145487021 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98c013040 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc973e00700 1 -- 172.21.15.63:0/1145487021 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fc98008fba0 con 0x5624e92af070 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 50 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 10 client.15804 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 20 client.15804 populate_metadata read hostname 'smithi063' 2024-05-11T13:40:48.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e9337af0 0x5624e9339ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:40:48.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 4 client.15804 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d73159a2", "release": true, "await": true}] 2024-05-11T13:40:48.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.349+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d73159a2", "release": true, "await": true}) -- 0x5624e9209500 con 0x5624e9337af0 2024-05-11T13:40:48.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.350+0000 7fc995e00700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e9337af0 0x5624e9339ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:40:48.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.351+0000 7fc995e00700 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e9337af0 0x5624e9339ee0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5d68e589af8bad server_cookie=6d5e7cef77406429 in_seq=0 out_seq=0 2024-05-11T13:40:48.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.351+0000 7fc973e00700 10 client.15804 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.403+0000 7fc973e00700 1 -- 172.21.15.63:0/1145487021 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5624e9209500 con 0x5624e9337af0 2024-05-11T13:40:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.403+0000 7fc973e00700 10 client.15804 handle_command_reply: tid=0 2024-05-11T13:40:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.403+0000 7fc9a31c5b80 1 client.15804 shutdown 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af070 msgr2=0x5624e92c1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af070 0x5624e92c1690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc98000dfd0 tx=0x7fc980004e60 comp rx=0 tx=0).stop 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 shutdown_connections 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624e92af990 0x5624e92c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5624e9337af0 0x5624e9339ee0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5624e92a9bc0 0x5624e92c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 --2- 172.21.15.63:0/1145487021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624e92af070 0x5624e92c1690 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 >> 172.21.15.63:0/1145487021 conn(0x5624e91d7c60 msgr2=0x5624e92afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.404+0000 7fc973e00700 0 client.15804 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.405+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 shutdown_connections 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.405+0000 7fc9a31c5b80 1 -- 172.21.15.63:0/1145487021 wait complete. 2024-05-11T13:40:48.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.405+0000 7fc9a31c5b80 20 client.15804 trim_cache size 0 max 16384 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 50, 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 24, 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout: "d73159a2": { 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 24, 2024-05-11T13:40:48.399 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 80.8, 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:40:48.400 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:40:48.401 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.406+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc9780b29f0 msgr2=0x7fc9780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.406+0000 7fc997200700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc9780b29f0 0x7fc9780b4eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009710 tx=0x7fc98c009f90 comp rx=0 tx=0).stop 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.406+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990102f90 msgr2=0x7fc9901a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.406+0000 7fc997200700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990102f90 0x7fc9901a1e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d950 tx=0x7fc98000dc60 comp rx=0 tx=0).stop 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 shutdown_connections 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc9780b29f0 0x7fc9780b4eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9901076c0 0x7fc99019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9901025e0 0x7fc9901a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 --2- 172.21.15.63:0/271362505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990102f90 0x7fc9901a1e70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 >> 172.21.15.63:0/271362505 conn(0x7fc9900fde70 msgr2=0x7fc990100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 shutdown_connections 2024-05-11T13:40:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:40:48.407+0000 7fc997200700 1 -- 172.21.15.63:0/271362505 wait complete. 2024-05-11T13:40:49.272 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd73159a2', seconds elapsed: 37.0 2024-05-11T13:40:49.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:49 smithi165 ceph-mon[33235]: pgmap v213: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-11T13:40:49.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:49 smithi063 ceph-mon[29809]: pgmap v213: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-11T13:40:49.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:49 smithi071 ceph-mon[33293]: pgmap v213: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-11T13:40:51.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:51 smithi165 ceph-mon[33235]: pgmap v214: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-05-11T13:40:51.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:51 smithi063 ceph-mon[29809]: pgmap v214: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-05-11T13:40:51.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:51 smithi071 ceph-mon[33293]: pgmap v214: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-05-11T13:40:52.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:52 smithi165 ceph-mon[33235]: pgmap v215: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-05-11T13:40:52.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:52 smithi063 ceph-mon[29809]: pgmap v215: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-05-11T13:40:52.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:52 smithi071 ceph-mon[33293]: pgmap v215: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-05-11T13:40:55.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:55 smithi063 ceph-mon[29809]: pgmap v216: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-11T13:40:55.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:55 smithi165 ceph-mon[33235]: pgmap v216: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-11T13:40:55.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:55 smithi071 ceph-mon[33293]: pgmap v216: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-11T13:40:57.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:57 smithi165 ceph-mon[33235]: pgmap v217: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 20 op/s 2024-05-11T13:40:57.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:57 smithi063 ceph-mon[29809]: pgmap v217: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 20 op/s 2024-05-11T13:40:57.692 INFO:teuthology.orchestra.run.smithi063.stderr:creating primary keys... 2024-05-11T13:40:57.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:57 smithi071 ceph-mon[33293]: pgmap v217: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 20 op/s 2024-05-11T13:40:59.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:40:59 smithi165 ceph-mon[33235]: pgmap v218: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.8 MiB/s wr, 32 op/s 2024-05-11T13:40:59.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:40:59 smithi063 ceph-mon[29809]: pgmap v218: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.8 MiB/s wr, 32 op/s 2024-05-11T13:40:59.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:40:59 smithi071 ceph-mon[33293]: pgmap v218: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.8 MiB/s wr, 32 op/s 2024-05-11T13:41:00.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:00 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:00.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:00 smithi165 ceph-mon[33235]: pgmap v219: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.8 MiB/s wr, 27 op/s 2024-05-11T13:41:00.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:00 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:00.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:00 smithi063 ceph-mon[29809]: pgmap v219: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.8 MiB/s wr, 27 op/s 2024-05-11T13:41:00.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:00 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:00.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:00 smithi071 ceph-mon[33293]: pgmap v219: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.8 MiB/s wr, 27 op/s 2024-05-11T13:41:03.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:03 smithi165 ceph-mon[33235]: pgmap v220: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.8 MiB/s wr, 27 op/s 2024-05-11T13:41:03.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:03 smithi063 ceph-mon[29809]: pgmap v220: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.8 MiB/s wr, 27 op/s 2024-05-11T13:41:03.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:03 smithi071 ceph-mon[33293]: pgmap v220: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.8 MiB/s wr, 27 op/s 2024-05-11T13:41:05.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:05 smithi165 ceph-mon[33235]: pgmap v221: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-05-11T13:41:05.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:05 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:05.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:05 smithi063 ceph-mon[29809]: pgmap v221: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-05-11T13:41:05.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:05 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:05.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:05 smithi071 ceph-mon[33293]: pgmap v221: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 55 op/s 2024-05-11T13:41:05.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:05 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:07.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:07 smithi165 ceph-mon[33235]: pgmap v222: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.8 MiB/s wr, 39 op/s 2024-05-11T13:41:07.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:07 smithi063 ceph-mon[29809]: pgmap v222: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.8 MiB/s wr, 39 op/s 2024-05-11T13:41:07.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:07 smithi071 ceph-mon[33293]: pgmap v222: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.8 MiB/s wr, 39 op/s 2024-05-11T13:41:08.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:08 smithi165 ceph-mon[33235]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 13 MiB/s wr, 48 op/s 2024-05-11T13:41:08.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:08 smithi063 ceph-mon[29809]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 13 MiB/s wr, 48 op/s 2024-05-11T13:41:08.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:08 smithi071 ceph-mon[33293]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 13 MiB/s wr, 48 op/s 2024-05-11T13:41:10.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:10 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:10.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:10 smithi165 ceph-mon[33235]: osdmap e94: 12 total, 12 up, 12 in 2024-05-11T13:41:10.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:10 smithi165 ceph-mon[33235]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 44 op/s 2024-05-11T13:41:10.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:10 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:10.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:10 smithi063 ceph-mon[29809]: osdmap e94: 12 total, 12 up, 12 in 2024-05-11T13:41:10.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:10 smithi063 ceph-mon[29809]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 44 op/s 2024-05-11T13:41:10.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:10 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:10.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:10 smithi071 ceph-mon[33293]: osdmap e94: 12 total, 12 up, 12 in 2024-05-11T13:41:10.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:10 smithi071 ceph-mon[33293]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 44 op/s 2024-05-11T13:41:13.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:13 smithi165 ceph-mon[33235]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 44 op/s 2024-05-11T13:41:13.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:13 smithi063 ceph-mon[29809]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 44 op/s 2024-05-11T13:41:13.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:13 smithi071 ceph-mon[33293]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 44 op/s 2024-05-11T13:41:15.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:15 smithi165 ceph-mon[33235]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.9 MiB/s wr, 24 op/s 2024-05-11T13:41:15.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:15 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:15.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:15 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:41:15.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:15 smithi165 ceph-mon[33235]: osdmap e95: 12 total, 12 up, 12 in 2024-05-11T13:41:15.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:15 smithi063 ceph-mon[29809]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.9 MiB/s wr, 24 op/s 2024-05-11T13:41:15.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:15 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:15.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:15 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:41:15.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:15 smithi063 ceph-mon[29809]: osdmap e95: 12 total, 12 up, 12 in 2024-05-11T13:41:15.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:15 smithi071 ceph-mon[33293]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.9 MiB/s wr, 24 op/s 2024-05-11T13:41:15.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:15 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:15.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:15 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:41:15.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:15 smithi071 ceph-mon[33293]: osdmap e95: 12 total, 12 up, 12 in 2024-05-11T13:41:17.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:17 smithi165 ceph-mon[33235]: pgmap v229: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.6 MiB/s wr, 16 op/s 2024-05-11T13:41:17.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:17 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:41:17.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:17 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:41:17.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:17 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:41:17.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:17 smithi063 ceph-mon[29809]: pgmap v229: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.6 MiB/s wr, 16 op/s 2024-05-11T13:41:17.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:17 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:41:17.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:17 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:41:17.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:17 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:41:17.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:17 smithi071 ceph-mon[33293]: pgmap v229: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.6 MiB/s wr, 16 op/s 2024-05-11T13:41:17.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:17 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:41:17.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:17 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:41:17.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:17 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:41:18.562 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:18 smithi165 ceph-mon[33235]: pgmap v230: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 11 MiB/s wr, 29 op/s 2024-05-11T13:41:18.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:18 smithi063 ceph-mon[29809]: pgmap v230: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 11 MiB/s wr, 29 op/s 2024-05-11T13:41:18.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:18 smithi071 ceph-mon[33293]: pgmap v230: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 11 MiB/s wr, 29 op/s 2024-05-11T13:41:20.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:20 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:20.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:20 smithi063 ceph-mon[29809]: pgmap v231: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-05-11T13:41:20.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:20 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:20.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:20 smithi071 ceph-mon[33293]: pgmap v231: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-05-11T13:41:20.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:20 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:20.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:20 smithi165 ceph-mon[33235]: pgmap v231: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-05-11T13:41:23.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:23 smithi165 ceph-mon[33235]: pgmap v232: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-05-11T13:41:23.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:23 smithi063 ceph-mon[29809]: pgmap v232: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-05-11T13:41:23.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:23 smithi071 ceph-mon[33293]: pgmap v232: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-05-11T13:41:25.505 INFO:teuthology.orchestra.run.smithi063.stderr:done in 347.80 s (drop tables 0.05 s, create tables 0.35 s, client-side generate 224.89 s, vacuum 94.70 s, primary keys 27.81 s). 2024-05-11T13:41:25.509 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:25 smithi063 ceph-mon[29809]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-11T13:41:25.517 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-11T13:41:25.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:25 smithi165 ceph-mon[33235]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-11T13:41:25.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:25 smithi071 ceph-mon[33293]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-11T13:41:26.143 INFO:teuthology.orchestra.run.smithi063.stderr:starting vacuum...end. 2024-05-11T13:41:27.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:27 smithi165 ceph-mon[33235]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.9 MiB/s wr, 25 op/s 2024-05-11T13:41:27.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:27 smithi063 ceph-mon[29809]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.9 MiB/s wr, 25 op/s 2024-05-11T13:41:27.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:27 smithi071 ceph-mon[33293]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.9 MiB/s wr, 25 op/s 2024-05-11T13:41:28.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:28 smithi165 ceph-mon[33235]: pgmap v235: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 41 op/s 2024-05-11T13:41:28.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:28 smithi063 ceph-mon[29809]: pgmap v235: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 41 op/s 2024-05-11T13:41:28.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:28 smithi071 ceph-mon[33293]: pgmap v235: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 41 op/s 2024-05-11T13:41:31.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 5.0 s, 272.8 tps, lat 325.106 ms stddev 669.241 2024-05-11T13:41:31.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:31 smithi165 ceph-mon[33235]: pgmap v236: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 30 op/s 2024-05-11T13:41:31.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:31 smithi063 ceph-mon[29809]: pgmap v236: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 30 op/s 2024-05-11T13:41:31.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:31 smithi071 ceph-mon[33293]: pgmap v236: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 30 op/s 2024-05-11T13:41:33.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:33 smithi071 ceph-mon[33293]: pgmap v237: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 30 op/s 2024-05-11T13:41:33.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:33 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:33.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:33 smithi165 ceph-mon[33235]: pgmap v237: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 30 op/s 2024-05-11T13:41:33.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:33 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:33.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:33 smithi063 ceph-mon[29809]: pgmap v237: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 30 op/s 2024-05-11T13:41:33.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:33 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:41:34.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:34 smithi165 ceph-mon[33235]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 60 op/s 2024-05-11T13:41:34.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:34 smithi063 ceph-mon[29809]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 60 op/s 2024-05-11T13:41:34.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:34 smithi071 ceph-mon[33293]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 60 op/s 2024-05-11T13:41:36.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 10.0 s, 542.4 tps, lat 150.264 ms stddev 374.914 2024-05-11T13:41:37.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:37 smithi165 ceph-mon[33235]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-11T13:41:37.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:37 smithi063 ceph-mon[29809]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-11T13:41:37.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:37 smithi071 ceph-mon[33293]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-11T13:41:39.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:39 smithi165 ceph-mon[33235]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 86 op/s 2024-05-11T13:41:39.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:39 smithi063 ceph-mon[29809]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 86 op/s 2024-05-11T13:41:39.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:39 smithi071 ceph-mon[33293]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 86 op/s 2024-05-11T13:41:40.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:40 smithi165 ceph-mon[33235]: pgmap v241: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 68 op/s 2024-05-11T13:41:40.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:40 smithi063 ceph-mon[29809]: pgmap v241: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 68 op/s 2024-05-11T13:41:40.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:40 smithi071 ceph-mon[33293]: pgmap v241: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 68 op/s 2024-05-11T13:41:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 15.0 s, 422.6 tps, lat 188.754 ms stddev 407.718 2024-05-11T13:41:43.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:43 smithi165 ceph-mon[33235]: pgmap v242: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 68 op/s 2024-05-11T13:41:43.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:43 smithi063 ceph-mon[29809]: pgmap v242: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 68 op/s 2024-05-11T13:41:43.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:43 smithi071 ceph-mon[33293]: pgmap v242: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.9 MiB/s wr, 68 op/s 2024-05-11T13:41:45.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:45 smithi165 ceph-mon[33235]: pgmap v243: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 113 op/s 2024-05-11T13:41:45.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:45 smithi063 ceph-mon[29809]: pgmap v243: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 113 op/s 2024-05-11T13:41:45.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:45 smithi071 ceph-mon[33293]: pgmap v243: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 113 op/s 2024-05-11T13:41:46.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 20.0 s, 746.5 tps, lat 185.582 ms stddev 449.265 2024-05-11T13:41:46.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:46 smithi165 ceph-mon[33235]: pgmap v244: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.0 MiB/s wr, 83 op/s 2024-05-11T13:41:46.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:46 smithi063 ceph-mon[29809]: pgmap v244: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.0 MiB/s wr, 83 op/s 2024-05-11T13:41:46.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:46 smithi071 ceph-mon[33293]: pgmap v244: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.0 MiB/s wr, 83 op/s 2024-05-11T13:41:49.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:49 smithi165 ceph-mon[33235]: pgmap v245: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 174 op/s 2024-05-11T13:41:49.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:49 smithi063 ceph-mon[29809]: pgmap v245: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 174 op/s 2024-05-11T13:41:49.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:49 smithi071 ceph-mon[33293]: pgmap v245: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 174 op/s 2024-05-11T13:41:51.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 25.0 s, 646.2 tps, lat 141.999 ms stddev 219.148 2024-05-11T13:41:51.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:51 smithi165 ceph-mon[33235]: pgmap v246: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 136 op/s 2024-05-11T13:41:51.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:51 smithi063 ceph-mon[29809]: pgmap v246: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 136 op/s 2024-05-11T13:41:51.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:51 smithi071 ceph-mon[33293]: pgmap v246: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 136 op/s 2024-05-11T13:41:52.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:52 smithi063 ceph-mon[29809]: pgmap v247: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 136 op/s 2024-05-11T13:41:52.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:52 smithi071 ceph-mon[33293]: pgmap v247: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 136 op/s 2024-05-11T13:41:52.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:52 smithi165 ceph-mon[33235]: pgmap v247: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 136 op/s 2024-05-11T13:41:55.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:55 smithi165 ceph-mon[33235]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 14 MiB/s wr, 429 op/s 2024-05-11T13:41:55.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:55 smithi063 ceph-mon[29809]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 14 MiB/s wr, 429 op/s 2024-05-11T13:41:55.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:55 smithi071 ceph-mon[33293]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 14 MiB/s wr, 429 op/s 2024-05-11T13:41:56.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 30.0 s, 153.7 tps, lat 250.485 ms stddev 486.750 2024-05-11T13:41:56.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:56 smithi165 ceph-mon[33235]: pgmap v249: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 11 MiB/s wr, 384 op/s 2024-05-11T13:41:56.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:56 smithi063 ceph-mon[29809]: pgmap v249: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 11 MiB/s wr, 384 op/s 2024-05-11T13:41:56.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:56 smithi071 ceph-mon[33293]: pgmap v249: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 11 MiB/s wr, 384 op/s 2024-05-11T13:41:59.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:41:59 smithi165 ceph-mon[33235]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 11 MiB/s wr, 394 op/s 2024-05-11T13:41:59.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:41:59 smithi063 ceph-mon[29809]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 11 MiB/s wr, 394 op/s 2024-05-11T13:41:59.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:41:59 smithi071 ceph-mon[33293]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 11 MiB/s wr, 394 op/s 2024-05-11T13:42:00.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:00 smithi063 ceph-mon[29809]: pgmap v251: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-05-11T13:42:00.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:00 smithi071 ceph-mon[33293]: pgmap v251: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-05-11T13:42:00.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:00 smithi165 ceph-mon[33235]: pgmap v251: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-05-11T13:42:01.148 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 35.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:02.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:02 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:42:02.810 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:02 smithi165 ceph-mon[33235]: pgmap v252: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-05-11T13:42:02.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:02 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:42:02.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:02 smithi063 ceph-mon[29809]: pgmap v252: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-05-11T13:42:02.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:02 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:42:02.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:02 smithi071 ceph-mon[33293]: pgmap v252: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 303 op/s 2024-05-11T13:42:05.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:05 smithi063 ceph-mon[29809]: osdmap e96: 12 total, 12 up, 12 in 2024-05-11T13:42:05.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:05 smithi063 ceph-mon[29809]: pgmap v254: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-11T13:42:05.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:05 smithi071 ceph-mon[33293]: osdmap e96: 12 total, 12 up, 12 in 2024-05-11T13:42:05.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:05 smithi071 ceph-mon[33293]: pgmap v254: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-11T13:42:05.513 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:05 smithi165 ceph-mon[33235]: osdmap e96: 12 total, 12 up, 12 in 2024-05-11T13:42:05.513 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:05 smithi165 ceph-mon[33235]: pgmap v254: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-11T13:42:06.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 40.0 s, 584.4 tps, lat 461.610 ms stddev 1646.593 2024-05-11T13:42:06.716 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.8 2024-05-11T13:42:06.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:42:07.418 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:42:07.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:07 smithi165 ceph-mon[33235]: pgmap v255: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-11T13:42:07.591 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:07 smithi063 ceph-mon[29809]: pgmap v255: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-11T13:42:07.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:07 smithi071 ceph-mon[33293]: pgmap v255: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-11T13:42:08.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:08 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:42:08.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:08 smithi063 ceph-mon[29809]: pgmap v256: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.2 MiB/s wr, 520 op/s 2024-05-11T13:42:08.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:08 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:42:08.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:08 smithi165 ceph-mon[33235]: pgmap v256: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.2 MiB/s wr, 520 op/s 2024-05-11T13:42:08.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:08 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:42:08.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:08 smithi071 ceph-mon[33293]: pgmap v256: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.2 MiB/s wr, 520 op/s 2024-05-11T13:42:10.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.096+0000 7f73a0200700 1 -- 172.21.15.63:0/3453435485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981095c0 msgr2=0x7f73981099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:10.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.096+0000 7f73a0200700 1 --2- 172.21.15.63:0/3453435485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981095c0 0x7f73981099a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f738c0098f0 tx=0x7f738c009c00 comp rx=0 tx=0).stop 2024-05-11T13:42:10.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.097+0000 7f73a0200700 1 -- 172.21.15.63:0/3453435485 shutdown_connections 2024-05-11T13:42:10.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.097+0000 7f73a0200700 1 --2- 172.21.15.63:0/3453435485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f739806bbf0 0x7f73981103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.097+0000 7f73a0200700 1 --2- 172.21.15.63:0/3453435485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981095c0 0x7f73981099a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.097+0000 7f73a0200700 1 --2- 172.21.15.63:0/3453435485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 0x7f739806b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.097+0000 7f73a0200700 1 -- 172.21.15.63:0/3453435485 >> 172.21.15.63:0/3453435485 conn(0x7f7398079400 msgr2=0x7f7398079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:10.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.097+0000 7f73a0200700 1 -- 172.21.15.63:0/3453435485 shutdown_connections 2024-05-11T13:42:10.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.098+0000 7f73a0200700 1 -- 172.21.15.63:0/3453435485 wait complete. 2024-05-11T13:42:10.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.099+0000 7f73a0200700 1 Processor -- start 2024-05-11T13:42:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.100+0000 7f73a0200700 1 -- start start 2024-05-11T13:42:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.100+0000 7f73a0200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 0x7f739819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.100+0000 7f73a0200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f739806bbf0 0x7f739819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:10.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.100+0000 7f73a0200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73981095c0 0x7f73981a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:10.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.100+0000 7f73a0200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73981135d0 con 0x7f739806b230 2024-05-11T13:42:10.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.100+0000 7f73a0200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7398113450 con 0x7f73981095c0 2024-05-11T13:42:10.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.100+0000 7f73a0200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7398113750 con 0x7f739806bbf0 2024-05-11T13:42:10.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739d800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 0x7f739819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739d800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 0x7f739819bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35444/0 (socket says 172.21.15.63:35444) 2024-05-11T13:42:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739d800700 1 -- 172.21.15.63:0/2268537034 learned_addr learned my addr 172.21.15.63:0/2268537034 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739ce00700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f739806bbf0 0x7f739819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739e200700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73981095c0 0x7f73981a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739d800700 1 -- 172.21.15.63:0/2268537034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f739806bbf0 msgr2=0x7f739819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:10.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739d800700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f739806bbf0 0x7f739819c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739d800700 1 -- 172.21.15.63:0/2268537034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73981095c0 msgr2=0x7f73981a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:10.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.101+0000 7f739d800700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73981095c0 0x7f73981a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.102+0000 7f739d800700 1 -- 172.21.15.63:0/2268537034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73981a0df0 con 0x7f739806b230 2024-05-11T13:42:10.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.102+0000 7f739d800700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 0x7f739819bde0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f738c005950 tx=0x7f738c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:10.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.103+0000 7f7396000700 1 -- 172.21.15.63:0/2268537034 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c00b8c0 con 0x7f739806b230 2024-05-11T13:42:10.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.104+0000 7f7396000700 1 -- 172.21.15.63:0/2268537034 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f738c00ba60 con 0x7f739806b230 2024-05-11T13:42:10.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.104+0000 7f7396000700 1 -- 172.21.15.63:0/2268537034 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c003ba0 con 0x7f739806b230 2024-05-11T13:42:10.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.104+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73981a1080 con 0x7f739806b230 2024-05-11T13:42:10.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.104+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981a8960 con 0x7f739806b230 2024-05-11T13:42:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.107+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7378005330 con 0x7f739806b230 2024-05-11T13:42:10.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.111+0000 7f7396000700 1 -- 172.21.15.63:0/2268537034 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f738c00bc10 con 0x7f739806b230 2024-05-11T13:42:10.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.115+0000 7f7396000700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f73800b2b40 0x7f73800b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:10.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.115+0000 7f7396000700 1 -- 172.21.15.63:0/2268537034 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f738c0e39e0 con 0x7f739806b230 2024-05-11T13:42:10.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.116+0000 7f739ce00700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f73800b2b40 0x7f73800b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:10.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.118+0000 7f739ce00700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f73800b2b40 0x7f73800b5000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f739810faf0 tx=0x7f7384009310 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.124+0000 7f7396000700 1 -- 172.21.15.63:0/2268537034 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f738c013030 con 0x7f739806b230 2024-05-11T13:42:10.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.595+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73780058f0 con 0x7f739806b230 2024-05-11T13:42:10.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.596+0000 7f7396000700 1 -- 172.21.15.63:0/2268537034 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11956 (secure 0 0 0) 0x7f738c0ab000 con 0x7f739806b230 2024-05-11T13:42:10.590 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:42:10.590 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:42:06.467762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:42:10.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.600+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f73800b2b40 msgr2=0x7f73800b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:10.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.600+0000 7f73a0200700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f73800b2b40 0x7f73800b5000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f739810faf0 tx=0x7f7384009310 comp rx=0 tx=0).stop 2024-05-11T13:42:10.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.600+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 msgr2=0x7f739819bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:10.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.600+0000 7f73a0200700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 0x7f739819bde0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f738c005950 tx=0x7f738c00b560 comp rx=0 tx=0).stop 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 shutdown_connections 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f73800b2b40 0x7f73800b5000 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73981095c0 0x7f73981a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f739806b230 0x7f739819bde0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 --2- 172.21.15.63:0/2268537034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f739806bbf0 0x7f739819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 >> 172.21.15.63:0/2268537034 conn(0x7f7398079400 msgr2=0x7f739810e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 shutdown_connections 2024-05-11T13:42:10.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:10.601+0000 7f73a0200700 1 -- 172.21.15.63:0/2268537034 wait complete. 2024-05-11T13:42:10.596 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 58 2024-05-11T13:42:11.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 45.0 s, 569.4 tps, lat 174.505 ms stddev 253.861 2024-05-11T13:42:11.308 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:11 smithi165 ceph-mon[33235]: osdmap e97: 12 total, 12 up, 12 in 2024-05-11T13:42:11.308 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:11 smithi165 ceph-mon[33235]: pgmap v258: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 10 MiB/s wr, 651 op/s 2024-05-11T13:42:11.308 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:11 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2268537034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:42:11.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:11 smithi063 ceph-mon[29809]: osdmap e97: 12 total, 12 up, 12 in 2024-05-11T13:42:11.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:11 smithi063 ceph-mon[29809]: pgmap v258: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 10 MiB/s wr, 651 op/s 2024-05-11T13:42:11.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:11 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2268537034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:42:11.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:11 smithi071 ceph-mon[33293]: osdmap e97: 12 total, 12 up, 12 in 2024-05-11T13:42:11.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:11 smithi071 ceph-mon[33293]: pgmap v258: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 10 MiB/s wr, 651 op/s 2024-05-11T13:42:11.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:11 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2268537034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:42:12.284 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 90.8 --await-for 42.0' 2024-05-11T13:42:12.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 90.8 --await-for 42.0 2024-05-11T13:42:13.045 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:42:13.483 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:13 smithi063 ceph-mon[29809]: pgmap v259: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 430 op/s 2024-05-11T13:42:13.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:13 smithi165 ceph-mon[33235]: pgmap v259: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 430 op/s 2024-05-11T13:42:13.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:13 smithi071 ceph-mon[33293]: pgmap v259: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 430 op/s 2024-05-11T13:42:14.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:14 smithi063 ceph-mon[29809]: pgmap v260: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 634 op/s 2024-05-11T13:42:14.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:14 smithi071 ceph-mon[33293]: pgmap v260: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 634 op/s 2024-05-11T13:42:14.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:14 smithi165 ceph-mon[33235]: pgmap v260: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 634 op/s 2024-05-11T13:42:15.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.730+0000 7fc57cc00700 1 -- 172.21.15.63:0/2889072434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc578075780 msgr2=0x7fc578075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:15.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.730+0000 7fc57cc00700 1 --2- 172.21.15.63:0/2889072434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc578075780 0x7fc578075b60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc568009990 tx=0x7fc568009ca0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.730+0000 7fc57cc00700 1 -- 172.21.15.63:0/2889072434 shutdown_connections 2024-05-11T13:42:15.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.730+0000 7fc57cc00700 1 --2- 172.21.15.63:0/2889072434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc57810ffb0 0x7fc5781123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.730+0000 7fc57cc00700 1 --2- 172.21.15.63:0/2889072434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc578075780 0x7fc578075b60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.730+0000 7fc57cc00700 1 --2- 172.21.15.63:0/2889072434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5780760a0 0x7fc57810fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.730+0000 7fc57cc00700 1 -- 172.21.15.63:0/2889072434 >> 172.21.15.63:0/2889072434 conn(0x7fc57807a6b0 msgr2=0x7fc57807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:15.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.731+0000 7fc57cc00700 1 -- 172.21.15.63:0/2889072434 shutdown_connections 2024-05-11T13:42:15.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.731+0000 7fc57cc00700 1 -- 172.21.15.63:0/2889072434 wait complete. 2024-05-11T13:42:15.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 Processor -- start 2024-05-11T13:42:15.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 -- start start 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc578075780 0x7fc5781a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5780760a0 0x7fc5781a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc57810ffb0 0x7fc57819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc578115870 con 0x7fc57810ffb0 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781156f0 con 0x7fc578075780 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.732+0000 7fc57cc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781159f0 con 0x7fc5780760a0 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc57810ffb0 0x7fc57819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc57810ffb0 0x7fc57819bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35458/0 (socket says 172.21.15.63:35458) 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 -- 172.21.15.63:0/601933480 learned_addr learned my addr 172.21.15.63:0/601933480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc575400700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5780760a0 0x7fc5781a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc575e00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc578075780 0x7fc5781a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 -- 172.21.15.63:0/601933480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5780760a0 msgr2=0x7fc5781a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5780760a0 0x7fc5781a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 -- 172.21.15.63:0/601933480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc578075780 msgr2=0x7fc5781a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:15.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc578075780 0x7fc5781a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.733+0000 7fc576800700 1 -- 172.21.15.63:0/601933480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57819c3f0 con 0x7fc57810ffb0 2024-05-11T13:42:15.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.734+0000 7fc576800700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc57810ffb0 0x7fc57819bb10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc57000efc0 tx=0x7fc57000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:15.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.734+0000 7fc56ea00700 1 -- 172.21.15.63:0/601933480 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5700092e0 con 0x7fc57810ffb0 2024-05-11T13:42:15.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.734+0000 7fc56ea00700 1 -- 172.21.15.63:0/601933480 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc570004bb0 con 0x7fc57810ffb0 2024-05-11T13:42:15.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.734+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57819c680 con 0x7fc57810ffb0 2024-05-11T13:42:15.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.734+0000 7fc56ea00700 1 -- 172.21.15.63:0/601933480 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570004020 con 0x7fc57810ffb0 2024-05-11T13:42:15.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.735+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5781a89c0 con 0x7fc57810ffb0 2024-05-11T13:42:15.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.740+0000 7fc56ea00700 1 -- 172.21.15.63:0/601933480 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc57001b030 con 0x7fc57810ffb0 2024-05-11T13:42:15.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.740+0000 7fc587581b80 1 Processor -- start 2024-05-11T13:42:15.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc587581b80 1 -- start start 2024-05-11T13:42:15.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc587581b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecafcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc587581b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc587581b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc587581b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55baeca40bb0 con 0x55baecafc870 2024-05-11T13:42:15.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc587581b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55baec9de8f0 con 0x55baecaf6e40 2024-05-11T13:42:15.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc587581b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55baeca3cb70 con 0x55baecaf7800 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.741+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecafcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecafcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35474/0 (socket says 172.21.15.63:35474) 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 -- 172.21.15.63:0/2798852495 learned_addr learned my addr 172.21.15.63:0/2798852495 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575e00700 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc56ea00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc55c0b2aa0 0x7fc55c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 -- 172.21.15.63:0/2798852495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 msgr2=0x55baecb03630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 -- 172.21.15.63:0/2798852495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 msgr2=0x55baecaf72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.742+0000 7fc575400700 1 -- 172.21.15.63:0/2798852495 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baec98da60 con 0x55baecafc870 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.743+0000 7fc576800700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc55c0b2aa0 0x7fc55c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.743+0000 7fc56ea00700 1 -- 172.21.15.63:0/601933480 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fc5700e2400 con 0x7fc57810ffb0 2024-05-11T13:42:15.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.743+0000 7fc575400700 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecafcc50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc564009fc0 tx=0x7fc5640098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41020f38d898ae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.743+0000 7fc563400700 1 -- 172.21.15.63:0/2798852495 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564004a30 con 0x55baecafc870 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.743+0000 7fc563400700 1 -- 172.21.15.63:0/2798852495 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5640054b0 con 0x55baecafc870 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.743+0000 7fc563400700 1 -- 172.21.15.63:0/2798852495 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640108c0 con 0x55baecafc870 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.744+0000 7fc587581b80 1 -- 172.21.15.63:0/2798852495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 msgr2=0x55baecafcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.744+0000 7fc587581b80 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecafcc50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc564009fc0 tx=0x7fc5640098c0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.744+0000 7fc576800700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc55c0b2aa0 0x7fc55c0b4f60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5700a5360 tx=0x7fc57006e7d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.744+0000 7fc587581b80 1 -- 172.21.15.63:0/2798852495 shutdown_connections 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.745+0000 7fc587581b80 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.745+0000 7fc587581b80 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecafcc50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.745+0000 7fc587581b80 1 --2- 172.21.15.63:0/2798852495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.745+0000 7fc587581b80 1 -- 172.21.15.63:0/2798852495 >> 172.21.15.63:0/2798852495 conn(0x55baeca1ccf0 msgr2=0x55baecae8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.745+0000 7fc587581b80 1 -- 172.21.15.63:0/2798852495 shutdown_connections 2024-05-11T13:42:15.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.745+0000 7fc587581b80 1 -- 172.21.15.63:0/2798852495 wait complete. 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.745+0000 7fc587581b80 1 -- start start 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 1 Processor -- start 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb00640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecafc870 0x55baecb00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55baecb03c30 con 0x55baecaf6e40 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55baec9a6cb0 con 0x55baecafc870 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55baecb03db0 con 0x55baecaf7800 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.746+0000 7fc587581b80 10 client.0 _ll_register_callbacks cb 0x55baec9e7e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35484/0 (socket says 172.21.15.63:35484) 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 -- 172.21.15.63:0/3709167466 learned_addr learned my addr 172.21.15.63:0/3709167466 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575e00700 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb00640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 -- 172.21.15.63:0/3709167466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 msgr2=0x55baecb00640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb00640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 -- 172.21.15.63:0/3709167466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecafc870 msgr2=0x55baecb00b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecafc870 0x55baecb00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 -- 172.21.15.63:0/3709167466 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baeca065b0 con 0x55baecaf6e40 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575e00700 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb00640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc575400700 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb032a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc5640097a0 tx=0x7fc564005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826a3e9b90c1fcc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.747+0000 7fc557e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.748+0000 7fc557e00700 1 -- 172.21.15.63:0/3709167466 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564004c60 con 0x55baecaf6e40 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.748+0000 7fc557e00700 1 -- 172.21.15.63:0/3709167466 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fc564010500 con 0x55baecaf6e40 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.748+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5640054b0 con 0x55baecaf6e40 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.748+0000 7fc557e00700 1 -- 172.21.15.63:0/3709167466 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc564031050 con 0x55baecaf6e40 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.748+0000 7fc587581b80 10 client.15822 fetch_fsmap learned FSMap version 58 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.748+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55baecb0e490 con 0x55baecaf6e40 2024-05-11T13:42:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc557e00700 1 -- 172.21.15.63:0/3709167466 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5640054b0 con 0x55baecaf6e40 2024-05-11T13:42:15.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc557e00700 1 -- 172.21.15.63:0/3709167466 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7fc56401d020 con 0x55baecaf6e40 2024-05-11T13:42:15.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc587581b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 58 2024-05-11T13:42:15.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc587581b80 10 client.15822 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:42:15.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc587581b80 20 client.15822 populate_metadata read hostname 'smithi063' 2024-05-11T13:42:15.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc587581b80 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb10960 0x55baecb12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc587581b80 4 client.15822 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:42:15.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.749+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baecabb340 con 0x55baecb10960 2024-05-11T13:42:15.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.750+0000 7fc575e00700 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb10960 0x55baecb12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.751+0000 7fc575e00700 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb10960 0x55baecb12d50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1eec5ba5e36ff3d server_cookie=92e6d5c5161ffc2c in_seq=0 out_seq=0 2024-05-11T13:42:15.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.751+0000 7fc557e00700 10 client.15822 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:15.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc557e00700 1 -- 172.21.15.63:0/3709167466 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55baecabb340 con 0x55baecb10960 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc557e00700 10 client.15822 handle_command_reply: tid=0 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 client.15822 shutdown 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 msgr2=0x55baecb032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb032a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc5640097a0 tx=0x7fc564005ce0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 shutdown_connections 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb10960 0x55baecb12d50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecafc870 0x55baecb00b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb032a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 --2- 172.21.15.63:0/3709167466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb00640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 >> 172.21.15.63:0/3709167466 conn(0x55baeca1ccf0 msgr2=0x55baecae39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.753+0000 7fc557e00700 0 client.15822 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.754+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 shutdown_connections 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.754+0000 7fc587581b80 1 -- 172.21.15.63:0/3709167466 wait complete. 2024-05-11T13:42:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.754+0000 7fc587581b80 20 client.15822 trim_cache size 0 max 16384 2024-05-11T13:42:15.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 Processor -- start 2024-05-11T13:42:15.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 -- start start 2024-05-11T13:42:15.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb0ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecb00bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ad3d0 con 0x55baecafc870 2024-05-11T13:42:15.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781af150 con 0x55baecaf6e40 2024-05-11T13:42:15.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.775+0000 7fc587581b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781adaa0 con 0x55baecaf7800 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecb00bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecb00bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35496/0 (socket says 172.21.15.63:35496) 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 -- 172.21.15.63:0/1382529452 learned_addr learned my addr 172.21.15.63:0/1382529452 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575e00700 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb0ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 -- 172.21.15.63:0/1382529452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 msgr2=0x55baecb0ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb0ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 -- 172.21.15.63:0/1382529452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 msgr2=0x55baecaf6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 -- 172.21.15.63:0/1382529452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baecb13310 con 0x55baecafc870 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575e00700 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb0ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc575400700 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecb00bb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc5640098c0 tx=0x7fc564000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49c8bc810cb2a63 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.776+0000 7fc562a00700 1 -- 172.21.15.63:0/1382529452 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564013bd0 con 0x55baecafc870 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc562a00700 1 -- 172.21.15.63:0/1382529452 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc564026030 con 0x55baecafc870 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc562a00700 1 -- 172.21.15.63:0/1382529452 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564004a60 con 0x55baecafc870 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 -- 172.21.15.63:0/1382529452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 msgr2=0x55baecb00bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:15.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecb00bb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc5640098c0 tx=0x7fc564000bc0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 -- 172.21.15.63:0/1382529452 shutdown_connections 2024-05-11T13:42:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecafc870 0x55baecb00bb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf6e40 0x55baecaf6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 --2- 172.21.15.63:0/1382529452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecaf7800 0x55baecb0ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 -- 172.21.15.63:0/1382529452 >> 172.21.15.63:0/1382529452 conn(0x55baeca1ccf0 msgr2=0x55baecafa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.777+0000 7fc587581b80 1 -- 172.21.15.63:0/1382529452 shutdown_connections 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.778+0000 7fc587581b80 1 -- 172.21.15.63:0/1382529452 wait complete. 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.778+0000 7fc587581b80 1 -- start start 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.778+0000 7fc587581b80 1 Processor -- start 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc587581b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc587581b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf7800 0x55baecb05fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc587581b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecafc870 0x55baecb06520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc587581b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ae9c0 con 0x55baecaf6e40 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc587581b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ae230 con 0x55baecaf7800 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc587581b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ab390 con 0x55baecafc870 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc587581b80 10 client.0 _ll_register_callbacks cb 0x55baec9e7e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35502/0 (socket says 172.21.15.63:35502) 2024-05-11T13:42:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 -- 172.21.15.63:0/525180270 learned_addr learned my addr 172.21.15.63:0/525180270 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 -- 172.21.15.63:0/525180270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecafc870 msgr2=0x55baecb06520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecafc870 0x55baecb06520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575e00700 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf7800 0x55baecb05fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 -- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf7800 msgr2=0x55baecb05fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf7800 0x55baecb05fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.779+0000 7fc575400700 1 -- 172.21.15.63:0/525180270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baecb06a60 con 0x55baecaf6e40 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc575400700 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb092d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5640097a0 tx=0x7fc564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab56b79ee70cbe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc557e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc557e00700 1 -- 172.21.15.63:0/525180270 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564004a60 con 0x55baecaf6e40 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc557e00700 1 -- 172.21.15.63:0/525180270 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fc564004c00 con 0x55baecaf6e40 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc564026030 con 0x55baecaf6e40 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc557e00700 1 -- 172.21.15.63:0/525180270 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc564013020 con 0x55baecaf6e40 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc587581b80 10 client.15828 fetch_fsmap learned FSMap version 58 2024-05-11T13:42:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.780+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55baecb7bd90 con 0x55baecaf6e40 2024-05-11T13:42:15.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc557e00700 1 -- 172.21.15.63:0/525180270 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc564026030 con 0x55baecaf6e40 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc557e00700 1 -- 172.21.15.63:0/525180270 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7fc56401d030 con 0x55baecaf6e40 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc587581b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 58 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc587581b80 10 client.15828 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc587581b80 20 client.15828 populate_metadata read hostname 'smithi063' 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc587581b80 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb7c450 0x55baecb7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc587581b80 4 client.15828 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.8, "await_for": 42.0}] 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.781+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.8, "await_for": 42.0}) -- 0x55baeca4e590 con 0x55baecb7c450 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.782+0000 7fc575e00700 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb7c450 0x55baecb7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.783+0000 7fc575e00700 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb7c450 0x55baecb7e8d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac71c7ac123a746 server_cookie=cea159a9cb894d95 in_seq=0 out_seq=0 2024-05-11T13:42:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:15.783+0000 7fc557e00700 10 client.15828 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:16.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc557e00700 1 -- 172.21.15.63:0/525180270 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55baeca4e590 con 0x55baecb7c450 2024-05-11T13:42:16.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc557e00700 10 client.15828 handle_command_reply: tid=0 2024-05-11T13:42:16.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 client.15828 shutdown 2024-05-11T13:42:16.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 msgr2=0x55baecb092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:16.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb092d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5640097a0 tx=0x7fc564004730 comp rx=0 tx=0).stop 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 shutdown_connections 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baecafc870 0x55baecb06520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55baecaf6e40 0x55baecb092d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55baecb7c450 0x55baecb7e8d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 --2- 172.21.15.63:0/525180270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55baecaf7800 0x55baecb05fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 >> 172.21.15.63:0/525180270 conn(0x55baeca1ccf0 msgr2=0x55baecafa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.100+0000 7fc557e00700 0 client.15828 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.101+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 shutdown_connections 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.101+0000 7fc587581b80 1 -- 172.21.15.63:0/525180270 wait complete. 2024-05-11T13:42:16.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.101+0000 7fc587581b80 20 client.15828 trim_cache size 0 max 16384 2024-05-11T13:42:16.096 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:42:16.096 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 58, 2024-05-11T13:42:16.096 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:42:16.096 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 26, 2024-05-11T13:42:16.096 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:42:16.096 INFO:teuthology.orchestra.run.smithi063.stdout: "ffbcecd9": { 2024-05-11T13:42:16.096 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 26, 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 90.8, 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:42:16.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.102+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc55c0b2aa0 msgr2=0x7fc55c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.102+0000 7fc57cc00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc55c0b2aa0 0x7fc55c0b4f60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5700a5360 tx=0x7fc57006e7d0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc57810ffb0 msgr2=0x7fc57819bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:16.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc57810ffb0 0x7fc57819bb10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc57000efc0 tx=0x7fc57000c3a0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 shutdown_connections 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc55c0b2aa0 0x7fc55c0b4f60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc57810ffb0 0x7fc57819bb10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc578075780 0x7fc5781a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 --2- 172.21.15.63:0/601933480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5780760a0 0x7fc5781a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 >> 172.21.15.63:0/601933480 conn(0x7fc57807a6b0 msgr2=0x7fc5781016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 shutdown_connections 2024-05-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:16.103+0000 7fc57cc00700 1 -- 172.21.15.63:0/601933480 wait complete. 2024-05-11T13:42:16.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 50.0 s, 420.0 tps, lat 187.617 ms stddev 271.190 2024-05-11T13:42:17.046 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ffbcecd9', quiesce took 10.3 seconds. Will release after: 30.8 2024-05-11T13:42:17.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:17 smithi165 ceph-mon[33235]: pgmap v261: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 634 op/s 2024-05-11T13:42:17.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:17 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:42:17.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:17 smithi063 ceph-mon[29809]: pgmap v261: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 634 op/s 2024-05-11T13:42:17.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:17 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:42:17.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:17 smithi071 ceph-mon[33293]: pgmap v261: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 634 op/s 2024-05-11T13:42:17.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:17 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:42:19.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:19 smithi165 ceph-mon[33235]: pgmap v262: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.9 MiB/s wr, 452 op/s 2024-05-11T13:42:19.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:19 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:42:19.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:19 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:42:19.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:19 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:42:19.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:19 smithi063 ceph-mon[29809]: pgmap v262: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.9 MiB/s wr, 452 op/s 2024-05-11T13:42:19.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:19 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:42:19.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:19 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:42:19.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:19 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:42:19.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:19 smithi071 ceph-mon[33293]: pgmap v262: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.9 MiB/s wr, 452 op/s 2024-05-11T13:42:19.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:19 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:42:19.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:19 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:42:19.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:19 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:42:20.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:20 smithi165 ceph-mon[33235]: pgmap v263: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.8 MiB/s wr, 444 op/s 2024-05-11T13:42:20.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:20 smithi063 ceph-mon[29809]: pgmap v263: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.8 MiB/s wr, 444 op/s 2024-05-11T13:42:20.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:20 smithi071 ceph-mon[33293]: pgmap v263: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.8 MiB/s wr, 444 op/s 2024-05-11T13:42:21.148 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 55.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:23.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:23 smithi165 ceph-mon[33235]: pgmap v264: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 376 op/s 2024-05-11T13:42:23.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:23 smithi063 ceph-mon[29809]: pgmap v264: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 376 op/s 2024-05-11T13:42:23.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:23 smithi071 ceph-mon[33293]: pgmap v264: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 376 op/s 2024-05-11T13:42:24.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:24 smithi063 ceph-mon[29809]: pgmap v265: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.5 MiB/s wr, 388 op/s 2024-05-11T13:42:24.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:24 smithi071 ceph-mon[33293]: pgmap v265: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.5 MiB/s wr, 388 op/s 2024-05-11T13:42:24.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:24 smithi165 ceph-mon[33235]: pgmap v265: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.5 MiB/s wr, 388 op/s 2024-05-11T13:42:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 60.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:27.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:27 smithi165 ceph-mon[33235]: pgmap v266: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.0 MiB/s wr, 149 op/s 2024-05-11T13:42:27.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:27 smithi063 ceph-mon[29809]: pgmap v266: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.0 MiB/s wr, 149 op/s 2024-05-11T13:42:27.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:27 smithi071 ceph-mon[33293]: pgmap v266: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.0 MiB/s wr, 149 op/s 2024-05-11T13:42:28.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:28 smithi063 ceph-mon[29809]: pgmap v267: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-05-11T13:42:28.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:28 smithi071 ceph-mon[33293]: pgmap v267: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-05-11T13:42:28.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:28 smithi165 ceph-mon[33235]: pgmap v267: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-05-11T13:42:31.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 65.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:31.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:31 smithi165 ceph-mon[33235]: pgmap v268: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 142 KiB/s wr, 25 op/s 2024-05-11T13:42:31.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:31 smithi063 ceph-mon[29809]: pgmap v268: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 142 KiB/s wr, 25 op/s 2024-05-11T13:42:31.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:31 smithi071 ceph-mon[33293]: pgmap v268: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 142 KiB/s wr, 25 op/s 2024-05-11T13:42:32.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:32 smithi063 ceph-mon[29809]: pgmap v269: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 142 KiB/s wr, 25 op/s 2024-05-11T13:42:32.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:32 smithi071 ceph-mon[33293]: pgmap v269: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 142 KiB/s wr, 25 op/s 2024-05-11T13:42:32.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:32 smithi165 ceph-mon[33235]: pgmap v269: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 142 KiB/s wr, 25 op/s 2024-05-11T13:42:35.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:35 smithi165 ceph-mon[33235]: pgmap v270: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 617 KiB/s wr, 88 op/s 2024-05-11T13:42:35.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:35 smithi063 ceph-mon[29809]: pgmap v270: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 617 KiB/s wr, 88 op/s 2024-05-11T13:42:35.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:35 smithi071 ceph-mon[33293]: pgmap v270: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 617 KiB/s wr, 88 op/s 2024-05-11T13:42:36.147 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 70.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:36.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:36 smithi165 ceph-mon[33235]: pgmap v271: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 553 KiB/s wr, 76 op/s 2024-05-11T13:42:36.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:36 smithi063 ceph-mon[29809]: pgmap v271: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 553 KiB/s wr, 76 op/s 2024-05-11T13:42:36.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:36 smithi071 ceph-mon[33293]: pgmap v271: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 553 KiB/s wr, 76 op/s 2024-05-11T13:42:39.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:39 smithi165 ceph-mon[33235]: pgmap v272: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 559 KiB/s wr, 82 op/s 2024-05-11T13:42:39.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:39 smithi063 ceph-mon[29809]: pgmap v272: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 559 KiB/s wr, 82 op/s 2024-05-11T13:42:39.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:39 smithi071 ceph-mon[33293]: pgmap v272: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 559 KiB/s wr, 82 op/s 2024-05-11T13:42:40.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:40 smithi165 ceph-mon[33235]: pgmap v273: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 481 KiB/s wr, 68 op/s 2024-05-11T13:42:40.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:40 smithi063 ceph-mon[29809]: pgmap v273: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 481 KiB/s wr, 68 op/s 2024-05-11T13:42:40.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:40 smithi071 ceph-mon[33293]: pgmap v273: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 481 KiB/s wr, 68 op/s 2024-05-11T13:42:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 75.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:42.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:42 smithi063 ceph-mon[29809]: pgmap v274: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 481 KiB/s wr, 68 op/s 2024-05-11T13:42:42.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:42 smithi071 ceph-mon[33293]: pgmap v274: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 481 KiB/s wr, 68 op/s 2024-05-11T13:42:42.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:42 smithi165 ceph-mon[33235]: pgmap v274: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 481 KiB/s wr, 68 op/s 2024-05-11T13:42:44.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:44 smithi063 ceph-mon[29809]: pgmap v275: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 486 KiB/s wr, 73 op/s 2024-05-11T13:42:44.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:44 smithi071 ceph-mon[33293]: pgmap v275: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 486 KiB/s wr, 73 op/s 2024-05-11T13:42:44.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:44 smithi165 ceph-mon[33235]: pgmap v275: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 486 KiB/s wr, 73 op/s 2024-05-11T13:42:46.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 80.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:46.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:46 smithi063 ceph-mon[29809]: pgmap v276: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-11T13:42:46.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:46 smithi071 ceph-mon[33293]: pgmap v276: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-11T13:42:46.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:46 smithi165 ceph-mon[33235]: pgmap v276: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-11T13:42:47.847 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:42:48.340 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:42:48.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:48 smithi063 ceph-mon[29809]: pgmap v277: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-11T13:42:48.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:48 smithi071 ceph-mon[33293]: pgmap v277: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-11T13:42:48.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:48 smithi165 ceph-mon[33235]: pgmap v277: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-11T13:42:50.397 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:50 smithi063 ceph-mon[29809]: pgmap v278: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:50.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:50 smithi071 ceph-mon[33293]: pgmap v278: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 -- 172.21.15.63:0/2406143847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f80106040 msgr2=0x7f4f80112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2406143847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f80106040 0x7f4f80112550 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800b0a0 tx=0x7f4f7800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 -- 172.21.15.63:0/2406143847 shutdown_connections 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2406143847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f80106040 0x7f4f80112550 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2406143847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f8010b680 0x7f4f8010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2406143847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f80105680 0x7f4f80105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 -- 172.21.15.63:0/2406143847 >> 172.21.15.63:0/2406143847 conn(0x7f4f80101170 msgr2=0x7f4f80103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 -- 172.21.15.63:0/2406143847 shutdown_connections 2024-05-11T13:42:50.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.712+0000 7f4f86a00700 1 -- 172.21.15.63:0/2406143847 wait complete. 2024-05-11T13:42:50.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 Processor -- start 2024-05-11T13:42:50.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 -- start start 2024-05-11T13:42:50.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f80105680 0x7f4f801a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:50.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f80106040 0x7f4f801a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8010b680 0x7f4f8019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80115780 con 0x7f4f8010b680 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80115600 con 0x7f4f80106040 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.714+0000 7f4f86a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80115900 con 0x7f4f80105680 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8010b680 0x7f4f8019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8010b680 0x7f4f8019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60084/0 (socket says 172.21.15.63:60084) 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 -- 172.21.15.63:0/2537374828 learned_addr learned my addr 172.21.15.63:0/2537374828 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f85600700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f80105680 0x7f4f801a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f84c00700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f80106040 0x7f4f801a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:50.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 -- 172.21.15.63:0/2537374828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f80105680 msgr2=0x7f4f801a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:50.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f80105680 0x7f4f801a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:50.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 -- 172.21.15.63:0/2537374828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f80106040 msgr2=0x7f4f801a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:50.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f80106040 0x7f4f801a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:50.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f86000700 1 -- 172.21.15.63:0/2537374828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8019c1e0 con 0x7f4f8010b680 2024-05-11T13:42:50.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.715+0000 7f4f85600700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f80105680 0x7f4f801a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:42:50.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.716+0000 7f4f86000700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8010b680 0x7f4f8019ba80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800b070 tx=0x7f4f78004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:50.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.716+0000 7f4f7e000700 1 -- 172.21.15.63:0/2537374828 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78021070 con 0x7f4f8010b680 2024-05-11T13:42:50.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.716+0000 7f4f7e000700 1 -- 172.21.15.63:0/2537374828 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f78007c00 con 0x7f4f8010b680 2024-05-11T13:42:50.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.716+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8019c470 con 0x7f4f8010b680 2024-05-11T13:42:50.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.716+0000 7f4f7e000700 1 -- 172.21.15.63:0/2537374828 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78004780 con 0x7f4f8010b680 2024-05-11T13:42:50.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.716+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f801a8790 con 0x7f4f8010b680 2024-05-11T13:42:50.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.717+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f80106dc0 con 0x7f4f8010b680 2024-05-11T13:42:50.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.722+0000 7f4f7e000700 1 -- 172.21.15.63:0/2537374828 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4f78003f20 con 0x7f4f8010b680 2024-05-11T13:42:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.723+0000 7f4f7e000700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f680b29f0 0x7f4f680b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.723+0000 7f4f7e000700 1 -- 172.21.15.63:0/2537374828 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f4f780e3f70 con 0x7f4f8010b680 2024-05-11T13:42:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.723+0000 7f4f85600700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f680b29f0 0x7f4f680b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:50.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.724+0000 7f4f85600700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f680b29f0 0x7f4f680b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f74005e10 tx=0x7f4f74005da0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:50.726+0000 7f4f7e000700 1 -- 172.21.15.63:0/2537374828 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f780ab590 con 0x7f4f8010b680 2024-05-11T13:42:50.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:50 smithi165 ceph-mon[33235]: pgmap v278: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:51.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.029+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f8019cfb0 con 0x7f4f8010b680 2024-05-11T13:42:51.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.031+0000 7f4f7e000700 1 -- 172.21.15.63:0/2537374828 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11956 (secure 0 0 0) 0x7f4f780aae90 con 0x7f4f8010b680 2024-05-11T13:42:51.023 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:42:51.023 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:42:06.467762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:42:51.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f680b29f0 msgr2=0x7f4f680b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f680b29f0 0x7f4f680b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f74005e10 tx=0x7f4f74005da0 comp rx=0 tx=0).stop 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8010b680 msgr2=0x7f4f8019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8010b680 0x7f4f8019ba80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800b070 tx=0x7f4f78004e00 comp rx=0 tx=0).stop 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 shutdown_connections 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4f680b29f0 0x7f4f680b4eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f8010b680 0x7f4f8019ba80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f80105680 0x7f4f801a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 --2- 172.21.15.63:0/2537374828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f80106040 0x7f4f801a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.034+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 >> 172.21.15.63:0/2537374828 conn(0x7f4f80101170 msgr2=0x7f4f801018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.035+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 shutdown_connections 2024-05-11T13:42:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:51.035+0000 7f4f86a00700 1 -- 172.21.15.63:0/2537374828 wait complete. 2024-05-11T13:42:51.028 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 58 2024-05-11T13:42:51.148 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 85.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:42:51.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:51 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2537374828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:42:51.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:51 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2537374828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:42:51.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:51 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2537374828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:42:51.864 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id ffbcecd9 --release --await' 2024-05-11T13:42:51.865 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id ffbcecd9 --release --await 2024-05-11T13:42:52.339 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:42:52.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:52 smithi063 ceph-mon[29809]: pgmap v279: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:52.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:52 smithi071 ceph-mon[33293]: pgmap v279: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:52.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:52 smithi165 ceph-mon[33235]: pgmap v279: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:54.496 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:54 smithi063 ceph-mon[29809]: pgmap v280: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-11T13:42:54.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:54 smithi071 ceph-mon[33293]: pgmap v280: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-11T13:42:54.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:54 smithi165 ceph-mon[33235]: pgmap v280: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-11T13:42:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 -- 172.21.15.63:0/1567962101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06c5e0 msgr2=0x7f3ffc110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 --2- 172.21.15.63:0/1567962101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06c5e0 0x7f3ffc110400 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b0a0 tx=0x7f3ff400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 -- 172.21.15.63:0/1567962101 shutdown_connections 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 --2- 172.21.15.63:0/1567962101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06c5e0 0x7f3ffc110400 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 --2- 172.21.15.63:0/1567962101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc1095e0 0x7f3ffc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 --2- 172.21.15.63:0/1567962101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc06bc20 0x7f3ffc06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 -- 172.21.15.63:0/1567962101 >> 172.21.15.63:0/1567962101 conn(0x7f3ffc079400 msgr2=0x7f3ffc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 -- 172.21.15.63:0/1567962101 shutdown_connections 2024-05-11T13:42:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.848+0000 7f4003c00700 1 -- 172.21.15.63:0/1567962101 wait complete. 2024-05-11T13:42:54.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.850+0000 7f4003c00700 1 Processor -- start 2024-05-11T13:42:54.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.850+0000 7f4003c00700 1 -- start start 2024-05-11T13:42:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4003c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06bc20 0x7f3ffc1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4003c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc06c5e0 0x7f3ffc1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4003c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc1095e0 0x7f3ffc19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4003c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1135c0 con 0x7f3ffc06bc20 2024-05-11T13:42:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4003c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc113440 con 0x7f3ffc1095e0 2024-05-11T13:42:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4003c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc113740 con 0x7f3ffc06c5e0 2024-05-11T13:42:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4001200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06bc20 0x7f3ffc1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4001200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06bc20 0x7f3ffc1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60114/0 (socket says 172.21.15.63:60114) 2024-05-11T13:42:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f4001200700 1 -- 172.21.15.63:0/2012689794 learned_addr learned my addr 172.21.15.63:0/2012689794 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.851+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc06c5e0 0x7f3ffc1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.852+0000 7f4001c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc1095e0 0x7f3ffc19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.852+0000 7f4001200700 1 -- 172.21.15.63:0/2012689794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc06c5e0 msgr2=0x7f3ffc1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.852+0000 7f4001200700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc06c5e0 0x7f3ffc1a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.852+0000 7f4001200700 1 -- 172.21.15.63:0/2012689794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc1095e0 msgr2=0x7f3ffc19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.852+0000 7f4001200700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc1095e0 0x7f3ffc19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.852+0000 7f4001200700 1 -- 172.21.15.63:0/2012689794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc19c400 con 0x7f3ffc06bc20 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.852+0000 7f4001200700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06bc20 0x7f3ffc1a1a50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000d950 tx=0x7f3ff000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.853+0000 7f3ff9600700 1 -- 172.21.15.63:0/2012689794 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0016070 con 0x7f3ffc06bc20 2024-05-11T13:42:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.853+0000 7f3ff9600700 1 -- 172.21.15.63:0/2012689794 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff0009dd0 con 0x7f3ffc06bc20 2024-05-11T13:42:54.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.853+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc19c6f0 con 0x7f3ffc06bc20 2024-05-11T13:42:54.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.853+0000 7f3ff9600700 1 -- 172.21.15.63:0/2012689794 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000f3c0 con 0x7f3ffc06bc20 2024-05-11T13:42:54.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.853+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a89d0 con 0x7f3ffc06bc20 2024-05-11T13:42:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.857+0000 7f3ff9600700 1 -- 172.21.15.63:0/2012689794 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3ff0019070 con 0x7f3ffc06bc20 2024-05-11T13:42:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.858+0000 7f400e5f8b80 1 Processor -- start 2024-05-11T13:42:54.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.858+0000 7f400e5f8b80 1 -- start start 2024-05-11T13:42:54.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.858+0000 7f3ff9600700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f3fe40b29f0 0x7f3fe40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f3ff9600700 1 -- 172.21.15.63:0/2012689794 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3ff00e2850 con 0x7f3ffc06bc20 2024-05-11T13:42:54.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0bf83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0bf2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f4001c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f3fe40b29f0 0x7f3fe40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0bfed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601e0b44b20 con 0x5601e0bf7fd0 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5601e0ae2860 con 0x5601e0bf25a0 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601e0b40ae0 con 0x5601e0bf2f60 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f4001200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0bf83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f4001200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0bf83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60126/0 (socket says 172.21.15.63:60126) 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f4001200700 1 -- 172.21.15.63:0/2780348652 learned_addr learned my addr 172.21.15.63:0/2780348652 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:54.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.859+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0bf2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001c00700 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0bfed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001200700 1 -- 172.21.15.63:0/2780348652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 msgr2=0x5601e0bfed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001200700 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0bfed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001200700 1 -- 172.21.15.63:0/2780348652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 msgr2=0x5601e0bf2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001200700 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0bf2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001200700 1 -- 172.21.15.63:0/2780348652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e0a919d0 con 0x5601e0bf7fd0 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001c00700 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0bfed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f3fe40b29f0 0x7f3fe40b4eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009e60 tx=0x7f3ff40090d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f4001200700 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0bf83b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00a93e0 tx=0x7f3ff00a8f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185822c92347855f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f3fee000700 1 -- 172.21.15.63:0/2780348652 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0020070 con 0x5601e0bf7fd0 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f3fee000700 1 -- 172.21.15.63:0/2780348652 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff00b2540 con 0x5601e0bf7fd0 2024-05-11T13:42:54.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.860+0000 7f3fee000700 1 -- 172.21.15.63:0/2780348652 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00e2640 con 0x5601e0bf7fd0 2024-05-11T13:42:54.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 -- 172.21.15.63:0/2780348652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 msgr2=0x5601e0bf83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0bf83b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00a93e0 tx=0x7f3ff00a8f80 comp rx=0 tx=0).stop 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 -- 172.21.15.63:0/2780348652 shutdown_connections 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0bfed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0bf83b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/2780348652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0bf2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 -- 172.21.15.63:0/2780348652 >> 172.21.15.63:0/2780348652 conn(0x5601e0b20c60 msgr2=0x5601e0bece90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.861+0000 7f400e5f8b80 1 -- 172.21.15.63:0/2780348652 shutdown_connections 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.862+0000 7f400e5f8b80 1 -- 172.21.15.63:0/2780348652 wait complete. 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.862+0000 7f400e5f8b80 1 -- start start 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.862+0000 7f400e5f8b80 1 Processor -- start 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf25a0 0x5601e0bfe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0bfbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf7fd0 0x5601e0bfc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601e0bff2d0 con 0x5601e0bf2f60 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5601e0aaac20 con 0x5601e0bf7fd0 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601e0bff450 con 0x5601e0bf25a0 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f400e5f8b80 10 client.0 _ll_register_callbacks cb 0x5601e0aebd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f3ffbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0bfbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f3ffbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0bfbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60130/0 (socket says 172.21.15.63:60130) 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.863+0000 7f3ffbe00700 1 -- 172.21.15.63:0/1449350646 learned_addr learned my addr 172.21.15.63:0/1449350646 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f4001200700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf25a0 0x5601e0bfe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f4001c00700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf7fd0 0x5601e0bfc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f3ffbe00700 1 -- 172.21.15.63:0/1449350646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf25a0 msgr2=0x5601e0bfe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf25a0 0x5601e0bfe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f3ffbe00700 1 -- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf7fd0 msgr2=0x5601e0bfc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf7fd0 0x5601e0bfc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f3ffbe00700 1 -- 172.21.15.63:0/1449350646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e0b0a520 con 0x5601e0bf2f60 2024-05-11T13:42:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f4001200700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf25a0 0x5601e0bfe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:42:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0bfbd30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800b9b0 tx=0x7f3fe800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec29f5f9c0700e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.864+0000 7f3fe2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:42:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.865+0000 7f3fe2a00700 1 -- 172.21.15.63:0/1449350646 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe801b440 con 0x5601e0bf2f60 2024-05-11T13:42:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.865+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff00b2540 con 0x5601e0bf2f60 2024-05-11T13:42:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.865+0000 7f3fe2a00700 1 -- 172.21.15.63:0/1449350646 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3fe801b5e0 con 0x5601e0bf2f60 2024-05-11T13:42:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.865+0000 7f3fe2a00700 1 -- 172.21.15.63:0/1449350646 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3fe8024050 con 0x5601e0bf2f60 2024-05-11T13:42:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.865+0000 7f400e5f8b80 10 client.15846 fetch_fsmap learned FSMap version 58 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.865+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5601e0c09ad0 con 0x5601e0bf2f60 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.865+0000 7f3fe2a00700 1 -- 172.21.15.63:0/1449350646 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff00b2540 con 0x5601e0bf2f60 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f3fe2a00700 1 -- 172.21.15.63:0/1449350646 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7f3fe8019020 con 0x5601e0bf2f60 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f400e5f8b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 58 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f400e5f8b80 10 client.15846 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f400e5f8b80 20 client.15846 populate_metadata read hostname 'smithi063' 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c0c160 0x5601e0c0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f400e5f8b80 4 client.15846 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601e0bbf2b0 con 0x5601e0c0c160 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.866+0000 7f4001200700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c0c160 0x5601e0c0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.867+0000 7f4001200700 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c0c160 0x5601e0c0e570 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae03e2e64a61ef0 server_cookie=67411e4dab561d0f in_seq=0 out_seq=0 2024-05-11T13:42:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.867+0000 7f3fe2a00700 10 client.15846 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:54.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.869+0000 7f3fe2a00700 1 -- 172.21.15.63:0/1449350646 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601e0bbf2b0 con 0x5601e0c0c160 2024-05-11T13:42:54.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.869+0000 7f3fe2a00700 10 client.15846 handle_command_reply: tid=0 2024-05-11T13:42:54.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.869+0000 7f400e5f8b80 1 client.15846 shutdown 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 msgr2=0x5601e0bfbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0bfbd30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800b9b0 tx=0x7f3fe800bd70 comp rx=0 tx=0).stop 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 shutdown_connections 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c0c160 0x5601e0c0e570 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf7fd0 0x5601e0bfc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf25a0 0x5601e0bfe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/1449350646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0bfbd30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f3fe2a00700 0 client.15846 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 >> 172.21.15.63:0/1449350646 conn(0x5601e0b20c60 msgr2=0x5601e0be7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 shutdown_connections 2024-05-11T13:42:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 1 -- 172.21.15.63:0/1449350646 wait complete. 2024-05-11T13:42:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.870+0000 7f400e5f8b80 20 client.15846 trim_cache size 0 max 16384 2024-05-11T13:42:54.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.892+0000 7f400e5f8b80 1 Processor -- start 2024-05-11T13:42:54.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.892+0000 7f400e5f8b80 1 -- start start 2024-05-11T13:42:54.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c08320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0c13e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf7fd0 0x5601e0c143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00ee40 con 0x5601e0bf2f60 2024-05-11T13:42:54.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc010bc0 con 0x5601e0bf25a0 2024-05-11T13:42:54.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00f510 con 0x5601e0bf7fd0 2024-05-11T13:42:54.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f3ffbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0c13e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f3ffbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0c13e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60138/0 (socket says 172.21.15.63:60138) 2024-05-11T13:42:54.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f3ffbe00700 1 -- 172.21.15.63:0/571091144 learned_addr learned my addr 172.21.15.63:0/571091144 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f4001200700 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c08320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.893+0000 7f4001c00700 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf7fd0 0x5601e0c143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3ffbe00700 1 -- 172.21.15.63:0/571091144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf7fd0 msgr2=0x5601e0c143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf7fd0 0x5601e0c143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3ffbe00700 1 -- 172.21.15.63:0/571091144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 msgr2=0x5601e0c08320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c08320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3ffbe00700 1 -- 172.21.15.63:0/571091144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e0bed3d0 con 0x5601e0bf2f60 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0c13e80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80119d0 tx=0x7f3fe8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d39b9863ca13d13 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3fed600700 1 -- 172.21.15.63:0/571091144 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe802c020 con 0x5601e0bf2f60 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3fed600700 1 -- 172.21.15.63:0/571091144 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fe8003960 con 0x5601e0bf2f60 2024-05-11T13:42:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.894+0000 7f3fed600700 1 -- 172.21.15.63:0/571091144 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe801bb30 con 0x5601e0bf2f60 2024-05-11T13:42:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 -- 172.21.15.63:0/571091144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 msgr2=0x5601e0c13e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0c13e80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80119d0 tx=0x7f3fe8005960 comp rx=0 tx=0).stop 2024-05-11T13:42:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 -- 172.21.15.63:0/571091144 shutdown_connections 2024-05-11T13:42:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf7fd0 0x5601e0c143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c08320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/571091144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf2f60 0x5601e0c13e80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 -- 172.21.15.63:0/571091144 >> 172.21.15.63:0/571091144 conn(0x5601e0b20c60 msgr2=0x5601e0bfa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 -- 172.21.15.63:0/571091144 shutdown_connections 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.895+0000 7f400e5f8b80 1 -- 172.21.15.63:0/571091144 wait complete. 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 -- start start 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 Processor -- start 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0c01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0c01db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc010430 con 0x5601e0bf7fd0 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00fca0 con 0x5601e0bf25a0 2024-05-11T13:42:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.896+0000 7f400e5f8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00ce00 con 0x5601e0bf2f60 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f400e5f8b80 10 client.0 _ll_register_callbacks cb 0x5601e0aebd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0c01db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0c01db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60142/0 (socket says 172.21.15.63:60142) 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 -- 172.21.15.63:0/186452061 learned_addr learned my addr 172.21.15.63:0/186452061 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001200700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f3ffbe00700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0c01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 -- 172.21.15.63:0/186452061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 msgr2=0x5601e0c01670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0c01670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 -- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 msgr2=0x5601e0c04da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c04da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001c00700 1 -- 172.21.15.63:0/186452061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e0c76a40 con 0x5601e0bf7fd0 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.897+0000 7f4001200700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c04da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f4001c00700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0c01db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b070 tx=0x7f3ff400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471ab9e73adfd87 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:42:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f3fe2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:42:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f3fe2a00700 1 -- 172.21.15.63:0/186452061 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4005280 con 0x5601e0bf7fd0 2024-05-11T13:42:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe8003960 con 0x5601e0bf7fd0 2024-05-11T13:42:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f3fe2a00700 1 -- 172.21.15.63:0/186452061 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3ff4005420 con 0x5601e0bf7fd0 2024-05-11T13:42:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f3fe2a00700 1 -- 172.21.15.63:0/186452061 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3ff403a050 con 0x5601e0bf7fd0 2024-05-11T13:42:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f400e5f8b80 10 client.15852 fetch_fsmap learned FSMap version 58 2024-05-11T13:42:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.898+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5601e0c77c50 con 0x5601e0bf7fd0 2024-05-11T13:42:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f3fe2a00700 1 -- 172.21.15.63:0/186452061 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fe8003960 con 0x5601e0bf7fd0 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f3fe2a00700 1 -- 172.21.15.63:0/186452061 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7f3ff402b070 con 0x5601e0bf7fd0 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f400e5f8b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 58 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f400e5f8b80 10 client.15852 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f400e5f8b80 20 client.15852 populate_metadata read hostname 'smithi063' 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c78450 0x5601e0c7a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f400e5f8b80 4 client.15852 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ffbcecd9", "release": true, "await": true}] 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.899+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ffbcecd9", "release": true, "await": true}) -- 0x5601e0b52500 con 0x5601e0c78450 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.900+0000 7f4001200700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c78450 0x5601e0c7a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.901+0000 7f4001200700 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c78450 0x5601e0c7a870 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be4655e4fe5bb9d server_cookie=bfc5730ef8f6c13c in_seq=0 out_seq=0 2024-05-11T13:42:54.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.901+0000 7f3fe2a00700 10 client.15852 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:54.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.978+0000 7f3fe2a00700 1 -- 172.21.15.63:0/186452061 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5601e0b52500 con 0x5601e0c78450 2024-05-11T13:42:54.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.978+0000 7f3fe2a00700 10 client.15852 handle_command_reply: tid=0 2024-05-11T13:42:54.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.978+0000 7f400e5f8b80 1 client.15852 shutdown 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.979+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 msgr2=0x5601e0c01db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.979+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0c01db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b070 tx=0x7f3ff400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.981+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 shutdown_connections 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.981+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601e0bf7fd0 0x5601e0c01db0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.981+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601e0c78450 0x5601e0c7a870 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.981+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e0bf25a0 0x5601e0c04da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.981+0000 7f400e5f8b80 1 --2- 172.21.15.63:0/186452061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601e0bf2f60 0x5601e0c01670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.981+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 >> 172.21.15.63:0/186452061 conn(0x5601e0b20c60 msgr2=0x5601e0bf5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.982+0000 7f3fe2a00700 0 client.15852 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.982+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 shutdown_connections 2024-05-11T13:42:54.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.982+0000 7f400e5f8b80 1 -- 172.21.15.63:0/186452061 wait complete. 2024-05-11T13:42:54.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.983+0000 7f400e5f8b80 20 client.15852 trim_cache size 0 max 16384 2024-05-11T13:42:54.976 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:42:54.976 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 58, 2024-05-11T13:42:54.976 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:42:54.976 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 28, 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "ffbcecd9": { 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 28, 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 90.8, 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:42:54.977 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:42:54.978 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f3fe40b29f0 msgr2=0x7f3fe40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f3fe40b29f0 0x7f3fe40b4eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009e60 tx=0x7f3ff40090d0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06bc20 msgr2=0x7f3ffc1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06bc20 0x7f3ffc1a1a50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000d950 tx=0x7f3ff000dc60 comp rx=0 tx=0).stop 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 shutdown_connections 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f3fe40b29f0 0x7f3fe40b4eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc1095e0 0x7f3ffc19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc06bc20 0x7f3ffc1a1a50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 --2- 172.21.15.63:0/2012689794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc06c5e0 0x7f3ffc1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:42:54.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.984+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 >> 172.21.15.63:0/2012689794 conn(0x7f3ffc079400 msgr2=0x7f3ffc10e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:42:54.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.985+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 shutdown_connections 2024-05-11T13:42:54.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:42:54.986+0000 7f4003c00700 1 -- 172.21.15.63:0/2012689794 wait complete. 2024-05-11T13:42:55.916 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ffbcecd9', seconds elapsed: 49.2 2024-05-11T13:42:56.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 90.0 s, 207.0 tps, lat 3973.408 ms stddev 11836.178 2024-05-11T13:42:56.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:56 smithi063 ceph-mon[29809]: pgmap v281: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:56.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:56 smithi071 ceph-mon[33293]: pgmap v281: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:56.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:56 smithi165 ceph-mon[33235]: pgmap v281: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-11T13:42:58.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:42:58 smithi063 ceph-mon[29809]: pgmap v282: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-05-11T13:42:58.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:42:58 smithi071 ceph-mon[33293]: pgmap v282: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-05-11T13:42:58.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:42:58 smithi165 ceph-mon[33235]: pgmap v282: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-05-11T13:43:00.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:00 smithi063 ceph-mon[29809]: pgmap v283: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-11T13:43:00.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:00 smithi071 ceph-mon[33293]: pgmap v283: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-11T13:43:00.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:00 smithi165 ceph-mon[33235]: pgmap v283: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-11T13:43:01.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 95.0 s, 532.0 tps, lat 188.154 ms stddev 291.170 2024-05-11T13:43:02.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:02 smithi063 ceph-mon[29809]: pgmap v284: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-11T13:43:02.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:02 smithi071 ceph-mon[33293]: pgmap v284: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-11T13:43:02.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:02 smithi165 ceph-mon[33235]: pgmap v284: 129 pgs: 129 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-11T13:43:05.398 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:05 smithi071 ceph-mon[33293]: osdmap e98: 12 total, 12 up, 12 in 2024-05-11T13:43:05.398 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:05 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:05.398 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:05 smithi071 ceph-mon[33293]: pgmap v286: 129 pgs: 129 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-05-11T13:43:05.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:05 smithi063 ceph-mon[29809]: osdmap e98: 12 total, 12 up, 12 in 2024-05-11T13:43:05.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:05 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:05.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:05 smithi063 ceph-mon[29809]: pgmap v286: 129 pgs: 129 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-05-11T13:43:05.510 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:05 smithi165 ceph-mon[33235]: osdmap e98: 12 total, 12 up, 12 in 2024-05-11T13:43:05.510 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:05 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:05.510 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:05 smithi165 ceph-mon[33235]: pgmap v286: 129 pgs: 129 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-05-11T13:43:06.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 100.0 s, 682.2 tps, lat 145.336 ms stddev 253.169 2024-05-11T13:43:06.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:06 smithi063 ceph-mon[29809]: pgmap v287: 129 pgs: 129 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-05-11T13:43:06.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:06 smithi071 ceph-mon[33293]: pgmap v287: 129 pgs: 129 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-05-11T13:43:06.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:06 smithi165 ceph-mon[33235]: pgmap v287: 129 pgs: 129 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-05-11T13:43:08.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:08 smithi063 ceph-mon[29809]: pgmap v288: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 13 MiB/s wr, 802 op/s 2024-05-11T13:43:08.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:08 smithi071 ceph-mon[33293]: pgmap v288: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 13 MiB/s wr, 802 op/s 2024-05-11T13:43:08.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:08 smithi165 ceph-mon[33235]: pgmap v288: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 13 MiB/s wr, 802 op/s 2024-05-11T13:43:10.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:10 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:10.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:10 smithi063 ceph-mon[29809]: osdmap e99: 12 total, 12 up, 12 in 2024-05-11T13:43:10.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:10 smithi063 ceph-mon[29809]: pgmap v290: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 16 MiB/s wr, 1.00k op/s 2024-05-11T13:43:10.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:10 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:10.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:10 smithi071 ceph-mon[33293]: osdmap e99: 12 total, 12 up, 12 in 2024-05-11T13:43:10.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:10 smithi071 ceph-mon[33293]: pgmap v290: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 16 MiB/s wr, 1.00k op/s 2024-05-11T13:43:10.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:10 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:10.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:10 smithi165 ceph-mon[33235]: osdmap e99: 12 total, 12 up, 12 in 2024-05-11T13:43:10.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:10 smithi165 ceph-mon[33235]: pgmap v290: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 16 MiB/s wr, 1.00k op/s 2024-05-11T13:43:11.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 105.0 s, 241.1 tps, lat 177.007 ms stddev 52.157 2024-05-11T13:43:12.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:12 smithi063 ceph-mon[29809]: pgmap v291: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.6 MiB/s wr, 466 op/s 2024-05-11T13:43:12.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:12 smithi071 ceph-mon[33293]: pgmap v291: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.6 MiB/s wr, 466 op/s 2024-05-11T13:43:12.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:12 smithi165 ceph-mon[33235]: pgmap v291: 129 pgs: 129 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 7.6 MiB/s wr, 466 op/s 2024-05-11T13:43:14.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:14 smithi063 ceph-mon[29809]: pgmap v292: 129 pgs: 129 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-05-11T13:43:14.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:14 smithi071 ceph-mon[33293]: pgmap v292: 129 pgs: 129 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-05-11T13:43:14.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:14 smithi165 ceph-mon[33235]: pgmap v292: 129 pgs: 129 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-05-11T13:43:16.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 110.0 s, 446.6 tps, lat 350.704 ms stddev 662.007 2024-05-11T13:43:16.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:16 smithi063 ceph-mon[29809]: pgmap v293: 129 pgs: 129 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-05-11T13:43:16.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:16 smithi071 ceph-mon[33293]: pgmap v293: 129 pgs: 129 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-05-11T13:43:16.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:16 smithi165 ceph-mon[33235]: pgmap v293: 129 pgs: 129 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-05-11T13:43:18.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:18 smithi063 ceph-mon[29809]: pgmap v294: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.6 MiB/s wr, 401 op/s 2024-05-11T13:43:18.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:18 smithi071 ceph-mon[33293]: pgmap v294: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.6 MiB/s wr, 401 op/s 2024-05-11T13:43:18.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:18 smithi165 ceph-mon[33235]: pgmap v294: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.6 MiB/s wr, 401 op/s 2024-05-11T13:43:19.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:19 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:43:19.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:19 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:43:19.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:19 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:43:20.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:43:20.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:43:20.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:43:20.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:20 smithi063 ceph-mon[29809]: pgmap v295: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.5 MiB/s wr, 395 op/s 2024-05-11T13:43:20.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:43:20.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:43:20.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:43:20.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:20 smithi071 ceph-mon[33293]: pgmap v295: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.5 MiB/s wr, 395 op/s 2024-05-11T13:43:20.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:43:20.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:43:20.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:43:20.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:20 smithi165 ceph-mon[33235]: pgmap v295: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.5 MiB/s wr, 395 op/s 2024-05-11T13:43:21.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 115.0 s, 248.1 tps, lat 236.248 ms stddev 40.916 2024-05-11T13:43:22.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:22 smithi063 ceph-mon[29809]: pgmap v296: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-05-11T13:43:22.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:22 smithi071 ceph-mon[33293]: pgmap v296: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-05-11T13:43:22.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:22 smithi165 ceph-mon[33235]: pgmap v296: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-05-11T13:43:24.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:24 smithi063 ceph-mon[29809]: pgmap v297: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 5.5 MiB/s wr, 498 op/s 2024-05-11T13:43:24.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:24 smithi071 ceph-mon[33293]: pgmap v297: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 5.5 MiB/s wr, 498 op/s 2024-05-11T13:43:24.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:24 smithi165 ceph-mon[33235]: pgmap v297: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 5.5 MiB/s wr, 498 op/s 2024-05-11T13:43:26.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 120.0 s, 452.2 tps, lat 312.726 ms stddev 491.653 2024-05-11T13:43:26.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:26 smithi063 ceph-mon[29809]: pgmap v298: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 365 op/s 2024-05-11T13:43:26.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:26 smithi071 ceph-mon[33293]: pgmap v298: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 365 op/s 2024-05-11T13:43:26.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:26 smithi165 ceph-mon[33235]: pgmap v298: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 365 op/s 2024-05-11T13:43:28.328 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:28 smithi063 ceph-mon[29809]: pgmap v299: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.3 MiB/s wr, 645 op/s 2024-05-11T13:43:28.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:28 smithi071 ceph-mon[33293]: pgmap v299: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.3 MiB/s wr, 645 op/s 2024-05-11T13:43:28.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:28 smithi165 ceph-mon[33235]: pgmap v299: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.3 MiB/s wr, 645 op/s 2024-05-11T13:43:30.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:30 smithi063 ceph-mon[29809]: pgmap v300: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-05-11T13:43:30.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:30 smithi071 ceph-mon[33293]: pgmap v300: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-05-11T13:43:30.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:30 smithi165 ceph-mon[33235]: pgmap v300: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-05-11T13:43:31.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 125.0 s, 455.4 tps, lat 220.078 ms stddev 57.309 2024-05-11T13:43:32.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:32 smithi063 ceph-mon[29809]: pgmap v301: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-05-11T13:43:32.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:32 smithi071 ceph-mon[33293]: pgmap v301: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-05-11T13:43:32.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:32 smithi165 ceph-mon[33235]: pgmap v301: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-05-11T13:43:34.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:34 smithi063 ceph-mon[29809]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 738 op/s 2024-05-11T13:43:34.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:34 smithi071 ceph-mon[33293]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 738 op/s 2024-05-11T13:43:34.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:34 smithi165 ceph-mon[33235]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 738 op/s 2024-05-11T13:43:36.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 130.0 s, 432.2 tps, lat 216.922 ms stddev 42.675 2024-05-11T13:43:36.609 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:36 smithi165 ceph-mon[33235]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-05-11T13:43:36.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:36 smithi063 ceph-mon[29809]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-05-11T13:43:36.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:36 smithi071 ceph-mon[33293]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-05-11T13:43:38.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:38 smithi063 ceph-mon[29809]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.1 MiB/s wr, 833 op/s 2024-05-11T13:43:38.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:38 smithi071 ceph-mon[33293]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.1 MiB/s wr, 833 op/s 2024-05-11T13:43:38.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:38 smithi165 ceph-mon[33235]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.1 MiB/s wr, 833 op/s 2024-05-11T13:43:40.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:40 smithi063 ceph-mon[29809]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-05-11T13:43:40.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:40 smithi071 ceph-mon[33293]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-05-11T13:43:40.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:40 smithi165 ceph-mon[33235]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-05-11T13:43:41.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 135.0 s, 393.6 tps, lat 256.171 ms stddev 90.643 2024-05-11T13:43:42.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:42 smithi063 ceph-mon[29809]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-05-11T13:43:42.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:42 smithi071 ceph-mon[33293]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-05-11T13:43:42.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:42 smithi165 ceph-mon[33235]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-05-11T13:43:44.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:44 smithi063 ceph-mon[29809]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.4 MiB/s wr, 789 op/s 2024-05-11T13:43:44.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:44 smithi071 ceph-mon[33293]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.4 MiB/s wr, 789 op/s 2024-05-11T13:43:44.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:44 smithi165 ceph-mon[33235]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.4 MiB/s wr, 789 op/s 2024-05-11T13:43:46.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 140.0 s, 414.2 tps, lat 253.975 ms stddev 100.558 2024-05-11T13:43:46.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:46 smithi063 ceph-mon[29809]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-05-11T13:43:46.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:46 smithi071 ceph-mon[33293]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-05-11T13:43:46.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:46 smithi165 ceph-mon[33235]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-05-11T13:43:48.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:48 smithi063 ceph-mon[29809]: pgmap v309: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 8.0 MiB/s wr, 750 op/s 2024-05-11T13:43:48.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:48 smithi071 ceph-mon[33293]: pgmap v309: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 8.0 MiB/s wr, 750 op/s 2024-05-11T13:43:48.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:48 smithi165 ceph-mon[33235]: pgmap v309: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 8.0 MiB/s wr, 750 op/s 2024-05-11T13:43:50.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:50 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:50.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:50 smithi063 ceph-mon[29809]: pgmap v310: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-11T13:43:50.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:50 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:50.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:50 smithi071 ceph-mon[33293]: pgmap v310: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-11T13:43:50.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:50 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:50.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:50 smithi165 ceph-mon[33235]: pgmap v310: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-11T13:43:51.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 145.0 s, 438.8 tps, lat 228.419 ms stddev 42.719 2024-05-11T13:43:52.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:52 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:52.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:52 smithi165 ceph-mon[33235]: pgmap v311: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-11T13:43:52.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:52 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:52.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:52 smithi063 ceph-mon[29809]: pgmap v311: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-11T13:43:52.944 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:52 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:52.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:52 smithi071 ceph-mon[33293]: pgmap v311: 129 pgs: 129 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-11T13:43:53.117 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.4 2024-05-11T13:43:53.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:43:53.785 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:43:54.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:54 smithi063 ceph-mon[29809]: pgmap v312: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.9 MiB/s wr, 747 op/s 2024-05-11T13:43:54.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:54 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:54.697 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:54 smithi071 ceph-mon[33293]: pgmap v312: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.9 MiB/s wr, 747 op/s 2024-05-11T13:43:54.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:54 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:54.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:54 smithi165 ceph-mon[33235]: pgmap v312: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.9 MiB/s wr, 747 op/s 2024-05-11T13:43:54.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:54 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:43:56.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 150.0 s, 460.2 tps, lat 218.159 ms stddev 50.174 2024-05-11T13:43:56.578 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:56 smithi063 ceph-mon[29809]: pgmap v313: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-05-11T13:43:56.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:56 smithi071 ceph-mon[33293]: pgmap v313: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-05-11T13:43:56.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:56 smithi165 ceph-mon[33235]: pgmap v313: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-05-11T13:43:58.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:43:58 smithi063 ceph-mon[29809]: pgmap v314: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.4 MiB/s wr, 791 op/s 2024-05-11T13:43:58.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:43:58 smithi071 ceph-mon[33293]: pgmap v314: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.4 MiB/s wr, 791 op/s 2024-05-11T13:43:58.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:43:58 smithi165 ceph-mon[33235]: pgmap v314: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.4 MiB/s wr, 791 op/s 2024-05-11T13:43:59.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.590+0000 7f1f61000700 1 -- 172.21.15.63:0/2057438456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c1034a0 msgr2=0x7f1f5c103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:43:59.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.590+0000 7f1f61000700 1 --2- 172.21.15.63:0/2057438456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c1034a0 0x7f1f5c103920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0099c0 tx=0x7f1f4c009cd0 comp rx=0 tx=0).stop 2024-05-11T13:43:59.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.592+0000 7f1f61000700 1 -- 172.21.15.63:0/2057438456 shutdown_connections 2024-05-11T13:43:59.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.592+0000 7f1f61000700 1 --2- 172.21.15.63:0/2057438456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f5c103e60 0x7f1f5c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:43:59.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.592+0000 7f1f61000700 1 --2- 172.21.15.63:0/2057438456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c109550 0x7f1f5c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:43:59.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.592+0000 7f1f61000700 1 --2- 172.21.15.63:0/2057438456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c1034a0 0x7f1f5c103920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:43:59.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.592+0000 7f1f61000700 1 -- 172.21.15.63:0/2057438456 >> 172.21.15.63:0/2057438456 conn(0x7f1f5c0fefd0 msgr2=0x7f1f5c1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:43:59.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.593+0000 7f1f61000700 1 -- 172.21.15.63:0/2057438456 shutdown_connections 2024-05-11T13:43:59.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.593+0000 7f1f61000700 1 -- 172.21.15.63:0/2057438456 wait complete. 2024-05-11T13:43:59.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.597+0000 7f1f61000700 1 Processor -- start 2024-05-11T13:43:59.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.597+0000 7f1f61000700 1 -- start start 2024-05-11T13:43:59.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.599+0000 7f1f61000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f5c1034a0 0x7f1f5c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:43:59.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.599+0000 7f1f61000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c103e60 0x7f1f5c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:43:59.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.599+0000 7f1f61000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c109550 0x7f1f5c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:43:59.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.600+0000 7f1f53e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c103e60 0x7f1f5c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:43:59.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.600+0000 7f1f53e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c103e60 0x7f1f5c19fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:40998/0 (socket says 172.21.15.63:40998) 2024-05-11T13:43:59.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.600+0000 7f1f5b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f5c1034a0 0x7f1f5c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:43:59.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.600+0000 7f1f5b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f5c1034a0 0x7f1f5c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35984/0 (socket says 172.21.15.63:35984) 2024-05-11T13:43:59.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.602+0000 7f1f61000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c113580 con 0x7f1f5c1034a0 2024-05-11T13:43:59.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.602+0000 7f1f61000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c113400 con 0x7f1f5c109550 2024-05-11T13:43:59.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.602+0000 7f1f61000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c113700 con 0x7f1f5c103e60 2024-05-11T13:43:59.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.602+0000 7f1f53e00700 1 -- 172.21.15.63:0/376917169 learned_addr learned my addr 172.21.15.63:0/376917169 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.602+0000 7f1f5be00700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c109550 0x7f1f5c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.603+0000 7f1f53e00700 1 -- 172.21.15.63:0/376917169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c109550 msgr2=0x7f1f5c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.603+0000 7f1f53e00700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c109550 0x7f1f5c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.603+0000 7f1f53e00700 1 -- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f5c1034a0 msgr2=0x7f1f5c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.603+0000 7f1f53e00700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f5c1034a0 0x7f1f5c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.603+0000 7f1f53e00700 1 -- 172.21.15.63:0/376917169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c199fc0 con 0x7f1f5c103e60 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.603+0000 7f1f5be00700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c109550 0x7f1f5c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:43:59.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.603+0000 7f1f53e00700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c103e60 0x7f1f5c19fbe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000c00 tx=0x7f1f4c01ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:43:59.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.604+0000 7f1f58c00700 1 -- 172.21.15.63:0/376917169 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c02a070 con 0x7f1f5c103e60 2024-05-11T13:43:59.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.604+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5c19a250 con 0x7f1f5c103e60 2024-05-11T13:43:59.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.604+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5c075780 con 0x7f1f5c103e60 2024-05-11T13:43:59.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.604+0000 7f1f58c00700 1 -- 172.21.15.63:0/376917169 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f4c005550 con 0x7f1f5c103e60 2024-05-11T13:43:59.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.604+0000 7f1f58c00700 1 -- 172.21.15.63:0/376917169 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00f460 con 0x7f1f5c103e60 2024-05-11T13:43:59.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.607+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f38005330 con 0x7f1f5c103e60 2024-05-11T13:43:59.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.611+0000 7f1f58c00700 1 -- 172.21.15.63:0/376917169 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1f4c02d070 con 0x7f1f5c103e60 2024-05-11T13:43:59.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.613+0000 7f1f58c00700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1f400b2aa0 0x7f1f400b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:43:59.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.614+0000 7f1f58c00700 1 -- 172.21.15.63:0/376917169 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f1f4c0e75b0 con 0x7f1f5c103e60 2024-05-11T13:43:59.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.614+0000 7f1f5b400700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1f400b2aa0 0x7f1f400b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:43:59.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.615+0000 7f1f5b400700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1f400b2aa0 0x7f1f400b4f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c19a9a0 tx=0x7f1f44009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:43:59.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:43:59.620+0000 7f1f58c00700 1 -- 172.21.15.63:0/376917169 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f4c034080 con 0x7f1f5c103e60 2024-05-11T13:44:00.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.097+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f38005620 con 0x7f1f5c103e60 2024-05-11T13:44:00.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.100+0000 7f1f58c00700 1 -- 172.21.15.63:0/376917169 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11945 (secure 0 0 0) 0x7f1f4c013070 con 0x7f1f5c103e60 2024-05-11T13:44:00.092 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:44:00.092 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:43:54.240202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34451,34472,34427,34403,24472]},"id":1}]} 2024-05-11T13:44:00.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.103+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1f400b2aa0 msgr2=0x7f1f400b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:00.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.103+0000 7f1f61000700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1f400b2aa0 0x7f1f400b4f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c19a9a0 tx=0x7f1f44009260 comp rx=0 tx=0).stop 2024-05-11T13:44:00.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.103+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c103e60 msgr2=0x7f1f5c19fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:00.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.103+0000 7f1f61000700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c103e60 0x7f1f5c19fbe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000c00 tx=0x7f1f4c01ca40 comp rx=0 tx=0).stop 2024-05-11T13:44:00.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 shutdown_connections 2024-05-11T13:44:00.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1f400b2aa0 0x7f1f400b4f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:00.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f5c109550 0x7f1f5c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:00.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f5c1034a0 0x7f1f5c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:00.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 --2- 172.21.15.63:0/376917169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c103e60 0x7f1f5c19fbe0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:00.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 >> 172.21.15.63:0/376917169 conn(0x7f1f5c0fefd0 msgr2=0x7f1f5c1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:00.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 shutdown_connections 2024-05-11T13:44:00.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:00.104+0000 7f1f61000700 1 -- 172.21.15.63:0/376917169 wait complete. 2024-05-11T13:44:00.098 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 63 2024-05-11T13:44:00.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:00 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/376917169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:44:00.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:00 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/376917169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:44:00.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:00 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/376917169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:44:00.938 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 91.4 --await-for 42.0' 2024-05-11T13:44:00.938 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 91.4 --await-for 42.0 2024-05-11T13:44:01.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 155.0 s, 414.2 tps, lat 234.775 ms stddev 47.451 2024-05-11T13:44:01.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:01 smithi063 ceph-mon[29809]: pgmap v315: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-05-11T13:44:01.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:01 smithi071 ceph-mon[33293]: pgmap v315: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-05-11T13:44:01.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:01 smithi165 ceph-mon[33235]: pgmap v315: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-05-11T13:44:01.598 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:44:02.379 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:02 smithi063 ceph-mon[29809]: pgmap v316: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-05-11T13:44:02.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:02 smithi071 ceph-mon[33293]: pgmap v316: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-05-11T13:44:02.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:02 smithi165 ceph-mon[33235]: pgmap v316: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-05-11T13:44:04.499 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:04 smithi063 ceph-mon[29809]: pgmap v317: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 8.4 MiB/s wr, 807 op/s 2024-05-11T13:44:04.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:04 smithi071 ceph-mon[33293]: pgmap v317: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 8.4 MiB/s wr, 807 op/s 2024-05-11T13:44:04.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.487+0000 7f4c5d000700 1 -- 172.21.15.63:0/973737394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c5810a550 msgr2=0x7f4c5810a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.487+0000 7f4c5d000700 1 --2- 172.21.15.63:0/973737394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c5810a550 0x7f4c5810a930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b0a0 tx=0x7f4c4800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.487+0000 7f4c5d000700 1 -- 172.21.15.63:0/973737394 shutdown_connections 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.487+0000 7f4c5d000700 1 --2- 172.21.15.63:0/973737394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c581166a0 0x7f4c58118a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.487+0000 7f4c5d000700 1 --2- 172.21.15.63:0/973737394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c5810a550 0x7f4c5810a930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.487+0000 7f4c5d000700 1 --2- 172.21.15.63:0/973737394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 0x7f4c58116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.487+0000 7f4c5d000700 1 -- 172.21.15.63:0/973737394 >> 172.21.15.63:0/973737394 conn(0x7f4c5806e6c0 msgr2=0x7f4c5806ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.488+0000 7f4c5d000700 1 -- 172.21.15.63:0/973737394 shutdown_connections 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.488+0000 7f4c5d000700 1 -- 172.21.15.63:0/973737394 wait complete. 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.489+0000 7f4c5d000700 1 Processor -- start 2024-05-11T13:44:04.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.490+0000 7f4c5d000700 1 -- start start 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.490+0000 7f4c5d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5810a550 0x7f4c58115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.490+0000 7f4c5d000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 0x7f4c58112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.490+0000 7f4c5d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c581166a0 0x7f4c58113530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.490+0000 7f4c5d000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5811e0e0 con 0x7f4c5810a550 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.490+0000 7f4c5d000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5811df60 con 0x7f4c581166a0 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.490+0000 7f4c5d000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5811e260 con 0x7f4c5810ae70 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.491+0000 7f4c4d400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 0x7f4c58112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.491+0000 7f4c4d400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 0x7f4c58112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:48972/0 (socket says 172.21.15.63:48972) 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.491+0000 7f4c4d400700 1 -- 172.21.15.63:0/3093087014 learned_addr learned my addr 172.21.15.63:0/3093087014 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.492+0000 7f4c56800700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c581166a0 0x7f4c58113530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.492+0000 7f4c55e00700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5810a550 0x7f4c58115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.492+0000 7f4c4d400700 1 -- 172.21.15.63:0/3093087014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c581166a0 msgr2=0x7f4c58113530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.492+0000 7f4c4d400700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c581166a0 0x7f4c58113530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.492+0000 7f4c4d400700 1 -- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5810a550 msgr2=0x7f4c58115ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.492+0000 7f4c4d400700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5810a550 0x7f4c58115ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.492+0000 7f4c4d400700 1 -- 172.21.15.63:0/3093087014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c58113d70 con 0x7f4c5810ae70 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.493+0000 7f4c4d400700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 0x7f4c58112ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000e9c0 tx=0x7f4c4000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.494+0000 7f4c4f400700 1 -- 172.21.15.63:0/3093087014 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000cbd0 con 0x7f4c5810ae70 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.494+0000 7f4c4f400700 1 -- 172.21.15.63:0/3093087014 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c4000cd70 con 0x7f4c5810ae70 2024-05-11T13:44:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.494+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c581a35b0 con 0x7f4c5810ae70 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.495+0000 7f4c4f400700 1 -- 172.21.15.63:0/3093087014 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40005610 con 0x7f4c5810ae70 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.495+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c581a3af0 con 0x7f4c5810ae70 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.505+0000 7f4c4f400700 1 -- 172.21.15.63:0/3093087014 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4c40021440 con 0x7f4c5810ae70 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.507+0000 7f4c4f400700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4c3c0b2bf0 0x7f4c3c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.508+0000 7f4c4f400700 1 -- 172.21.15.63:0/3093087014 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f4c400e2e90 con 0x7f4c5810ae70 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.510+0000 7f4c55e00700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4c3c0b2bf0 0x7f4c3c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.512+0000 7f4c55e00700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4c3c0b2bf0 0x7f4c3c0b50b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009e60 tx=0x7f4c4800b740 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.513+0000 7f4c67902b80 1 Processor -- start 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.513+0000 7f4c67902b80 1 -- start start 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.514+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a811d060 0x5650a811d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.514+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8117ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.514+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8123e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.514+0000 7f4c67902b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5650a8069bb0 con 0x5650a8117ff0 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.514+0000 7f4c67902b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5650a80078f0 con 0x5650a811d060 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.514+0000 7f4c67902b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5650a8065b70 con 0x5650a8117630 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.514+0000 7f4c55e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a811d060 0x5650a811d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.515+0000 7f4c55e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a811d060 0x5650a811d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:53654/0 (socket says 172.21.15.63:53654) 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.515+0000 7f4c55e00700 1 -- 172.21.15.63:0/588778056 learned_addr learned my addr 172.21.15.63:0/588778056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.515+0000 7f4c4d400700 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8117ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.515+0000 7f4c55e00700 1 -- 172.21.15.63:0/588778056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 msgr2=0x5650a8117ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.515+0000 7f4c55e00700 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8117ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.515+0000 7f4c55e00700 1 -- 172.21.15.63:0/588778056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 msgr2=0x5650a8123e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.516+0000 7f4c56800700 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8123e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.517+0000 7f4c55e00700 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8123e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.517+0000 7f4c55e00700 1 -- 172.21.15.63:0/588778056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a7fb6a60 con 0x5650a811d060 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.517+0000 7f4c56800700 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8123e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.518+0000 7f4c55e00700 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a811d060 0x5650a811d440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800bd40 tx=0x7f4c4801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad03c860f2a44db3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.518+0000 7f4c47400700 1 -- 172.21.15.63:0/588778056 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48022070 con 0x5650a811d060 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.518+0000 7f4c47400700 1 -- 172.21.15.63:0/588778056 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c4800bd70 con 0x5650a811d060 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.519+0000 7f4c47400700 1 -- 172.21.15.63:0/588778056 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48029850 con 0x5650a811d060 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.520+0000 7f4c67902b80 1 -- 172.21.15.63:0/588778056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a811d060 msgr2=0x5650a811d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.520+0000 7f4c67902b80 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a811d060 0x5650a811d440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800bd40 tx=0x7f4c4801c000 comp rx=0 tx=0).stop 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.520+0000 7f4c67902b80 1 -- 172.21.15.63:0/588778056 shutdown_connections 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.520+0000 7f4c67902b80 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8123e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.520+0000 7f4c67902b80 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a811d060 0x5650a811d440 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800bd40 tx=0x7f4c4801c000 comp rx=0 tx=0).stop 2024-05-11T13:44:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.520+0000 7f4c67902b80 1 --2- 172.21.15.63:0/588778056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8117ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.520+0000 7f4c67902b80 1 -- 172.21.15.63:0/588778056 >> 172.21.15.63:0/588778056 conn(0x5650a8045cf0 msgr2=0x5650a8111f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.522+0000 7f4c67902b80 1 -- 172.21.15.63:0/588778056 shutdown_connections 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.522+0000 7f4c67902b80 1 -- 172.21.15.63:0/588778056 wait complete. 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.523+0000 7f4c67902b80 1 -- start start 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.523+0000 7f4c67902b80 1 Processor -- start 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.524+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8123d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.524+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8121050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.524+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8121a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.524+0000 7f4c67902b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5650a8124360 con 0x5650a8117ff0 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.524+0000 7f4c67902b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5650a7fcfcb0 con 0x5650a81215e0 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.524+0000 7f4c67902b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5650a81244e0 con 0x5650a8117630 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.525+0000 7f4c67902b80 10 client.0 _ll_register_callbacks cb 0x5650a8010e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:44:04.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.525+0000 7f4c56800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8121a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.525+0000 7f4c56800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8121a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:53658/0 (socket says 172.21.15.63:53658) 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.525+0000 7f4c56800700 1 -- 172.21.15.63:0/1901137525 learned_addr learned my addr 172.21.15.63:0/1901137525 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.526+0000 7f4c56800700 1 -- 172.21.15.63:0/1901137525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 msgr2=0x5650a8123d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.526+0000 7f4c56800700 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8123d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.526+0000 7f4c56800700 1 -- 172.21.15.63:0/1901137525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 msgr2=0x5650a8121050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.526+0000 7f4c56800700 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8121050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.526+0000 7f4c56800700 1 -- 172.21.15.63:0/1901137525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a802f5b0 con 0x5650a81215e0 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.528+0000 7f4c56800700 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8121a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c50011690 tx=0x7f4c50011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c47290d16cbcad server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.528+0000 7f4c3be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.530+0000 7f4c3be00700 1 -- 172.21.15.63:0/1901137525 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50012860 con 0x5650a81215e0 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.530+0000 7f4c3be00700 1 -- 172.21.15.63:0/1901137525 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f4c50012a00 con 0x5650a81215e0 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.530+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4800bd70 con 0x5650a81215e0 2024-05-11T13:44:04.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.531+0000 7f4c3be00700 1 -- 172.21.15.63:0/1901137525 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f4c50012c00 con 0x5650a81215e0 2024-05-11T13:44:04.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.531+0000 7f4c67902b80 10 client.35450 fetch_fsmap learned FSMap version 63 2024-05-11T13:44:04.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.531+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5650a812f240 con 0x5650a81215e0 2024-05-11T13:44:04.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c3be00700 1 -- 172.21.15.63:0/1901137525 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c4800bd70 con 0x5650a81215e0 2024-05-11T13:44:04.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c3be00700 1 -- 172.21.15.63:0/1901137525 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f4c5001a3f0 con 0x5650a81215e0 2024-05-11T13:44:04.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c67902b80 10 client.35450 fetch_fsmap finished waiting for FSMap version 63 2024-05-11T13:44:04.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c67902b80 10 client.35450 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c67902b80 20 client.35450 populate_metadata read hostname 'smithi063' 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a8131880 0x5650a8133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c67902b80 4 client.35450 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.532+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a80e4340 con 0x5650a8131880 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.535+0000 7f4c55e00700 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a8131880 0x5650a8133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.535+0000 7f4c55e00700 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a8131880 0x5650a8133cb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c6d4ee0a72c291 server_cookie=2b2b94aa47aca6b8 in_seq=0 out_seq=0 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.536+0000 7f4c3be00700 10 client.35450 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.538+0000 7f4c3be00700 1 -- 172.21.15.63:0/1901137525 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650a80e4340 con 0x5650a8131880 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.538+0000 7f4c3be00700 10 client.35450 handle_command_reply: tid=0 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.540+0000 7f4c67902b80 1 client.35450 shutdown 2024-05-11T13:44:04.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.541+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 msgr2=0x5650a8121a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.541+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8121a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c50011690 tx=0x7f4c50011a50 comp rx=0 tx=0).stop 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.541+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 shutdown_connections 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.542+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a8131880 0x5650a8133cb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.542+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8121a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.542+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8123d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.542+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1901137525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8121050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.542+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 >> 172.21.15.63:0/1901137525 conn(0x5650a8045cf0 msgr2=0x5650a810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.542+0000 7f4c3be00700 0 client.35450 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.544+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 shutdown_connections 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.544+0000 7f4c67902b80 1 -- 172.21.15.63:0/1901137525 wait complete. 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.544+0000 7f4c67902b80 20 client.35450 trim_cache size 0 max 16384 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.587+0000 7f4c67902b80 1 Processor -- start 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.587+0000 7f4c67902b80 1 -- start start 2024-05-11T13:44:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.588+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a8117630 0x5650a811eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.588+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117ff0 0x5650a812d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a81215e0 0x5650a8111db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c67902b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c581c6b60 con 0x5650a81215e0 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c67902b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c581c87c0 con 0x5650a8117630 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c67902b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4c581c7230 con 0x5650a8117ff0 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c55e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a8117630 0x5650a811eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c55e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a8117630 0x5650a811eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:53664/0 (socket says 172.21.15.63:53664) 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c55e00700 1 -- 172.21.15.63:0/1676354097 learned_addr learned my addr 172.21.15.63:0/1676354097 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c56800700 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a81215e0 0x5650a8111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.589+0000 7f4c4d400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117ff0 0x5650a812d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.591+0000 7f4c4d400700 1 -- 172.21.15.63:0/1676354097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a8117630 msgr2=0x5650a811eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.591+0000 7f4c4d400700 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a8117630 0x5650a811eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.591+0000 7f4c4d400700 1 -- 172.21.15.63:0/1676354097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a81215e0 msgr2=0x5650a8111db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.591+0000 7f4c4d400700 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a81215e0 0x5650a8111db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.591+0000 7f4c4d400700 1 -- 172.21.15.63:0/1676354097 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a810eb20 con 0x5650a8117ff0 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.591+0000 7f4c55e00700 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a8117630 0x5650a811eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.591+0000 7f4c56800700 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a81215e0 0x5650a8111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.592+0000 7f4c4d400700 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117ff0 0x5650a812d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c400e0190 tx=0x7f4c400a9790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c88ca233f14194 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.592+0000 7f4c46a00700 1 -- 172.21.15.63:0/1676354097 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400c0070 con 0x5650a8117ff0 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.593+0000 7f4c46a00700 1 -- 172.21.15.63:0/1676354097 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c400aa420 con 0x5650a8117ff0 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.593+0000 7f4c46a00700 1 -- 172.21.15.63:0/1676354097 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400a9f80 con 0x5650a8117ff0 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.594+0000 7f4c67902b80 1 -- 172.21.15.63:0/1676354097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117ff0 msgr2=0x5650a812d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.594+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117ff0 0x5650a812d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c400e0190 tx=0x7f4c400a9790 comp rx=0 tx=0).stop 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.594+0000 7f4c67902b80 1 -- 172.21.15.63:0/1676354097 shutdown_connections 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.594+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a81215e0 0x5650a8111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.594+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a8117630 0x5650a811eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.594+0000 7f4c67902b80 1 --2- 172.21.15.63:0/1676354097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117ff0 0x5650a812d3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.594+0000 7f4c67902b80 1 -- 172.21.15.63:0/1676354097 >> 172.21.15.63:0/1676354097 conn(0x5650a8045cf0 msgr2=0x5650a8116eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.595+0000 7f4c67902b80 1 -- 172.21.15.63:0/1676354097 shutdown_connections 2024-05-11T13:44:04.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.595+0000 7f4c67902b80 1 -- 172.21.15.63:0/1676354097 wait complete. 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.596+0000 7f4c67902b80 1 -- start start 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.596+0000 7f4c67902b80 1 Processor -- start 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.597+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8129ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.597+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8124730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c67902b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8124c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c67902b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c581c8030 con 0x5650a8117ff0 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c67902b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c581c7900 con 0x5650a81215e0 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c67902b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4c581aaaa0 con 0x5650a8117630 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c67902b80 10 client.0 _ll_register_callbacks cb 0x5650a8010e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c56800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8124c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c56800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8124c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:53678/0 (socket says 172.21.15.63:53678) 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.598+0000 7f4c56800700 1 -- 172.21.15.63:0/2764695640 learned_addr learned my addr 172.21.15.63:0/2764695640 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.599+0000 7f4c56800700 1 -- 172.21.15.63:0/2764695640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 msgr2=0x5650a8129ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.599+0000 7f4c56800700 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8129ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.599+0000 7f4c56800700 1 -- 172.21.15.63:0/2764695640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 msgr2=0x5650a8124730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:04.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.599+0000 7f4c56800700 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8124730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.599+0000 7f4c56800700 1 -- 172.21.15.63:0/2764695640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a8125320 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.600+0000 7f4c56800700 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8124c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c50011690 tx=0x7f4c5001a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318c09877b5bcdca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.601+0000 7f4c3be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.601+0000 7f4c3be00700 1 -- 172.21.15.63:0/2764695640 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50012870 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.601+0000 7f4c3be00700 1 -- 172.21.15.63:0/2764695640 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f4c500050b0 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.601+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c400aa420 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.601+0000 7f4c3be00700 1 -- 172.21.15.63:0/2764695640 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f4c50012a10 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.601+0000 7f4c67902b80 10 client.35456 fetch_fsmap learned FSMap version 63 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.602+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5650a819d010 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.602+0000 7f4c3be00700 1 -- 172.21.15.63:0/2764695640 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c400aa420 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.603+0000 7f4c3be00700 1 -- 172.21.15.63:0/2764695640 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f4c50003bf0 con 0x5650a81215e0 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.604+0000 7f4c67902b80 10 client.35456 fetch_fsmap finished waiting for FSMap version 63 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.604+0000 7f4c67902b80 10 client.35456 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.604+0000 7f4c67902b80 20 client.35456 populate_metadata read hostname 'smithi063' 2024-05-11T13:44:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.605+0000 7f4c67902b80 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a819d840 0x5650a819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.605+0000 7f4c67902b80 4 client.35456 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.4, "await_for": 42.0}] 2024-05-11T13:44:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.605+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.4, "await_for": 42.0}) -- 0x5650a8077590 con 0x5650a819d840 2024-05-11T13:44:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.605+0000 7f4c55e00700 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a819d840 0x5650a819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.606+0000 7f4c55e00700 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a819d840 0x5650a819fcc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52e3d1d48a72e30 server_cookie=26af8b0c82e64218 in_seq=0 out_seq=0 2024-05-11T13:44:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.606+0000 7f4c3be00700 10 client.35456 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:04.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:04 smithi165 ceph-mon[33235]: pgmap v317: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 8.4 MiB/s wr, 807 op/s 2024-05-11T13:44:04.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.910+0000 7f4c3be00700 1 -- 172.21.15.63:0/2764695640 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5650a8077590 con 0x5650a819d840 2024-05-11T13:44:04.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.910+0000 7f4c3be00700 10 client.35456 handle_command_reply: tid=0 2024-05-11T13:44:04.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.910+0000 7f4c67902b80 1 client.35456 shutdown 2024-05-11T13:44:04.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 msgr2=0x5650a8124c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8124c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c50011690 tx=0x7f4c5001a6d0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 shutdown_connections 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5650a819d840 0x5650a819fcc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a81215e0 0x5650a8124c70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650a8117630 0x5650a8129ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 --2- 172.21.15.63:0/2764695640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5650a8117ff0 0x5650a8124730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c3be00700 0 client.35456 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 >> 172.21.15.63:0/2764695640 conn(0x5650a8045cf0 msgr2=0x5650a810c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 shutdown_connections 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.911+0000 7f4c67902b80 1 -- 172.21.15.63:0/2764695640 wait complete. 2024-05-11T13:44:04.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.912+0000 7f4c67902b80 20 client.35456 trim_cache size 0 max 16384 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 63, 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 30, 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "7ca62e7": { 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 30, 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:44:04.904 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 91.4, 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:04.905 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:44:04.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4c3c0b2bf0 msgr2=0x7f4c3c0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4c3c0b2bf0 0x7f4c3c0b50b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009e60 tx=0x7f4c4800b740 comp rx=0 tx=0).stop 2024-05-11T13:44:04.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 msgr2=0x7f4c58112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:04.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 0x7f4c58112ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000e9c0 tx=0x7f4c4000ed80 comp rx=0 tx=0).stop 2024-05-11T13:44:04.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 shutdown_connections 2024-05-11T13:44:04.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4c3c0b2bf0 0x7f4c3c0b50b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c581166a0 0x7f4c58113530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5810a550 0x7f4c58115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 --2- 172.21.15.63:0/3093087014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c5810ae70 0x7f4c58112ff0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:04.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 >> 172.21.15.63:0/3093087014 conn(0x7f4c5806e6c0 msgr2=0x7f4c58073280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:04.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 shutdown_connections 2024-05-11T13:44:04.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:04.913+0000 7f4c5d000700 1 -- 172.21.15.63:0/3093087014 wait complete. 2024-05-11T13:44:05.686 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7ca62e7', quiesce took 12.6 seconds. Will release after: 31.4 2024-05-11T13:44:06.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 160.0 s, 355.5 tps, lat 214.520 ms stddev 61.683 2024-05-11T13:44:06.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:06 smithi165 ceph-mon[33235]: pgmap v318: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.7 MiB/s wr, 551 op/s 2024-05-11T13:44:06.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:06 smithi063 ceph-mon[29809]: pgmap v318: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.7 MiB/s wr, 551 op/s 2024-05-11T13:44:06.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:06 smithi071 ceph-mon[33293]: pgmap v318: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.7 MiB/s wr, 551 op/s 2024-05-11T13:44:08.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:08 smithi063 ceph-mon[29809]: pgmap v319: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 643 op/s 2024-05-11T13:44:08.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:08 smithi071 ceph-mon[33293]: pgmap v319: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 643 op/s 2024-05-11T13:44:08.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:08 smithi165 ceph-mon[33235]: pgmap v319: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 643 op/s 2024-05-11T13:44:10.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:10 smithi063 ceph-mon[29809]: osdmap e100: 12 total, 12 up, 12 in 2024-05-11T13:44:10.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:10 smithi063 ceph-mon[29809]: pgmap v321: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-05-11T13:44:10.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:10 smithi071 ceph-mon[33293]: osdmap e100: 12 total, 12 up, 12 in 2024-05-11T13:44:10.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:10 smithi071 ceph-mon[33293]: pgmap v321: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-05-11T13:44:10.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:10 smithi165 ceph-mon[33235]: osdmap e100: 12 total, 12 up, 12 in 2024-05-11T13:44:10.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:10 smithi165 ceph-mon[33235]: pgmap v321: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-05-11T13:44:11.147 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 165.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:44:12.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:12 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:44:12.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:12 smithi165 ceph-mon[33235]: pgmap v322: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-05-11T13:44:12.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:12 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:44:12.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:12 smithi063 ceph-mon[29809]: pgmap v322: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-05-11T13:44:12.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:12 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:44:12.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:12 smithi071 ceph-mon[33293]: pgmap v322: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-05-11T13:44:14.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:14 smithi063 ceph-mon[29809]: pgmap v323: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-11T13:44:14.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:14 smithi071 ceph-mon[33293]: pgmap v323: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-11T13:44:14.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:14 smithi165 ceph-mon[33235]: pgmap v323: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-11T13:44:16.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 170.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:44:16.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:16 smithi063 ceph-mon[29809]: osdmap e101: 12 total, 12 up, 12 in 2024-05-11T13:44:16.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:16 smithi071 ceph-mon[33293]: osdmap e101: 12 total, 12 up, 12 in 2024-05-11T13:44:16.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:16 smithi165 ceph-mon[33235]: osdmap e101: 12 total, 12 up, 12 in 2024-05-11T13:44:17.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:17 smithi063 ceph-mon[29809]: pgmap v325: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 114 KiB/s wr, 20 op/s 2024-05-11T13:44:17.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:17 smithi071 ceph-mon[33293]: pgmap v325: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 114 KiB/s wr, 20 op/s 2024-05-11T13:44:17.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:17 smithi165 ceph-mon[33235]: pgmap v325: 129 pgs: 129 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 114 KiB/s wr, 20 op/s 2024-05-11T13:44:18.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:18 smithi063 ceph-mon[29809]: pgmap v326: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 840 KiB/s wr, 113 op/s 2024-05-11T13:44:18.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:18 smithi071 ceph-mon[33293]: pgmap v326: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 840 KiB/s wr, 113 op/s 2024-05-11T13:44:18.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:18 smithi165 ceph-mon[33235]: pgmap v326: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 840 KiB/s wr, 113 op/s 2024-05-11T13:44:20.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:20 smithi063 ceph-mon[29809]: pgmap v327: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 748 KiB/s wr, 101 op/s 2024-05-11T13:44:20.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:20 smithi071 ceph-mon[33293]: pgmap v327: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 748 KiB/s wr, 101 op/s 2024-05-11T13:44:20.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:20 smithi165 ceph-mon[33235]: pgmap v327: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 748 KiB/s wr, 101 op/s 2024-05-11T13:44:21.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 175.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:44:21.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:21 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:44:21.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:21 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:44:21.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:21 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:44:22.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:22 smithi063 ceph-mon[29809]: pgmap v328: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 748 KiB/s wr, 101 op/s 2024-05-11T13:44:22.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:22 smithi071 ceph-mon[33293]: pgmap v328: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 748 KiB/s wr, 101 op/s 2024-05-11T13:44:22.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:22 smithi165 ceph-mon[33235]: pgmap v328: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 748 KiB/s wr, 101 op/s 2024-05-11T13:44:24.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:24 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:24 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:24 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:24 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:24 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:24 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:24 smithi165 ceph-mon[33235]: pgmap v329: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 716 KiB/s wr, 97 op/s 2024-05-11T13:44:24.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:24 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:24 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:24 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:24 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:24 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:24 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.926 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:24 smithi063 ceph-mon[29809]: pgmap v329: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 716 KiB/s wr, 97 op/s 2024-05-11T13:44:24.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:24 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:24 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:24 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:24 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:24 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:24 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:24.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:24 smithi071 ceph-mon[33293]: pgmap v329: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 716 KiB/s wr, 97 op/s 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi071 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:44:25.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:25 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi071 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:44:25.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:25 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:25.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.945 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi071 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:44:25.946 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:44:25.947 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:25 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:44:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 180.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:44:26.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:26 smithi165 ceph-mon[33235]: pgmap v330: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 641 KiB/s wr, 87 op/s 2024-05-11T13:44:26.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:26 smithi063 ceph-mon[29809]: pgmap v330: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 641 KiB/s wr, 87 op/s 2024-05-11T13:44:26.944 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:26 smithi071 ceph-mon[33293]: pgmap v330: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 641 KiB/s wr, 87 op/s 2024-05-11T13:44:28.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:28 smithi063 ceph-mon[29809]: pgmap v331: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 602 KiB/s wr, 86 op/s 2024-05-11T13:44:28.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:28 smithi071 ceph-mon[33293]: pgmap v331: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 602 KiB/s wr, 86 op/s 2024-05-11T13:44:28.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:28 smithi165 ceph-mon[33235]: pgmap v331: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 602 KiB/s wr, 86 op/s 2024-05-11T13:44:30.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:30 smithi063 ceph-mon[29809]: pgmap v332: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-11T13:44:30.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:30 smithi071 ceph-mon[33293]: pgmap v332: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-11T13:44:30.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:30 smithi165 ceph-mon[33235]: pgmap v332: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-11T13:44:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 185.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:44:32.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:32 smithi063 ceph-mon[29809]: pgmap v333: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-11T13:44:32.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:32 smithi071 ceph-mon[33293]: pgmap v333: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-11T13:44:32.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:32 smithi165 ceph-mon[33235]: pgmap v333: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-11T13:44:34.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:34 smithi063 ceph-mon[29809]: pgmap v334: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-05-11T13:44:34.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:34 smithi071 ceph-mon[33293]: pgmap v334: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-05-11T13:44:34.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:34 smithi165 ceph-mon[33235]: pgmap v334: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-05-11T13:44:36.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 190.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:44:36.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:36 smithi165 ceph-mon[33235]: pgmap v335: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2024-05-11T13:44:36.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:36 smithi063 ceph-mon[29809]: pgmap v335: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2024-05-11T13:44:36.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:36 smithi071 ceph-mon[33293]: pgmap v335: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2024-05-11T13:44:37.087 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:44:37.558 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:44:38.574 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:38 smithi063 ceph-mon[29809]: pgmap v336: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-05-11T13:44:38.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:38 smithi071 ceph-mon[33293]: pgmap v336: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-05-11T13:44:38.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:38 smithi165 ceph-mon[33235]: pgmap v336: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-05-11T13:44:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 -- 172.21.15.63:0/1504142009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50114720 msgr2=0x7fdf50116b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 --2- 172.21.15.63:0/1504142009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50114720 0x7fdf50116b10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009990 tx=0x7fdf48009ca0 comp rx=0 tx=0).stop 2024-05-11T13:44:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 -- 172.21.15.63:0/1504142009 shutdown_connections 2024-05-11T13:44:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 --2- 172.21.15.63:0/1504142009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50114720 0x7fdf50116b10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 --2- 172.21.15.63:0/1504142009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf50075780 0x7fdf50075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 --2- 172.21.15.63:0/1504142009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf500760a0 0x7fdf501141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 -- 172.21.15.63:0/1504142009 >> 172.21.15.63:0/1504142009 conn(0x7fdf5007a6b0 msgr2=0x7fdf5007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 -- 172.21.15.63:0/1504142009 shutdown_connections 2024-05-11T13:44:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.997+0000 7fdf56800700 1 -- 172.21.15.63:0/1504142009 wait complete. 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 Processor -- start 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 -- start start 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50075780 0x7fdf501a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf500760a0 0x7fdf501a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf50114720 0x7fdf501a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50119ba0 con 0x7fdf50075780 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50119a20 con 0x7fdf500760a0 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.998+0000 7fdf56800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50119d20 con 0x7fdf50114720 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50075780 0x7fdf501a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50075780 0x7fdf501a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43392/0 (socket says 172.21.15.63:43392) 2024-05-11T13:44:39.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 -- 172.21.15.63:0/3777541981 learned_addr learned my addr 172.21.15.63:0/3777541981 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:39.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf46a00700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf500760a0 0x7fdf501a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:39.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4fe00700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf50114720 0x7fdf501a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 -- 172.21.15.63:0/3777541981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf50114720 msgr2=0x7fdf501a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf50114720 0x7fdf501a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 -- 172.21.15.63:0/3777541981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf500760a0 msgr2=0x7fdf501a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf500760a0 0x7fdf501a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:39.999+0000 7fdf4f400700 1 -- 172.21.15.63:0/3777541981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf501a5020 con 0x7fdf50075780 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.000+0000 7fdf4f400700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50075780 0x7fdf501a0010 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdf40007e20 tx=0x7fdf4000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.000+0000 7fdf4cc00700 1 -- 172.21.15.63:0/3777541981 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40016020 con 0x7fdf50075780 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.000+0000 7fdf4cc00700 1 -- 172.21.15.63:0/3777541981 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf4000ae00 con 0x7fdf50075780 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.000+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf501a5310 con 0x7fdf50075780 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.000+0000 7fdf4cc00700 1 -- 172.21.15.63:0/3777541981 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40011670 con 0x7fdf50075780 2024-05-11T13:44:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.000+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf501acb90 con 0x7fdf50075780 2024-05-11T13:44:39.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.002+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5006a240 con 0x7fdf50075780 2024-05-11T13:44:39.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.004+0000 7fdf4cc00700 1 -- 172.21.15.63:0/3777541981 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdf40019020 con 0x7fdf50075780 2024-05-11T13:44:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.007+0000 7fdf4cc00700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fdf340b29f0 0x7fdf340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.007+0000 7fdf4cc00700 1 -- 172.21.15.63:0/3777541981 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fdf400e2b00 con 0x7fdf50075780 2024-05-11T13:44:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.007+0000 7fdf46a00700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fdf340b29f0 0x7fdf340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.008+0000 7fdf46a00700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fdf340b29f0 0x7fdf340b4eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf38005950 tx=0x7fdf380058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:40.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.010+0000 7fdf4cc00700 1 -- 172.21.15.63:0/3777541981 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf400aa1a0 con 0x7fdf50075780 2024-05-11T13:44:40.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.314+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf501a1370 con 0x7fdf50075780 2024-05-11T13:44:40.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.315+0000 7fdf4cc00700 1 -- 172.21.15.63:0/3777541981 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+11943 (secure 0 0 0) 0x7fdf400b2220 con 0x7fdf50075780 2024-05-11T13:44:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:44:40.307 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:44:11.471748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:44:40.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fdf340b29f0 msgr2=0x7fdf340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fdf340b29f0 0x7fdf340b4eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf38005950 tx=0x7fdf380058e0 comp rx=0 tx=0).stop 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50075780 msgr2=0x7fdf501a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50075780 0x7fdf501a0010 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdf40007e20 tx=0x7fdf4000a340 comp rx=0 tx=0).stop 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 shutdown_connections 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fdf340b29f0 0x7fdf340b4eb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf50114720 0x7fdf501a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf50075780 0x7fdf501a0010 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 --2- 172.21.15.63:0/3777541981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf500760a0 0x7fdf501a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 >> 172.21.15.63:0/3777541981 conn(0x7fdf5007a6b0 msgr2=0x7fdf501101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:40.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 shutdown_connections 2024-05-11T13:44:40.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:40.319+0000 7fdf56800700 1 -- 172.21.15.63:0/3777541981 wait complete. 2024-05-11T13:44:40.312 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 64 2024-05-11T13:44:40.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:40 smithi063 ceph-mon[29809]: pgmap v337: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-11T13:44:40.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:40 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3777541981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:44:40.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:40 smithi071 ceph-mon[33293]: pgmap v337: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-11T13:44:40.695 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:40 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3777541981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:44:40.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:40 smithi165 ceph-mon[33235]: pgmap v337: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-11T13:44:40.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:40 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3777541981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:44:41.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 195.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:44:41.245 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 7ca62e7 --release --await' 2024-05-11T13:44:41.246 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 7ca62e7 --release --await 2024-05-11T13:44:41.704 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:44:42.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:42 smithi063 ceph-mon[29809]: pgmap v338: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-11T13:44:42.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:42 smithi071 ceph-mon[33293]: pgmap v338: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-11T13:44:42.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:42 smithi165 ceph-mon[33235]: pgmap v338: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-11T13:44:44.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 -- 172.21.15.63:0/4100297744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 msgr2=0x7f6830103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 --2- 172.21.15.63:0/4100297744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830103920 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f681c0099c0 tx=0x7f681c009cd0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 -- 172.21.15.63:0/4100297744 shutdown_connections 2024-05-11T13:44:44.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 --2- 172.21.15.63:0/4100297744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6830103e60 0x7f6830110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 --2- 172.21.15.63:0/4100297744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830109550 0x7f6830109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 --2- 172.21.15.63:0/4100297744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830103920 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 -- 172.21.15.63:0/4100297744 >> 172.21.15.63:0/4100297744 conn(0x7f68300fefd0 msgr2=0x7f68301013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:44.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 -- 172.21.15.63:0/4100297744 shutdown_connections 2024-05-11T13:44:44.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.214+0000 7f682fe00700 1 -- 172.21.15.63:0/4100297744 wait complete. 2024-05-11T13:44:44.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.215+0000 7f682fe00700 1 Processor -- start 2024-05-11T13:44:44.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682fe00700 1 -- start start 2024-05-11T13:44:44.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830103e60 0x7f6830075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6830109550 0x7f68300762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6830113560 con 0x7f68301034a0 2024-05-11T13:44:44.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682fe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68301133e0 con 0x7f6830103e60 2024-05-11T13:44:44.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682fe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68301136e0 con 0x7f6830109550 2024-05-11T13:44:44.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43418/0 (socket says 172.21.15.63:43418) 2024-05-11T13:44:44.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.216+0000 7f682ea00700 1 -- 172.21.15.63:0/1468603743 learned_addr learned my addr 172.21.15.63:0/1468603743 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:44.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682f400700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6830109550 0x7f68300762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682e000700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830103e60 0x7f6830075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682ea00700 1 -- 172.21.15.63:0/1468603743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6830109550 msgr2=0x7f68300762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682ea00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6830109550 0x7f68300762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682ea00700 1 -- 172.21.15.63:0/1468603743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830103e60 msgr2=0x7f6830075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682ea00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830103e60 0x7f6830075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682ea00700 1 -- 172.21.15.63:0/1468603743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68301a7f40 con 0x7f68301034a0 2024-05-11T13:44:44.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682e000700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830103e60 0x7f6830075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:44:44.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f682ea00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830075850 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9a0 tx=0x7f682000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:44.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.217+0000 7f6827400700 1 -- 172.21.15.63:0/1468603743 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682000cbd0 con 0x7f68301034a0 2024-05-11T13:44:44.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.218+0000 7f6827400700 1 -- 172.21.15.63:0/1468603743 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f682000cd70 con 0x7f68301034a0 2024-05-11T13:44:44.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.218+0000 7f6827400700 1 -- 172.21.15.63:0/1468603743 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820018680 con 0x7f68301034a0 2024-05-11T13:44:44.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.218+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68301a8230 con 0x7f68301034a0 2024-05-11T13:44:44.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.218+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301a87c0 con 0x7f68301034a0 2024-05-11T13:44:44.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.221+0000 7f6827400700 1 -- 172.21.15.63:0/1468603743 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6820010950 con 0x7f68301034a0 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.223+0000 7f6827400700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f68140b2a70 0x7f68140b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.223+0000 7f6827400700 1 -- 172.21.15.63:0/1468603743 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f68200e2c90 con 0x7f68301034a0 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.223+0000 7f6840625b80 1 Processor -- start 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.223+0000 7f682e000700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f68140b2a70 0x7f68140b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.223+0000 7f6840625b80 1 -- start start 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f6840625b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75070 0x561afcd75450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f6840625b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75990 0x561afcd6f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f6840625b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd70020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f6840625b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561afccc1b20 con 0x561afcd75070 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f6840625b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561afcc5f860 con 0x561afcd75990 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f6840625b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561afccbdae0 con 0x561afcd6fbc0 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75070 0x561afcd75450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75070 0x561afcd75450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43434/0 (socket says 172.21.15.63:43434) 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.224+0000 7f682ea00700 1 -- 172.21.15.63:0/1836399565 learned_addr learned my addr 172.21.15.63:0/1836399565 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682e000700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f68140b2a70 0x7f68140b4f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f681c006010 tx=0x7f681c003680 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682f400700 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd70020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682e000700 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75990 0x561afcd6f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682ea00700 1 -- 172.21.15.63:0/1836399565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 msgr2=0x561afcd70020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682ea00700 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd70020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682ea00700 1 -- 172.21.15.63:0/1836399565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75990 msgr2=0x561afcd6f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682ea00700 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75990 0x561afcd6f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682ea00700 1 -- 172.21.15.63:0/1836399565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afcc0e9d0 con 0x561afcd75070 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682e000700 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75990 0x561afcd6f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:44:44.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f682ea00700 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75070 0x561afcd75450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6820009230 tx=0x7f68200a9a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0d096dc599bdf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.225+0000 7f681be00700 1 -- 172.21.15.63:0/1836399565 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200aa220 con 0x561afcd75070 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.226+0000 7f681be00700 1 -- 172.21.15.63:0/1836399565 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68200aae50 con 0x561afcd75070 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.226+0000 7f681be00700 1 -- 172.21.15.63:0/1836399565 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200b2730 con 0x561afcd75070 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.226+0000 7f6840625b80 1 -- 172.21.15.63:0/1836399565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75070 msgr2=0x561afcd75450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.226+0000 7f6840625b80 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75070 0x561afcd75450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6820009230 tx=0x7f68200a9a00 comp rx=0 tx=0).stop 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 -- 172.21.15.63:0/1836399565 shutdown_connections 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd70020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75070 0x561afcd75450 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 --2- 172.21.15.63:0/1836399565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75990 0x561afcd6f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 -- 172.21.15.63:0/1836399565 >> 172.21.15.63:0/1836399565 conn(0x561afcc9dc60 msgr2=0x561afcd6a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 -- 172.21.15.63:0/1836399565 shutdown_connections 2024-05-11T13:44:44.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 -- 172.21.15.63:0/1836399565 wait complete. 2024-05-11T13:44:44.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.227+0000 7f6840625b80 1 -- start start 2024-05-11T13:44:44.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 1 Processor -- start 2024-05-11T13:44:44.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd7b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd78ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75990 0x561afcd79000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561afcd7c4b0 con 0x561afcd75990 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561afcc27c20 con 0x561afcd75070 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561afcd7c630 con 0x561afcd6fbc0 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.228+0000 7f6840625b80 10 client.0 _ll_register_callbacks cb 0x561afcc68d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75990 0x561afcd79000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75990 0x561afcd79000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43444/0 (socket says 172.21.15.63:43444) 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 -- 172.21.15.63:0/1029575001 learned_addr learned my addr 172.21.15.63:0/1029575001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682ea00700 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682e000700 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd78ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 -- 172.21.15.63:0/1029575001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 msgr2=0x561afcd7b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd7b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 -- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 msgr2=0x561afcd78ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd78ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 -- 172.21.15.63:0/1029575001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afcc87520 con 0x561afcd75990 2024-05-11T13:44:44.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f682f400700 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75990 0x561afcd79000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6828007970 tx=0x7f6828007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11255d3913bd30d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:44.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.229+0000 7f6818c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:44:44.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.230+0000 7f6818c00700 1 -- 172.21.15.63:0/1029575001 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682801a020 con 0x561afcd75990 2024-05-11T13:44:44.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.230+0000 7f6818c00700 1 -- 172.21.15.63:0/1029575001 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f68280118d0 con 0x561afcd75990 2024-05-11T13:44:44.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.230+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68200aae50 con 0x561afcd75990 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.230+0000 7f6818c00700 1 -- 172.21.15.63:0/1029575001 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f6828025050 con 0x561afcd75990 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.230+0000 7f6840625b80 10 client.15885 fetch_fsmap learned FSMap version 64 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.230+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561afcd86d10 con 0x561afcd75990 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.230+0000 7f6818c00700 1 -- 172.21.15.63:0/1029575001 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68200aae50 con 0x561afcd75990 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.231+0000 7f6818c00700 1 -- 172.21.15.63:0/1029575001 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f6828016b50 con 0x561afcd75990 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.231+0000 7f6840625b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 64 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.231+0000 7f6840625b80 10 client.15885 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:44:44.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.231+0000 7f6840625b80 20 client.15885 populate_metadata read hostname 'smithi063' 2024-05-11T13:44:44.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.231+0000 7f6840625b80 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcd893f0 0x561afcd8b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.231+0000 7f6840625b80 4 client.15885 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:44:44.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.231+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561afcd3c2b0 con 0x561afcd893f0 2024-05-11T13:44:44.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.232+0000 7f682ea00700 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcd893f0 0x561afcd8b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.232+0000 7f682ea00700 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcd893f0 0x561afcd8b7e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e210c5d16e16db6f server_cookie=101b9bf148c7fa7c in_seq=0 out_seq=0 2024-05-11T13:44:44.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.232+0000 7f6818c00700 10 client.15885 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:44.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.234+0000 7f6818c00700 1 -- 172.21.15.63:0/1029575001 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561afcd3c2b0 con 0x561afcd893f0 2024-05-11T13:44:44.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.234+0000 7f6818c00700 10 client.15885 handle_command_reply: tid=0 2024-05-11T13:44:44.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 client.15885 shutdown 2024-05-11T13:44:44.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75990 msgr2=0x561afcd79000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75990 0x561afcd79000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6828007970 tx=0x7f6828007d30 comp rx=0 tx=0).stop 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 shutdown_connections 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd75990 0x561afcd79000 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd6fbc0 0x561afcd7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcd893f0 0x561afcd8b7e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 --2- 172.21.15.63:0/1029575001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd78ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 >> 172.21.15.63:0/1029575001 conn(0x561afcc9dc60 msgr2=0x561afcd6e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6818c00700 0 client.15885 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:44.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 shutdown_connections 2024-05-11T13:44:44.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 1 -- 172.21.15.63:0/1029575001 wait complete. 2024-05-11T13:44:44.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.235+0000 7f6840625b80 20 client.15885 trim_cache size 0 max 16384 2024-05-11T13:44:44.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.258+0000 7f6840625b80 1 Processor -- start 2024-05-11T13:44:44.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.258+0000 7f6840625b80 1 -- start start 2024-05-11T13:44:44.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f6840625b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd6e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f6840625b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd85540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f6840625b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f6840625b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b1760 con 0x561afcd6fbc0 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f6840625b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b3390 con 0x561afcd75070 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f6840625b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b1e30 con 0x561afcd75990 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd6e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd6e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43458/0 (socket says 172.21.15.63:43458) 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682ea00700 1 -- 172.21.15.63:0/188421711 learned_addr learned my addr 172.21.15.63:0/188421711 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682e000700 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd85540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682f400700 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682ea00700 1 -- 172.21.15.63:0/188421711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 msgr2=0x561afcd773b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682ea00700 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd773b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682ea00700 1 -- 172.21.15.63:0/188421711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 msgr2=0x561afcd85540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.259+0000 7f682ea00700 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd85540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.260+0000 7f682ea00700 1 -- 172.21.15.63:0/188421711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afcd66a30 con 0x561afcd6fbc0 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.260+0000 7f682ea00700 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd6e9f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f682000e970 tx=0x7f6820091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b019ff0535060845 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.260+0000 7f681b400700 1 -- 172.21.15.63:0/188421711 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200aa220 con 0x561afcd6fbc0 2024-05-11T13:44:44.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.260+0000 7f681b400700 1 -- 172.21.15.63:0/188421711 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68200aae50 con 0x561afcd6fbc0 2024-05-11T13:44:44.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.260+0000 7f681b400700 1 -- 172.21.15.63:0/188421711 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820010430 con 0x561afcd6fbc0 2024-05-11T13:44:44.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 -- 172.21.15.63:0/188421711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 msgr2=0x561afcd6e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd6e9f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f682000e970 tx=0x7f6820091040 comp rx=0 tx=0).stop 2024-05-11T13:44:44.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 -- 172.21.15.63:0/188421711 shutdown_connections 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd6e9f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 --2- 172.21.15.63:0/188421711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd85540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 -- 172.21.15.63:0/188421711 >> 172.21.15.63:0/188421711 conn(0x561afcc9dc60 msgr2=0x561afcd78a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 -- 172.21.15.63:0/188421711 shutdown_connections 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.261+0000 7f6840625b80 1 -- 172.21.15.63:0/188421711 wait complete. 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.262+0000 7f6840625b80 1 -- start start 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.262+0000 7f6840625b80 1 Processor -- start 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.262+0000 7f6840625b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd81f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.262+0000 7f6840625b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f6840625b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f6840625b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b2c00 con 0x561afcd6fbc0 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f6840625b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b2500 con 0x561afcd75070 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f6840625b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68301af720 con 0x561afcd75990 2024-05-11T13:44:44.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f6840625b80 10 client.0 _ll_register_callbacks cb 0x561afcc68d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd81f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd81f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43464/0 (socket says 172.21.15.63:43464) 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 -- 172.21.15.63:0/1773446225 learned_addr learned my addr 172.21.15.63:0/1773446225 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682e000700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682f400700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 -- 172.21.15.63:0/1773446225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 msgr2=0x561afcd7efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd7efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 -- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 msgr2=0x561afcd7e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd7e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682ea00700 1 -- 172.21.15.63:0/1773446225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afcdf3c70 con 0x561afcd6fbc0 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.263+0000 7f682f400700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd7efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f682ea00700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd81f70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6820009880 tx=0x7f6820010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5442070724affa0c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:44:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f6818c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:44:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f6818c00700 1 -- 172.21.15.63:0/1773446225 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200a6110 con 0x561afcd6fbc0 2024-05-11T13:44:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f6818c00700 1 -- 172.21.15.63:0/1773446225 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f68200a9f90 con 0x561afcd6fbc0 2024-05-11T13:44:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68200aae50 con 0x561afcd6fbc0 2024-05-11T13:44:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f6818c00700 1 -- 172.21.15.63:0/1773446225 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f68200b24f0 con 0x561afcd6fbc0 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f6840625b80 10 client.15891 fetch_fsmap learned FSMap version 64 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.264+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561afcdf4e00 con 0x561afcd6fbc0 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6818c00700 1 -- 172.21.15.63:0/1773446225 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68200aae50 con 0x561afcd6fbc0 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6818c00700 1 -- 172.21.15.63:0/1773446225 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f68200a5490 con 0x561afcd6fbc0 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6840625b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 64 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6840625b80 10 client.15891 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6840625b80 20 client.15891 populate_metadata read hostname 'smithi063' 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6840625b80 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcdf5020 0x561afcdf7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6840625b80 4 client.15891 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7ca62e7", "release": true, "await": true}] 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.265+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7ca62e7", "release": true, "await": true}) -- 0x561afcccf500 con 0x561afcdf5020 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.266+0000 7f682e000700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcdf5020 0x561afcdf7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.267+0000 7f682e000700 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcdf5020 0x561afcdf7410 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd691b06094c703 server_cookie=fa579a49314a08ac in_seq=0 out_seq=0 2024-05-11T13:44:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.267+0000 7f6818c00700 10 client.15891 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.341+0000 7f6818c00700 1 -- 172.21.15.63:0/1773446225 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561afcccf500 con 0x561afcdf5020 2024-05-11T13:44:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.341+0000 7f6818c00700 10 client.15891 handle_command_reply: tid=0 2024-05-11T13:44:44.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.341+0000 7f6840625b80 1 client.15891 shutdown 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.341+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 msgr2=0x561afcd81f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.341+0000 7f6840625b80 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd81f70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6820009880 tx=0x7f6820010c40 comp rx=0 tx=0).stop 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.342+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 shutdown_connections 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.342+0000 7f6840625b80 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afcd75990 0x561afcd7efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.342+0000 7f6840625b80 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561afcd6fbc0 0x561afcd81f70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.342+0000 7f6840625b80 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x561afcdf5020 0x561afcdf7410 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.342+0000 7f6840625b80 1 --2- 172.21.15.63:0/1773446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561afcd75070 0x561afcd7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.342+0000 7f6818c00700 0 client.15891 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:44:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.342+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 >> 172.21.15.63:0/1773446225 conn(0x561afcc9dc60 msgr2=0x561afcd72af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.343+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 shutdown_connections 2024-05-11T13:44:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.343+0000 7f6840625b80 1 -- 172.21.15.63:0/1773446225 wait complete. 2024-05-11T13:44:44.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.343+0000 7f6840625b80 20 client.15891 trim_cache size 0 max 16384 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 64, 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 32, 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "7ca62e7": { 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 32, 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:44:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 91.4, 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:44:44.336 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.344+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f68140b2a70 msgr2=0x7f68140b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.344+0000 7f682fe00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f68140b2a70 0x7f68140b4f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f681c006010 tx=0x7f681c003680 comp rx=0 tx=0).stop 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.344+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 msgr2=0x7f6830075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.344+0000 7f682fe00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830075850 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9a0 tx=0x7f682000ed60 comp rx=0 tx=0).stop 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 shutdown_connections 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f68140b2a70 0x7f68140b4f30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6830109550 0x7f68300762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68301034a0 0x7f6830075850 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 --2- 172.21.15.63:0/1468603743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830103e60 0x7f6830075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 >> 172.21.15.63:0/1468603743 conn(0x7f68300fefd0 msgr2=0x7f6830100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:44:44.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 shutdown_connections 2024-05-11T13:44:44.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:44:44.345+0000 7f682fe00700 1 -- 172.21.15.63:0/1468603743 wait complete. 2024-05-11T13:44:44.401 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:44 smithi063 ceph-mon[29809]: pgmap v339: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2024-05-11T13:44:44.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:44 smithi071 ceph-mon[33293]: pgmap v339: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2024-05-11T13:44:44.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:44 smithi165 ceph-mon[33235]: pgmap v339: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2024-05-11T13:44:45.266 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7ca62e7', seconds elapsed: 52.1 2024-05-11T13:44:46.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 200.0 s, 156.7 tps, lat 5273.892 ms stddev 13192.587 2024-05-11T13:44:46.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:46 smithi063 ceph-mon[29809]: pgmap v340: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-11T13:44:46.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:46 smithi071 ceph-mon[33293]: pgmap v340: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-11T13:44:46.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:46 smithi165 ceph-mon[33235]: pgmap v340: 129 pgs: 129 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-11T13:44:48.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:48 smithi063 ceph-mon[29809]: pgmap v341: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-05-11T13:44:48.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:48 smithi071 ceph-mon[33293]: pgmap v341: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-05-11T13:44:48.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:48 smithi165 ceph-mon[33235]: pgmap v341: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-05-11T13:44:50.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:50 smithi063 ceph-mon[29809]: pgmap v342: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-05-11T13:44:50.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:50 smithi071 ceph-mon[33293]: pgmap v342: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-05-11T13:44:50.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:50 smithi165 ceph-mon[33235]: pgmap v342: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-05-11T13:44:51.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 205.0 s, 465.4 tps, lat 216.404 ms stddev 37.169 2024-05-11T13:44:52.678 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:52 smithi063 ceph-mon[29809]: pgmap v343: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-05-11T13:44:52.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:52 smithi071 ceph-mon[33293]: pgmap v343: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-05-11T13:44:52.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:52 smithi165 ceph-mon[33235]: pgmap v343: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-05-11T13:44:54.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:54 smithi063 ceph-mon[29809]: pgmap v344: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.2 MiB/s wr, 403 op/s 2024-05-11T13:44:54.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:54 smithi071 ceph-mon[33293]: pgmap v344: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.2 MiB/s wr, 403 op/s 2024-05-11T13:44:54.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:54 smithi165 ceph-mon[33235]: pgmap v344: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.2 MiB/s wr, 403 op/s 2024-05-11T13:44:56.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 210.0 s, 450.4 tps, lat 220.591 ms stddev 42.104 2024-05-11T13:44:56.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:56 smithi063 ceph-mon[29809]: pgmap v345: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.2 MiB/s wr, 399 op/s 2024-05-11T13:44:56.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:56 smithi071 ceph-mon[33293]: pgmap v345: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.2 MiB/s wr, 399 op/s 2024-05-11T13:44:56.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:56 smithi165 ceph-mon[33235]: pgmap v345: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.2 MiB/s wr, 399 op/s 2024-05-11T13:44:58.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:44:58 smithi063 ceph-mon[29809]: pgmap v346: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-05-11T13:44:58.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:44:58 smithi071 ceph-mon[33293]: pgmap v346: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-05-11T13:44:58.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:44:58 smithi165 ceph-mon[33235]: pgmap v346: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-05-11T13:45:00.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:00 smithi063 ceph-mon[29809]: pgmap v347: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-05-11T13:45:00.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:00 smithi071 ceph-mon[33293]: pgmap v347: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-05-11T13:45:00.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:00 smithi165 ceph-mon[33235]: pgmap v347: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-05-11T13:45:01.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 215.0 s, 469.8 tps, lat 214.942 ms stddev 57.915 2024-05-11T13:45:02.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:02 smithi063 ceph-mon[29809]: pgmap v348: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-05-11T13:45:02.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:02 smithi071 ceph-mon[33293]: pgmap v348: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-05-11T13:45:02.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:02 smithi165 ceph-mon[33235]: pgmap v348: 129 pgs: 129 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-05-11T13:45:05.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:05 smithi063 ceph-mon[29809]: osdmap e102: 12 total, 12 up, 12 in 2024-05-11T13:45:05.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:05 smithi063 ceph-mon[29809]: pgmap v350: 129 pgs: 129 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-11T13:45:05.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:05 smithi071 ceph-mon[33293]: osdmap e102: 12 total, 12 up, 12 in 2024-05-11T13:45:05.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:05 smithi071 ceph-mon[33293]: pgmap v350: 129 pgs: 129 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-11T13:45:05.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:05 smithi165 ceph-mon[33235]: osdmap e102: 12 total, 12 up, 12 in 2024-05-11T13:45:05.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:05 smithi165 ceph-mon[33235]: pgmap v350: 129 pgs: 129 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-11T13:45:06.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 220.0 s, 463.8 tps, lat 213.373 ms stddev 62.565 2024-05-11T13:45:06.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:06 smithi063 ceph-mon[29809]: pgmap v351: 129 pgs: 129 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-11T13:45:06.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:06 smithi071 ceph-mon[33293]: pgmap v351: 129 pgs: 129 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-11T13:45:06.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:06 smithi165 ceph-mon[33235]: pgmap v351: 129 pgs: 129 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-11T13:45:08.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:08 smithi063 ceph-mon[29809]: pgmap v352: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 647 op/s 2024-05-11T13:45:08.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:08 smithi071 ceph-mon[33293]: pgmap v352: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 647 op/s 2024-05-11T13:45:08.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:08 smithi165 ceph-mon[33235]: pgmap v352: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 647 op/s 2024-05-11T13:45:11.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 225.0 s, 493.0 tps, lat 204.075 ms stddev 32.887 2024-05-11T13:45:11.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:11 smithi063 ceph-mon[29809]: osdmap e103: 12 total, 12 up, 12 in 2024-05-11T13:45:11.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:11 smithi063 ceph-mon[29809]: pgmap v354: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 8.0 MiB/s wr, 809 op/s 2024-05-11T13:45:11.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:11 smithi071 ceph-mon[33293]: osdmap e103: 12 total, 12 up, 12 in 2024-05-11T13:45:11.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:11 smithi071 ceph-mon[33293]: pgmap v354: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 8.0 MiB/s wr, 809 op/s 2024-05-11T13:45:11.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:11 smithi165 ceph-mon[33235]: osdmap e103: 12 total, 12 up, 12 in 2024-05-11T13:45:11.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:11 smithi165 ceph-mon[33235]: pgmap v354: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 8.0 MiB/s wr, 809 op/s 2024-05-11T13:45:12.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:12 smithi063 ceph-mon[29809]: pgmap v355: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.7 MiB/s wr, 375 op/s 2024-05-11T13:45:12.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:12 smithi071 ceph-mon[33293]: pgmap v355: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.7 MiB/s wr, 375 op/s 2024-05-11T13:45:12.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:12 smithi165 ceph-mon[33235]: pgmap v355: 129 pgs: 129 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.7 MiB/s wr, 375 op/s 2024-05-11T13:45:14.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:14 smithi063 ceph-mon[29809]: pgmap v356: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-05-11T13:45:14.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:14 smithi071 ceph-mon[33293]: pgmap v356: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-05-11T13:45:14.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:14 smithi165 ceph-mon[33235]: pgmap v356: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-05-11T13:45:16.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 230.0 s, 471.8 tps, lat 211.474 ms stddev 49.171 2024-05-11T13:45:16.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:16 smithi063 ceph-mon[29809]: pgmap v357: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-05-11T13:45:16.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:16 smithi071 ceph-mon[33293]: pgmap v357: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-05-11T13:45:16.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:16 smithi165 ceph-mon[33235]: pgmap v357: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-05-11T13:45:18.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:18 smithi063 ceph-mon[29809]: pgmap v358: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-05-11T13:45:18.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:18 smithi071 ceph-mon[33293]: pgmap v358: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-05-11T13:45:18.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:18 smithi165 ceph-mon[33235]: pgmap v358: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-05-11T13:45:20.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:20 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:20.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:20 smithi063 ceph-mon[29809]: pgmap v359: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 644 op/s 2024-05-11T13:45:20.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:20 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:20.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:20 smithi071 ceph-mon[33293]: pgmap v359: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 644 op/s 2024-05-11T13:45:20.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:20 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:20.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:20 smithi165 ceph-mon[33235]: pgmap v359: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 644 op/s 2024-05-11T13:45:21.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 235.0 s, 462.0 tps, lat 215.821 ms stddev 92.561 2024-05-11T13:45:22.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:22 smithi063 ceph-mon[29809]: pgmap v360: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.5 MiB/s wr, 546 op/s 2024-05-11T13:45:22.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:22 smithi071 ceph-mon[33293]: pgmap v360: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.5 MiB/s wr, 546 op/s 2024-05-11T13:45:22.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:22 smithi165 ceph-mon[33235]: pgmap v360: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.5 MiB/s wr, 546 op/s 2024-05-11T13:45:24.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:24 smithi063 ceph-mon[29809]: pgmap v361: 129 pgs: 129 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.1 MiB/s wr, 817 op/s 2024-05-11T13:45:24.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:24 smithi071 ceph-mon[33293]: pgmap v361: 129 pgs: 129 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.1 MiB/s wr, 817 op/s 2024-05-11T13:45:24.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:24 smithi165 ceph-mon[33235]: pgmap v361: 129 pgs: 129 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.1 MiB/s wr, 817 op/s 2024-05-11T13:45:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 240.0 s, 413.6 tps, lat 241.997 ms stddev 92.155 2024-05-11T13:45:26.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:26 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:45:26.445 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:26 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:45:26.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:26 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:45:27.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:27 smithi063 ceph-mon[29809]: pgmap v362: 129 pgs: 129 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2024-05-11T13:45:27.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:27 smithi071 ceph-mon[33293]: pgmap v362: 129 pgs: 129 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2024-05-11T13:45:27.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:27 smithi165 ceph-mon[33235]: pgmap v362: 129 pgs: 129 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2024-05-11T13:45:28.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:28 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:45:28.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:28 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:45:28.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:28 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:45:28.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:28 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:45:28.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:28 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:45:28.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:28 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:45:28.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:28 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:45:28.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:28 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:45:28.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:28 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:45:29.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:29 smithi063 ceph-mon[29809]: pgmap v363: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.6 MiB/s wr, 767 op/s 2024-05-11T13:45:29.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:29 smithi071 ceph-mon[33293]: pgmap v363: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.6 MiB/s wr, 767 op/s 2024-05-11T13:45:29.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:29 smithi165 ceph-mon[33235]: pgmap v363: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.6 MiB/s wr, 767 op/s 2024-05-11T13:45:30.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:30 smithi063 ceph-mon[29809]: pgmap v364: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-11T13:45:30.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:30 smithi071 ceph-mon[33293]: pgmap v364: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-11T13:45:30.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:30 smithi165 ceph-mon[33235]: pgmap v364: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-11T13:45:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 245.0 s, 443.6 tps, lat 226.094 ms stddev 42.538 2024-05-11T13:45:32.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:32 smithi063 ceph-mon[29809]: pgmap v365: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-11T13:45:32.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:32 smithi071 ceph-mon[33293]: pgmap v365: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-11T13:45:32.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:32 smithi165 ceph-mon[33235]: pgmap v365: 129 pgs: 129 active+clean; 2.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-11T13:45:34.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:34 smithi063 ceph-mon[29809]: pgmap v366: 129 pgs: 129 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-05-11T13:45:34.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:34 smithi071 ceph-mon[33293]: pgmap v366: 129 pgs: 129 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-05-11T13:45:34.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:34 smithi165 ceph-mon[33235]: pgmap v366: 129 pgs: 129 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-05-11T13:45:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 250.0 s, 429.6 tps, lat 217.763 ms stddev 82.365 2024-05-11T13:45:36.609 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:36 smithi165 ceph-mon[33235]: pgmap v367: 129 pgs: 129 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-11T13:45:36.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:36 smithi063 ceph-mon[29809]: pgmap v367: 129 pgs: 129 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-11T13:45:36.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:36 smithi071 ceph-mon[33293]: pgmap v367: 129 pgs: 129 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-11T13:45:38.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:38 smithi063 ceph-mon[29809]: pgmap v368: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 7.5 MiB/s wr, 753 op/s 2024-05-11T13:45:38.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:38 smithi071 ceph-mon[33293]: pgmap v368: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 7.5 MiB/s wr, 753 op/s 2024-05-11T13:45:38.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:38 smithi165 ceph-mon[33235]: pgmap v368: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 7.5 MiB/s wr, 753 op/s 2024-05-11T13:45:40.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:40 smithi063 ceph-mon[29809]: pgmap v369: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-11T13:45:40.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:40 smithi071 ceph-mon[33293]: pgmap v369: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-11T13:45:40.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:40 smithi165 ceph-mon[33235]: pgmap v369: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-11T13:45:41.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 255.0 s, 406.6 tps, lat 257.579 ms stddev 83.731 2024-05-11T13:45:42.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:42 smithi063 ceph-mon[29809]: pgmap v370: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-11T13:45:42.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:42 smithi071 ceph-mon[33293]: pgmap v370: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-11T13:45:42.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:42 smithi165 ceph-mon[33235]: pgmap v370: 129 pgs: 129 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-11T13:45:43.582 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-11T13:45:43.582 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.6 2024-05-11T13:45:43.583 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:45:44.234 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:45:45.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:45 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:45.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:45 smithi063 ceph-mon[29809]: pgmap v371: 129 pgs: 129 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 9.3 MiB/s wr, 813 op/s 2024-05-11T13:45:45.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:45 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:45.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:45 smithi071 ceph-mon[33293]: pgmap v371: 129 pgs: 129 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 9.3 MiB/s wr, 813 op/s 2024-05-11T13:45:45.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:45 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:45.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:45 smithi165 ceph-mon[33235]: pgmap v371: 129 pgs: 129 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 9.3 MiB/s wr, 813 op/s 2024-05-11T13:45:46.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 260.0 s, 382.6 tps, lat 257.712 ms stddev 98.317 2024-05-11T13:45:46.655 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:46 smithi063 ceph-mon[29809]: pgmap v372: 129 pgs: 129 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-05-11T13:45:46.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:46 smithi071 ceph-mon[33293]: pgmap v372: 129 pgs: 129 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-05-11T13:45:46.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:46 smithi165 ceph-mon[33235]: pgmap v372: 129 pgs: 129 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-05-11T13:45:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 -- 172.21.15.63:0/3733439654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d0108480 msgr2=0x7f50d006b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 --2- 172.21.15.63:0/3733439654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d0108480 0x7f50d006b260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009990 tx=0x7f50bc009ca0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 -- 172.21.15.63:0/3733439654 shutdown_connections 2024-05-11T13:45:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 --2- 172.21.15.63:0/3733439654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0105f90 0x7f50d0106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 --2- 172.21.15.63:0/3733439654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d0108480 0x7f50d006b260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 --2- 172.21.15.63:0/3733439654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50d006b7a0 0x7f50d006bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 -- 172.21.15.63:0/3733439654 >> 172.21.15.63:0/3733439654 conn(0x7f50d0079400 msgr2=0x7f50d0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 -- 172.21.15.63:0/3733439654 shutdown_connections 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.324+0000 7f50d7400700 1 -- 172.21.15.63:0/3733439654 wait complete. 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.325+0000 7f50d7400700 1 Processor -- start 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.325+0000 7f50d7400700 1 -- start start 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.325+0000 7f50d7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d006b7a0 0x7f50d01a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.326+0000 7f50d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50d0105f90 0x7f50d01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.326+0000 7f50d7400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0108480 0x7f50d01a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.326+0000 7f50d7400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0119be0 con 0x7f50d0105f90 2024-05-11T13:45:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.326+0000 7f50d7400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0119a60 con 0x7f50d006b7a0 2024-05-11T13:45:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.326+0000 7f50d7400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0119d60 con 0x7f50d0108480 2024-05-11T13:45:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.327+0000 7f50d5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0108480 0x7f50d01a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.327+0000 7f50d4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d006b7a0 0x7f50d01a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.327+0000 7f50d5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0108480 0x7f50d01a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:35652/0 (socket says 172.21.15.63:35652) 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.327+0000 7f50d5400700 1 -- 172.21.15.63:0/3412608159 learned_addr learned my addr 172.21.15.63:0/3412608159 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.327+0000 7f50cfe00700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50d0105f90 0x7f50d01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.328+0000 7f50d5400700 1 -- 172.21.15.63:0/3412608159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d006b7a0 msgr2=0x7f50d01a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.328+0000 7f50d5400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d006b7a0 0x7f50d01a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.328+0000 7f50d5400700 1 -- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50d0105f90 msgr2=0x7f50d01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.328+0000 7f50d5400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50d0105f90 0x7f50d01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.328+0000 7f50d5400700 1 -- 172.21.15.63:0/3412608159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d01a5180 con 0x7f50d0108480 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.328+0000 7f50d5400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0108480 0x7f50d01a4a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f50c800ada0 tx=0x7f50c800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.329+0000 7f50cd600700 1 -- 172.21.15.63:0/3412608159 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8017020 con 0x7f50d0108480 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.329+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d01a5470 con 0x7f50d0108480 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.329+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d006cf70 con 0x7f50d0108480 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.329+0000 7f50cd600700 1 -- 172.21.15.63:0/3412608159 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50c800cd60 con 0x7f50d0108480 2024-05-11T13:45:48.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.329+0000 7f50cd600700 1 -- 172.21.15.63:0/3412608159 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8005280 con 0x7f50d0108480 2024-05-11T13:45:48.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.331+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b0005330 con 0x7f50d0108480 2024-05-11T13:45:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.337+0000 7f50cd600700 1 -- 172.21.15.63:0/3412608159 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f50c800a090 con 0x7f50d0108480 2024-05-11T13:45:48.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.339+0000 7f50cd600700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50b80b2a70 0x7f50b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:48.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.339+0000 7f50cd600700 1 -- 172.21.15.63:0/3412608159 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f50c80e2530 con 0x7f50d0108480 2024-05-11T13:45:48.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.340+0000 7f50d4a00700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50b80b2a70 0x7f50b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:48.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.344+0000 7f50d4a00700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50b80b2a70 0x7f50b80b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00b2e0 tx=0x7f50bc005fb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:48.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.347+0000 7f50cd600700 1 -- 172.21.15.63:0/3412608159 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50c8021020 con 0x7f50d0108480 2024-05-11T13:45:48.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:48 smithi063 ceph-mon[29809]: pgmap v373: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 10 MiB/s wr, 782 op/s 2024-05-11T13:45:48.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:48 smithi071 ceph-mon[33293]: pgmap v373: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 10 MiB/s wr, 782 op/s 2024-05-11T13:45:48.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:48 smithi165 ceph-mon[33235]: pgmap v373: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 10 MiB/s wr, 782 op/s 2024-05-11T13:45:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.821+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50b0005620 con 0x7f50d0108480 2024-05-11T13:45:48.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.824+0000 7f50cd600700 1 -- 172.21.15.63:0/3412608159 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11936 (secure 0 0 0) 0x7f50c80a9bb0 con 0x7f50d0108480 2024-05-11T13:45:48.815 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:45:48.815 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:45:44.076626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:45:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50b80b2a70 msgr2=0x7f50b80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50b80b2a70 0x7f50b80b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00b2e0 tx=0x7f50bc005fb0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0108480 msgr2=0x7f50d01a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0108480 0x7f50d01a4a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f50c800ada0 tx=0x7f50c800c330 comp rx=0 tx=0).stop 2024-05-11T13:45:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 shutdown_connections 2024-05-11T13:45:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f50b80b2a70 0x7f50b80b4f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50d0108480 0x7f50d01a4a40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d006b7a0 0x7f50d01a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 --2- 172.21.15.63:0/3412608159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50d0105f90 0x7f50d01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 >> 172.21.15.63:0/3412608159 conn(0x7f50d0079400 msgr2=0x7f50d0101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 shutdown_connections 2024-05-11T13:45:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:48.828+0000 7f50d7400700 1 -- 172.21.15.63:0/3412608159 wait complete. 2024-05-11T13:45:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 66 2024-05-11T13:45:49.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:49 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3412608159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:45:49.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:49 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3412608159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:45:49.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:49 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3412608159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:45:49.828 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 87.6 --await-for 42.0' 2024-05-11T13:45:49.828 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 87.6 --await-for 42.0 2024-05-11T13:45:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:45:50.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:50 smithi063 ceph-mon[29809]: pgmap v374: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.5 MiB/s wr, 532 op/s 2024-05-11T13:45:50.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:50 smithi071 ceph-mon[33293]: pgmap v374: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.5 MiB/s wr, 532 op/s 2024-05-11T13:45:50.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:50 smithi165 ceph-mon[33235]: pgmap v374: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.5 MiB/s wr, 532 op/s 2024-05-11T13:45:51.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 265.0 s, 388.6 tps, lat 265.505 ms stddev 92.215 2024-05-11T13:45:52.616 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:52 smithi063 ceph-mon[29809]: pgmap v375: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.5 MiB/s wr, 532 op/s 2024-05-11T13:45:52.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:52 smithi071 ceph-mon[33293]: pgmap v375: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.5 MiB/s wr, 532 op/s 2024-05-11T13:45:52.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:52 smithi165 ceph-mon[33235]: pgmap v375: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.5 MiB/s wr, 532 op/s 2024-05-11T13:45:52.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 -- 172.21.15.63:0/2253226639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 msgr2=0x7efd441123a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:52.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2253226639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 0x7efd441123a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009f90 tx=0x7efd3c00b330 comp rx=0 tx=0).stop 2024-05-11T13:45:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 -- 172.21.15.63:0/2253226639 shutdown_connections 2024-05-11T13:45:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2253226639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 0x7efd441123a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2253226639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd44075780 0x7efd44075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2253226639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 0x7efd4410fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 -- 172.21.15.63:0/2253226639 >> 172.21.15.63:0/2253226639 conn(0x7efd4407a6b0 msgr2=0x7efd4407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.877+0000 7efd4ac00700 1 -- 172.21.15.63:0/2253226639 shutdown_connections 2024-05-11T13:45:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.878+0000 7efd4ac00700 1 -- 172.21.15.63:0/2253226639 wait complete. 2024-05-11T13:45:52.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 Processor -- start 2024-05-11T13:45:52.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 -- start start 2024-05-11T13:45:52.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd44075780 0x7efd4410f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 0x7efd4410c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 0x7efd4410cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd441157d0 con 0x7efd44075780 2024-05-11T13:45:52.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd44115650 con 0x7efd440760a0 2024-05-11T13:45:52.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.880+0000 7efd4ac00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd44115950 con 0x7efd4410ffb0 2024-05-11T13:45:52.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.881+0000 7efd43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd44075780 0x7efd4410f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.881+0000 7efd43400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 0x7efd4410c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.881+0000 7efd43400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 0x7efd4410c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:46402/0 (socket says 172.21.15.63:46402) 2024-05-11T13:45:52.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.881+0000 7efd43400700 1 -- 172.21.15.63:0/2734199501 learned_addr learned my addr 172.21.15.63:0/2734199501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:45:52.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.881+0000 7efd48c00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 0x7efd4410cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.882+0000 7efd43400700 1 -- 172.21.15.63:0/2734199501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 msgr2=0x7efd4410cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:52.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.882+0000 7efd43400700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 0x7efd4410cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.882+0000 7efd43400700 1 -- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd44075780 msgr2=0x7efd4410f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:52.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.882+0000 7efd43400700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd44075780 0x7efd4410f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.882+0000 7efd43400700 1 -- 172.21.15.63:0/2734199501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4410d240 con 0x7efd440760a0 2024-05-11T13:45:52.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.882+0000 7efd43400700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 0x7efd4410c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e9f0 tx=0x7efd3000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:52.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.882+0000 7efd40c00700 1 -- 172.21.15.63:0/2734199501 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000cbd0 con 0x7efd440760a0 2024-05-11T13:45:52.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.883+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd441ac510 con 0x7efd440760a0 2024-05-11T13:45:52.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.883+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd441ac900 con 0x7efd440760a0 2024-05-11T13:45:52.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.883+0000 7efd40c00700 1 -- 172.21.15.63:0/2734199501 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd3000cd70 con 0x7efd440760a0 2024-05-11T13:45:52.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.883+0000 7efd40c00700 1 -- 172.21.15.63:0/2734199501 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30005540 con 0x7efd440760a0 2024-05-11T13:45:52.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.888+0000 7efd40c00700 1 -- 172.21.15.63:0/2734199501 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7efd30004750 con 0x7efd440760a0 2024-05-11T13:45:52.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.890+0000 7efd40c00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd280b2b20 0x7efd280b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.890+0000 7efd40c00700 1 -- 172.21.15.63:0/2734199501 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7efd300e29f0 con 0x7efd440760a0 2024-05-11T13:45:52.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.890+0000 7efd55513b80 1 Processor -- start 2024-05-11T13:45:52.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.890+0000 7efd43e00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd280b2b20 0x7efd280b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.890+0000 7efd55513b80 1 -- start start 2024-05-11T13:45:52.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd55513b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd55513b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cfb50 0x55e2be3c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd55513b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd55513b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e2be31bbb0 con 0x55e2be3c9d50 2024-05-11T13:45:52.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd55513b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e2be2b98f0 con 0x55e2be3cfb50 2024-05-11T13:45:52.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd55513b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e2be317b70 con 0x55e2be3cf230 2024-05-11T13:45:52.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd43e00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd280b2b20 0x7efd280b4fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd34009c60 tx=0x7efd34009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.891+0000 7efd48c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:36086/0 (socket says 172.21.15.63:36086) 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43e00700 1 -- 172.21.15.63:0/494905842 learned_addr learned my addr 172.21.15.63:0/494905842 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43400700 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cfb50 0x55e2be3c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43400700 1 -- 172.21.15.63:0/494905842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 msgr2=0x55e2be3cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43400700 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43400700 1 -- 172.21.15.63:0/494905842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 msgr2=0x55e2be3ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43400700 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.892+0000 7efd43400700 1 -- 172.21.15.63:0/494905842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2be268a60 con 0x55e2be3cfb50 2024-05-11T13:45:52.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.893+0000 7efd43400700 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cfb50 0x55e2be3c9730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd300dfd50 tx=0x7efd300a9050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8885e5e87b5cc64a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:52.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.893+0000 7efd39600700 1 -- 172.21.15.63:0/494905842 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300a9fe0 con 0x55e2be3cfb50 2024-05-11T13:45:52.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.893+0000 7efd39600700 1 -- 172.21.15.63:0/494905842 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd300a98e0 con 0x55e2be3cfb50 2024-05-11T13:45:52.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.893+0000 7efd39600700 1 -- 172.21.15.63:0/494905842 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300a5b70 con 0x55e2be3cfb50 2024-05-11T13:45:52.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 -- 172.21.15.63:0/494905842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cfb50 msgr2=0x55e2be3c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:52.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cfb50 0x55e2be3c9730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd300dfd50 tx=0x7efd300a9050 comp rx=0 tx=0).stop 2024-05-11T13:45:52.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 -- 172.21.15.63:0/494905842 shutdown_connections 2024-05-11T13:45:52.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 --2- 172.21.15.63:0/494905842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cfb50 0x55e2be3c9730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 -- 172.21.15.63:0/494905842 >> 172.21.15.63:0/494905842 conn(0x55e2be2f7cf0 msgr2=0x55e2be3c4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:52.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 -- 172.21.15.63:0/494905842 shutdown_connections 2024-05-11T13:45:52.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.895+0000 7efd55513b80 1 -- 172.21.15.63:0/494905842 wait complete. 2024-05-11T13:45:52.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.896+0000 7efd55513b80 1 -- start start 2024-05-11T13:45:52.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.896+0000 7efd55513b80 1 Processor -- start 2024-05-11T13:45:52.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.896+0000 7efd55513b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3c9d50 0x55e2be3d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd55513b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd55513b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd55513b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e2be3d6670 con 0x55e2be3cfb50 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd55513b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e2be281cb0 con 0x55e2be3c9d50 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd55513b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e2be3d67f0 con 0x55e2be3cf230 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd55513b80 10 client.0 _ll_register_callbacks cb 0x55e2be2af510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd43e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3c9d50 0x55e2be3d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd43e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3c9d50 0x55e2be3d58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:46424/0 (socket says 172.21.15.63:46424) 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd43e00700 1 -- 172.21.15.63:0/4056501413 learned_addr learned my addr 172.21.15.63:0/4056501413 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:45:52.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd48c00700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.897+0000 7efd43400700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd43e00700 1 -- 172.21.15.63:0/4056501413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 msgr2=0x55e2be3d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd43e00700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3d2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd43e00700 1 -- 172.21.15.63:0/4056501413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 msgr2=0x55e2be3d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd43e00700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd43e00700 1 -- 172.21.15.63:0/4056501413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2be2e15b0 con 0x55e2be3c9d50 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd48c00700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd43e00700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3c9d50 0x55e2be3d58a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd34009b20 tx=0x7efd340047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1b5aed937127de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:52.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd2e000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:45:52.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.898+0000 7efd2e000700 1 -- 172.21.15.63:0/4056501413 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3401d820 con 0x55e2be3c9d50 2024-05-11T13:45:52.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.899+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd300a98e0 con 0x55e2be3c9d50 2024-05-11T13:45:52.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.899+0000 7efd2e000700 1 -- 172.21.15.63:0/4056501413 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7efd3401d9c0 con 0x55e2be3c9d50 2024-05-11T13:45:52.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.899+0000 7efd2e000700 1 -- 172.21.15.63:0/4056501413 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7efd3401dbc0 con 0x55e2be3c9d50 2024-05-11T13:45:52.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.899+0000 7efd55513b80 10 client.35492 fetch_fsmap learned FSMap version 66 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.899+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55e2be3e0fb0 con 0x55e2be3c9d50 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd2e000700 1 -- 172.21.15.63:0/4056501413 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd300a98e0 con 0x55e2be3c9d50 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd2e000700 1 -- 172.21.15.63:0/4056501413 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 66) ==== 8366+0+0 (secure 0 0 0) 0x7efd3401c410 con 0x55e2be3c9d50 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd55513b80 10 client.35492 fetch_fsmap finished waiting for FSMap version 66 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd55513b80 10 client.35492 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd55513b80 20 client.35492 populate_metadata read hostname 'smithi063' 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd55513b80 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be3e3590 0x55e2be3e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd55513b80 4 client.35492 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.900+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2be3961f0 con 0x55e2be3e3590 2024-05-11T13:45:52.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.901+0000 7efd43400700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be3e3590 0x55e2be3e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.902+0000 7efd2e000700 10 client.35492 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:45:52.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.902+0000 7efd43400700 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be3e3590 0x55e2be3e59c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d6c9429b15f95b server_cookie=7b90cffcd24c5d20 in_seq=0 out_seq=0 2024-05-11T13:45:52.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd2e000700 1 -- 172.21.15.63:0/4056501413 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2be3961f0 con 0x55e2be3e3590 2024-05-11T13:45:52.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd2e000700 10 client.35492 handle_command_reply: tid=0 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd55513b80 1 client.35492 shutdown 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3c9d50 msgr2=0x55e2be3d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd55513b80 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3c9d50 0x55e2be3d58a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd34009b20 tx=0x7efd340047c0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 shutdown_connections 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd55513b80 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.903+0000 7efd55513b80 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be3e3590 0x55e2be3e59c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.904+0000 7efd55513b80 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3c9d50 0x55e2be3d58a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.904+0000 7efd55513b80 1 --2- 172.21.15.63:0/4056501413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cf230 0x55e2be3d2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.904+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 >> 172.21.15.63:0/4056501413 conn(0x55e2be2f7cf0 msgr2=0x55e2be3c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.904+0000 7efd2e000700 0 client.35492 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:45:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.904+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 shutdown_connections 2024-05-11T13:45:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.904+0000 7efd55513b80 1 -- 172.21.15.63:0/4056501413 wait complete. 2024-05-11T13:45:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.904+0000 7efd55513b80 20 client.35492 trim_cache size 0 max 16384 2024-05-11T13:45:52.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.935+0000 7efd55513b80 1 Processor -- start 2024-05-11T13:45:52.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.936+0000 7efd55513b80 1 -- start start 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.936+0000 7efd55513b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3c9d50 0x55e2be3c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.936+0000 7efd55513b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.936+0000 7efd55513b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.936+0000 7efd55513b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd2400ee40 con 0x55e2be3cfb50 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.936+0000 7efd55513b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd240109b0 con 0x55e2be3cf230 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.936+0000 7efd55513b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd2400f510 con 0x55e2be3c9d50 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.937+0000 7efd43400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.937+0000 7efd48c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.937+0000 7efd43400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3df730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:46426/0 (socket says 172.21.15.63:46426) 2024-05-11T13:45:52.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.937+0000 7efd43400700 1 -- 172.21.15.63:0/3947708687 learned_addr learned my addr 172.21.15.63:0/3947708687 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:45:52.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.937+0000 7efd43e00700 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3c9d50 0x55e2be3c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.937+0000 7efd43400700 1 -- 172.21.15.63:0/3947708687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3c9d50 msgr2=0x55e2be3c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.937+0000 7efd43400700 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3c9d50 0x55e2be3c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.938+0000 7efd43400700 1 -- 172.21.15.63:0/3947708687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 msgr2=0x55e2be3cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.938+0000 7efd43400700 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.938+0000 7efd43400700 1 -- 172.21.15.63:0/3947708687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2be3c6200 con 0x55e2be3cf230 2024-05-11T13:45:52.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.938+0000 7efd43400700 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3df730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd300a91c0 tx=0x7efd3000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a7ffe21c84685e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:52.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.938+0000 7efd38c00700 1 -- 172.21.15.63:0/3947708687 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300a9fe0 con 0x55e2be3cf230 2024-05-11T13:45:52.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.939+0000 7efd38c00700 1 -- 172.21.15.63:0/3947708687 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd300b2740 con 0x55e2be3cf230 2024-05-11T13:45:52.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.939+0000 7efd38c00700 1 -- 172.21.15.63:0/3947708687 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300a5b70 con 0x55e2be3cf230 2024-05-11T13:45:52.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 -- 172.21.15.63:0/3947708687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 msgr2=0x55e2be3df730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:52.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3df730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd300a91c0 tx=0x7efd3000ef90 comp rx=0 tx=0).stop 2024-05-11T13:45:52.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 -- 172.21.15.63:0/3947708687 shutdown_connections 2024-05-11T13:45:52.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3cfb50 0x55e2be3cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3c9d50 0x55e2be3c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 --2- 172.21.15.63:0/3947708687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3df730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 -- 172.21.15.63:0/3947708687 >> 172.21.15.63:0/3947708687 conn(0x55e2be2f7cf0 msgr2=0x55e2be3cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:52.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 -- 172.21.15.63:0/3947708687 shutdown_connections 2024-05-11T13:45:52.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.940+0000 7efd55513b80 1 -- 172.21.15.63:0/3947708687 wait complete. 2024-05-11T13:45:52.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.941+0000 7efd55513b80 1 -- start start 2024-05-11T13:45:52.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.941+0000 7efd55513b80 1 Processor -- start 2024-05-11T13:45:52.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd55513b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd55513b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3d8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd55513b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cfb50 0x55e2be44de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd55513b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd240102e0 con 0x55e2be3c9d50 2024-05-11T13:45:52.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd55513b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd2400fbe0 con 0x55e2be3cf230 2024-05-11T13:45:52.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd55513b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd2400ce00 con 0x55e2be3cfb50 2024-05-11T13:45:52.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd55513b80 10 client.0 _ll_register_callbacks cb 0x55e2be2af510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd43400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3d8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd43400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3d8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:46442/0 (socket says 172.21.15.63:46442) 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.942+0000 7efd43400700 1 -- 172.21.15.63:0/752360796 learned_addr learned my addr 172.21.15.63:0/752360796 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd48c00700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cfb50 0x55e2be44de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd43400700 1 -- 172.21.15.63:0/752360796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cfb50 msgr2=0x55e2be44de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd43e00700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd43400700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cfb50 0x55e2be44de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd43400700 1 -- 172.21.15.63:0/752360796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 msgr2=0x55e2be3d8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd43400700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3d8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:52.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd43400700 1 -- 172.21.15.63:0/752360796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2be44e390 con 0x55e2be3cf230 2024-05-11T13:45:52.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.943+0000 7efd43e00700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3d8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:45:52.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.944+0000 7efd43400700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3d8fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd300cc330 tx=0x7efd300a5860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b9ce77245d417f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:45:52.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.944+0000 7efd2e000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:45:52.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.944+0000 7efd2e000700 1 -- 172.21.15.63:0/752360796 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300a9fe0 con 0x55e2be3cf230 2024-05-11T13:45:52.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.944+0000 7efd2e000700 1 -- 172.21.15.63:0/752360796 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7efd300c5070 con 0x55e2be3cf230 2024-05-11T13:45:52.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.945+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd300b2740 con 0x55e2be3cf230 2024-05-11T13:45:52.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.945+0000 7efd2e000700 1 -- 172.21.15.63:0/752360796 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7efd300056e0 con 0x55e2be3cf230 2024-05-11T13:45:52.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.945+0000 7efd55513b80 10 client.35498 fetch_fsmap learned FSMap version 66 2024-05-11T13:45:52.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.945+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55e2be44f490 con 0x55e2be3cf230 2024-05-11T13:45:52.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd2e000700 1 -- 172.21.15.63:0/752360796 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd300b2740 con 0x55e2be3cf230 2024-05-11T13:45:52.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd2e000700 1 -- 172.21.15.63:0/752360796 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 66) ==== 8366+0+0 (secure 0 0 0) 0x7efd300b22b0 con 0x55e2be3cf230 2024-05-11T13:45:52.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd55513b80 10 client.35498 fetch_fsmap finished waiting for FSMap version 66 2024-05-11T13:45:52.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd55513b80 10 client.35498 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:45:52.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd55513b80 20 client.35498 populate_metadata read hostname 'smithi063' 2024-05-11T13:45:52.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd55513b80 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be44f6c0 0x55e2be451ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:45:52.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd55513b80 4 client.35498 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 87.6, "await_for": 42.0}] 2024-05-11T13:45:52.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.946+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 87.6, "await_for": 42.0}) -- 0x55e2be329590 con 0x55e2be44f6c0 2024-05-11T13:45:52.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.947+0000 7efd43e00700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be44f6c0 0x55e2be451ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:45:52.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.948+0000 7efd43e00700 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be44f6c0 0x55e2be451ab0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228217cb8bc8aa05 server_cookie=268265785b9e677c in_seq=0 out_seq=0 2024-05-11T13:45:52.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:52.948+0000 7efd2e000700 10 client.35498 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.226+0000 7efd2e000700 1 -- 172.21.15.63:0/752360796 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e2be329590 con 0x55e2be44f6c0 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.226+0000 7efd2e000700 10 client.35498 handle_command_reply: tid=0 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.226+0000 7efd55513b80 1 client.35498 shutdown 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 msgr2=0x55e2be3d8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3d8fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd300cc330 tx=0x7efd300a5860 comp rx=0 tx=0).stop 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 shutdown_connections 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2be3cfb50 0x55e2be44de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2be3c9d50 0x55e2be3d8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e2be44f6c0 0x55e2be451ab0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 --2- 172.21.15.63:0/752360796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2be3cf230 0x55e2be3d8fc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 >> 172.21.15.63:0/752360796 conn(0x55e2be2f7cf0 msgr2=0x55e2be3cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd2e000700 0 client.35498 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 shutdown_connections 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 1 -- 172.21.15.63:0/752360796 wait complete. 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.227+0000 7efd55513b80 20 client.35498 trim_cache size 0 max 16384 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 66, 2024-05-11T13:45:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 34, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "f3cd8d5d": { 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 34, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 87.6, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:45:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:45:53.221 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.228+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd280b2b20 msgr2=0x7efd280b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.228+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd280b2b20 0x7efd280b4fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd34009c60 tx=0x7efd34009260 comp rx=0 tx=0).stop 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.228+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 msgr2=0x7efd4410c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.228+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 0x7efd4410c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e9f0 tx=0x7efd3000ed00 comp rx=0 tx=0).stop 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 shutdown_connections 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd280b2b20 0x7efd280b4fe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4410ffb0 0x7efd4410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd44075780 0x7efd4410f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 --2- 172.21.15.63:0/2734199501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd440760a0 0x7efd4410c6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 >> 172.21.15.63:0/2734199501 conn(0x7efd4407a6b0 msgr2=0x7efd44100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:45:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 shutdown_connections 2024-05-11T13:45:53.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:45:53.229+0000 7efd4ac00700 1 -- 172.21.15.63:0/2734199501 wait complete. 2024-05-11T13:45:54.015 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f3cd8d5d', quiesce took 10.4 seconds. Will release after: 27.6 2024-05-11T13:45:54.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:54 smithi063 ceph-mon[29809]: pgmap v376: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 11 MiB/s wr, 766 op/s 2024-05-11T13:45:54.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:54 smithi071 ceph-mon[33293]: pgmap v376: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 11 MiB/s wr, 766 op/s 2024-05-11T13:45:54.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:54 smithi165 ceph-mon[33235]: pgmap v376: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 11 MiB/s wr, 766 op/s 2024-05-11T13:45:56.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 270.0 s, 221.2 tps, lat 176.832 ms stddev 35.790 2024-05-11T13:45:56.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:56 smithi063 ceph-mon[29809]: pgmap v377: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 7.1 MiB/s wr, 472 op/s 2024-05-11T13:45:56.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:56 smithi071 ceph-mon[33293]: pgmap v377: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 7.1 MiB/s wr, 472 op/s 2024-05-11T13:45:56.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:56 smithi165 ceph-mon[33235]: pgmap v377: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 7.1 MiB/s wr, 472 op/s 2024-05-11T13:45:58.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:58 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:58.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:45:58 smithi063 ceph-mon[29809]: pgmap v378: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-11T13:45:58.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:58 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:58.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:45:58 smithi071 ceph-mon[33293]: pgmap v378: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-11T13:45:58.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:58 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:45:58.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:45:58 smithi165 ceph-mon[33235]: pgmap v378: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-11T13:46:00.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:00 smithi063 ceph-mon[29809]: pgmap v379: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-11T13:46:00.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:00 smithi071 ceph-mon[33293]: pgmap v379: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-11T13:46:00.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:00 smithi165 ceph-mon[33235]: pgmap v379: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-11T13:46:01.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 275.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:02.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:02 smithi063 ceph-mon[29809]: pgmap v380: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-11T13:46:02.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:02 smithi071 ceph-mon[33293]: pgmap v380: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-11T13:46:02.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:02 smithi165 ceph-mon[33235]: pgmap v380: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-11T13:46:04.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:04 smithi063 ceph-mon[29809]: pgmap v381: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.8 MiB/s wr, 277 op/s 2024-05-11T13:46:04.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:04 smithi071 ceph-mon[33293]: pgmap v381: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.8 MiB/s wr, 277 op/s 2024-05-11T13:46:04.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:04 smithi165 ceph-mon[33235]: pgmap v381: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.8 MiB/s wr, 277 op/s 2024-05-11T13:46:06.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 280.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:06.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:06 smithi063 ceph-mon[29809]: pgmap v382: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 187 KiB/s wr, 42 op/s 2024-05-11T13:46:06.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:06 smithi071 ceph-mon[33293]: pgmap v382: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 187 KiB/s wr, 42 op/s 2024-05-11T13:46:06.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:06 smithi165 ceph-mon[33235]: pgmap v382: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 187 KiB/s wr, 42 op/s 2024-05-11T13:46:08.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:08 smithi063 ceph-mon[29809]: pgmap v383: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 265 KiB/s wr, 57 op/s 2024-05-11T13:46:08.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:08 smithi071 ceph-mon[33293]: pgmap v383: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 265 KiB/s wr, 57 op/s 2024-05-11T13:46:08.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:08 smithi165 ceph-mon[33235]: pgmap v383: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 265 KiB/s wr, 57 op/s 2024-05-11T13:46:10.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:10 smithi063 ceph-mon[29809]: osdmap e104: 12 total, 12 up, 12 in 2024-05-11T13:46:10.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:10 smithi063 ceph-mon[29809]: pgmap v385: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 244 KiB/s wr, 54 op/s 2024-05-11T13:46:10.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:10 smithi071 ceph-mon[33293]: osdmap e104: 12 total, 12 up, 12 in 2024-05-11T13:46:10.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:10 smithi071 ceph-mon[33293]: pgmap v385: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 244 KiB/s wr, 54 op/s 2024-05-11T13:46:10.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:10 smithi165 ceph-mon[33235]: osdmap e104: 12 total, 12 up, 12 in 2024-05-11T13:46:10.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:10 smithi165 ceph-mon[33235]: pgmap v385: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 244 KiB/s wr, 54 op/s 2024-05-11T13:46:11.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 285.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:12.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:12 smithi063 ceph-mon[29809]: pgmap v386: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 244 KiB/s wr, 54 op/s 2024-05-11T13:46:12.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:12 smithi071 ceph-mon[33293]: pgmap v386: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 244 KiB/s wr, 54 op/s 2024-05-11T13:46:12.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:12 smithi165 ceph-mon[33235]: pgmap v386: 129 pgs: 129 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 244 KiB/s wr, 54 op/s 2024-05-11T13:46:14.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:14 smithi063 ceph-mon[29809]: pgmap v387: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 182 KiB/s wr, 32 op/s 2024-05-11T13:46:14.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:14 smithi071 ceph-mon[33293]: pgmap v387: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 182 KiB/s wr, 32 op/s 2024-05-11T13:46:14.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:14 smithi165 ceph-mon[33235]: pgmap v387: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 182 KiB/s wr, 32 op/s 2024-05-11T13:46:16.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 290.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:16.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:16 smithi063 ceph-mon[29809]: osdmap e105: 12 total, 12 up, 12 in 2024-05-11T13:46:16.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:16 smithi071 ceph-mon[33293]: osdmap e105: 12 total, 12 up, 12 in 2024-05-11T13:46:16.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:16 smithi165 ceph-mon[33235]: osdmap e105: 12 total, 12 up, 12 in 2024-05-11T13:46:17.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:17 smithi063 ceph-mon[29809]: pgmap v389: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 110 KiB/s wr, 19 op/s 2024-05-11T13:46:17.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:17 smithi071 ceph-mon[33293]: pgmap v389: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 110 KiB/s wr, 19 op/s 2024-05-11T13:46:17.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:17 smithi165 ceph-mon[33235]: pgmap v389: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 110 KiB/s wr, 19 op/s 2024-05-11T13:46:18.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:18 smithi063 ceph-mon[29809]: pgmap v390: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-05-11T13:46:18.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:18 smithi071 ceph-mon[33293]: pgmap v390: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-05-11T13:46:18.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:18 smithi165 ceph-mon[33235]: pgmap v390: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-05-11T13:46:20.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:20 smithi063 ceph-mon[29809]: pgmap v391: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-11T13:46:20.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:20 smithi071 ceph-mon[33293]: pgmap v391: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-11T13:46:20.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:20 smithi165 ceph-mon[33235]: pgmap v391: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-11T13:46:21.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 295.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:21.617 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:46:22.119 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:46:22.663 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:22 smithi063 ceph-mon[29809]: pgmap v392: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-11T13:46:22.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:22 smithi071 ceph-mon[33293]: pgmap v392: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-11T13:46:22.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:22 smithi165 ceph-mon[33235]: pgmap v392: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-11T13:46:24.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 -- 172.21.15.63:0/337722246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 msgr2=0x7fe040075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:24.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 --2- 172.21.15.63:0/337722246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe040075d00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe030009990 tx=0x7fe030009ca0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 -- 172.21.15.63:0/337722246 shutdown_connections 2024-05-11T13:46:24.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 --2- 172.21.15.63:0/337722246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe04010fd20 0x7fe040112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 --2- 172.21.15.63:0/337722246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe040075d00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 --2- 172.21.15.63:0/337722246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040076240 0x7fe04010f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 -- 172.21.15.63:0/337722246 >> 172.21.15.63:0/337722246 conn(0x7fe0400fefd0 msgr2=0x7fe0401013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:24.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 -- 172.21.15.63:0/337722246 shutdown_connections 2024-05-11T13:46:24.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.346+0000 7fe03f200700 1 -- 172.21.15.63:0/337722246 wait complete. 2024-05-11T13:46:24.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.347+0000 7fe03f200700 1 Processor -- start 2024-05-11T13:46:24.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03f200700 1 -- start start 2024-05-11T13:46:24.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe04019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:24.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03f200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040076240 0x7fe04019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:24.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe04010fd20 0x7fe0401a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe040115780 con 0x7fe040075920 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe040115600 con 0x7fe04010fd20 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03f200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe040115900 con 0x7fe040076240 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe04019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe04019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39694/0 (socket says 172.21.15.63:39694) 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.348+0000 7fe03de00700 1 -- 172.21.15.63:0/3748765472 learned_addr learned my addr 172.21.15.63:0/3748765472 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe035400700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040076240 0x7fe04019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe03e800700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe04010fd20 0x7fe0401a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe03de00700 1 -- 172.21.15.63:0/3748765472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040076240 msgr2=0x7fe04019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe03de00700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040076240 0x7fe04019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe03de00700 1 -- 172.21.15.63:0/3748765472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe04010fd20 msgr2=0x7fe0401a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe03de00700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe04010fd20 0x7fe0401a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe03de00700 1 -- 172.21.15.63:0/3748765472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0401a0d20 con 0x7fe040075920 2024-05-11T13:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe03de00700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe04019bd10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe030000c00 tx=0x7fe030005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:24.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.349+0000 7fe037400700 1 -- 172.21.15.63:0/3748765472 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030027070 con 0x7fe040075920 2024-05-11T13:46:24.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.350+0000 7fe037400700 1 -- 172.21.15.63:0/3748765472 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe03000fb80 con 0x7fe040075920 2024-05-11T13:46:24.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.350+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0401a0fb0 con 0x7fe040075920 2024-05-11T13:46:24.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.350+0000 7fe037400700 1 -- 172.21.15.63:0/3748765472 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030022c30 con 0x7fe040075920 2024-05-11T13:46:24.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.350+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0401a8890 con 0x7fe040075920 2024-05-11T13:46:24.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.351+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04006a240 con 0x7fe040075920 2024-05-11T13:46:24.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.353+0000 7fe037400700 1 -- 172.21.15.63:0/3748765472 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe03002a070 con 0x7fe040075920 2024-05-11T13:46:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.355+0000 7fe037400700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe0240b2a70 0x7fe0240b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.355+0000 7fe037400700 1 -- 172.21.15.63:0/3748765472 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7fe0300e3940 con 0x7fe040075920 2024-05-11T13:46:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.355+0000 7fe035400700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe0240b2a70 0x7fe0240b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:24.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.356+0000 7fe035400700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe0240b2a70 0x7fe0240b4f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe028005950 tx=0x7fe0280058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:24.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.360+0000 7fe037400700 1 -- 172.21.15.63:0/3748765472 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe030013070 con 0x7fe040075920 2024-05-11T13:46:24.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.648+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe04019d070 con 0x7fe040075920 2024-05-11T13:46:24.639 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:24 smithi063 ceph-mon[29809]: pgmap v393: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 200 KiB/s wr, 33 op/s 2024-05-11T13:46:24.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.650+0000 7fe037400700 1 -- 172.21.15.63:0/3748765472 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11935 (secure 0 0 0) 0x7fe0300aaf60 con 0x7fe040075920 2024-05-11T13:46:24.641 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:46:24.641 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:45:56.474021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34451,34472,34427,34403,24472]},"id":1}]} 2024-05-11T13:46:24.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe0240b2a70 msgr2=0x7fe0240b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:24.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe0240b2a70 0x7fe0240b4f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe028005950 tx=0x7fe0280058e0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 msgr2=0x7fe04019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:24.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe04019bd10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe030000c00 tx=0x7fe030005ef0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 shutdown_connections 2024-05-11T13:46:24.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe0240b2a70 0x7fe0240b4f30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe04010fd20 0x7fe0401a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe040075920 0x7fe04019bd10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 --2- 172.21.15.63:0/3748765472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040076240 0x7fe04019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:24.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 >> 172.21.15.63:0/3748765472 conn(0x7fe0400fefd0 msgr2=0x7fe0400ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:24.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 shutdown_connections 2024-05-11T13:46:24.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:24.655+0000 7fe03f200700 1 -- 172.21.15.63:0/3748765472 wait complete. 2024-05-11T13:46:24.647 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 67 2024-05-11T13:46:24.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:24 smithi071 ceph-mon[33293]: pgmap v393: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 200 KiB/s wr, 33 op/s 2024-05-11T13:46:24.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:24 smithi165 ceph-mon[33235]: pgmap v393: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 200 KiB/s wr, 33 op/s 2024-05-11T13:46:25.470 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id f3cd8d5d --release --await' 2024-05-11T13:46:25.470 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id f3cd8d5d --release --await 2024-05-11T13:46:25.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:25 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3748765472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:46:25.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:25 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3748765472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:46:25.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:25 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3748765472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:46:25.957 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:46:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 300.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:26.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:26 smithi063 ceph-mon[29809]: pgmap v394: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 179 KiB/s wr, 30 op/s 2024-05-11T13:46:26.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:26 smithi071 ceph-mon[33293]: pgmap v394: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 179 KiB/s wr, 30 op/s 2024-05-11T13:46:26.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:26 smithi165 ceph-mon[33235]: pgmap v394: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 179 KiB/s wr, 30 op/s 2024-05-11T13:46:28.257 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:28 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:46:28.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:28 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 -- 172.21.15.63:0/3347171318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734105770 msgr2=0x7ff734105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 --2- 172.21.15.63:0/3347171318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734105770 0x7ff734105bf0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff720009990 tx=0x7ff720009ca0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 -- 172.21.15.63:0/3347171318 shutdown_connections 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 --2- 172.21.15.63:0/3347171318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff734106130 0x7ff734112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 --2- 172.21.15.63:0/3347171318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff73410b770 0x7ff73410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 --2- 172.21.15.63:0/3347171318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734105770 0x7ff734105bf0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 -- 172.21.15.63:0/3347171318 >> 172.21.15.63:0/3347171318 conn(0x7ff734101260 msgr2=0x7ff734103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 -- 172.21.15.63:0/3347171318 shutdown_connections 2024-05-11T13:46:28.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.339+0000 7ff73a200700 1 -- 172.21.15.63:0/3347171318 wait complete. 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 Processor -- start 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 -- start start 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff734105770 0x7ff73419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff734106130 0x7ff73419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff73410b770 0x7ff7341a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff734115870 con 0x7ff73410b770 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341156f0 con 0x7ff734105770 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.341+0000 7ff73a200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341159f0 con 0x7ff734106130 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff73410b770 0x7ff7341a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff734105770 0x7ff73419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff73410b770 0x7ff7341a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39706/0 (socket says 172.21.15.63:39706) 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 -- 172.21.15.63:0/4059016330 learned_addr learned my addr 172.21.15.63:0/4059016330 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff732a00700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff734106130 0x7ff73419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 -- 172.21.15.63:0/4059016330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff734106130 msgr2=0x7ff73419c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff734106130 0x7ff73419c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 -- 172.21.15.63:0/4059016330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff734105770 msgr2=0x7ff73419bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff734105770 0x7ff73419bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.342+0000 7ff733e00700 1 -- 172.21.15.63:0/4059016330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7341a0d50 con 0x7ff73410b770 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.343+0000 7ff733e00700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff73410b770 0x7ff7341a0610 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00ef30 tx=0x7ff72c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.343+0000 7ff72be00700 1 -- 172.21.15.63:0/4059016330 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0092e0 con 0x7ff73410b770 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.343+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7341a0fe0 con 0x7ff73410b770 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.343+0000 7ff72be00700 1 -- 172.21.15.63:0/4059016330 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72c004bb0 con 0x7ff73410b770 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.343+0000 7ff72be00700 1 -- 172.21.15.63:0/4059016330 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c004020 con 0x7ff73410b770 2024-05-11T13:46:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.343+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7341a88c0 con 0x7ff73410b770 2024-05-11T13:46:28.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.347+0000 7ff72be00700 1 -- 172.21.15.63:0/4059016330 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff72c01b030 con 0x7ff73410b770 2024-05-11T13:46:28.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.348+0000 7ff744adfb80 1 Processor -- start 2024-05-11T13:46:28.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.348+0000 7ff744adfb80 1 -- start start 2024-05-11T13:46:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff744adfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b281507b20 con 0x55b2815b5cc0 2024-05-11T13:46:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff744adfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b2814a5860 con 0x55b2815bbac0 2024-05-11T13:46:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff744adfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b281503ae0 con 0x55b2815bb1a0 2024-05-11T13:46:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff72be00700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff7180b29f0 0x7ff7180b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff72be00700 1 -- 172.21.15.63:0/4059016330 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7ff72c0e2520 con 0x7ff73410b770 2024-05-11T13:46:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff732a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff733400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff733e00700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff7180b29f0 0x7ff7180b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff733400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:45586/0 (socket says 172.21.15.63:45586) 2024-05-11T13:46:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.349+0000 7ff733400700 1 -- 172.21.15.63:0/3826541774 learned_addr learned my addr 172.21.15.63:0/3826541774 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff732a00700 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff733e00700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff7180b29f0 0x7ff7180b4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00d010 tx=0x7ff72c00cfa0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff733400700 1 -- 172.21.15.63:0/3826541774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 msgr2=0x55b2815bb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff733400700 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff733400700 1 -- 172.21.15.63:0/3826541774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 msgr2=0x55b2815b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff733400700 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff733400700 1 -- 172.21.15.63:0/3826541774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2814549d0 con 0x55b2815bbac0 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.350+0000 7ff732a00700 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.351+0000 7ff733400700 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815b56a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff724009d00 tx=0x7ff724011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2bdebd43ac276f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.351+0000 7ff728c00700 1 -- 172.21.15.63:0/3826541774 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724004230 con 0x55b2815bbac0 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.351+0000 7ff728c00700 1 -- 172.21.15.63:0/3826541774 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7240043d0 con 0x55b2815bbac0 2024-05-11T13:46:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.351+0000 7ff728c00700 1 -- 172.21.15.63:0/3826541774 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724019680 con 0x55b2815bbac0 2024-05-11T13:46:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 -- 172.21.15.63:0/3826541774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 msgr2=0x55b2815b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815b56a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff724009d00 tx=0x7ff724011720 comp rx=0 tx=0).stop 2024-05-11T13:46:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 -- 172.21.15.63:0/3826541774 shutdown_connections 2024-05-11T13:46:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3826541774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815b56a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 -- 172.21.15.63:0/3826541774 >> 172.21.15.63:0/3826541774 conn(0x55b2814e3c60 msgr2=0x55b2815b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.352+0000 7ff744adfb80 1 -- 172.21.15.63:0/3826541774 shutdown_connections 2024-05-11T13:46:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.353+0000 7ff744adfb80 1 -- 172.21.15.63:0/3826541774 wait complete. 2024-05-11T13:46:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.353+0000 7ff744adfb80 1 -- start start 2024-05-11T13:46:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.353+0000 7ff744adfb80 1 Processor -- start 2024-05-11T13:46:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815c1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815bf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff744adfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b2815c25e0 con 0x55b2815b5cc0 2024-05-11T13:46:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff744adfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b28146dc20 con 0x55b2815bbac0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff744adfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b2815c2760 con 0x55b2815bb1a0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff744adfb80 10 client.0 _ll_register_callbacks cb 0x55b28149b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff732a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff732a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815c1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39738/0 (socket says 172.21.15.63:39738) 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff732a00700 1 -- 172.21.15.63:0/2128545939 learned_addr learned my addr 172.21.15.63:0/2128545939 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.354+0000 7ff733400700 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff732a00700 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff732a00700 1 -- 172.21.15.63:0/2128545939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 msgr2=0x55b2815bead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff732a00700 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff732a00700 1 -- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 msgr2=0x55b2815bf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff732a00700 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815bf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff732a00700 1 -- 172.21.15.63:0/2128545939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2814cd520 con 0x55b2815b5cc0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff732a00700 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815c1730 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb46016594f6b8d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff71d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff71d600700 1 -- 172.21.15.63:0/2128545939 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72000b8b0 con 0x55b2815b5cc0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff71d600700 1 -- 172.21.15.63:0/2128545939 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7ff7200056b0 con 0x55b2815b5cc0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.355+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7240043d0 con 0x55b2815b5cc0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.356+0000 7ff71d600700 1 -- 172.21.15.63:0/2128545939 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff720032050 con 0x55b2815b5cc0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.356+0000 7ff744adfb80 10 client.15930 fetch_fsmap learned FSMap version 67 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.356+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55b2815ccd90 con 0x55b2815b5cc0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.356+0000 7ff71d600700 1 -- 172.21.15.63:0/2128545939 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7240043d0 con 0x55b2815b5cc0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff71d600700 1 -- 172.21.15.63:0/2128545939 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 67) ==== 8362+0+0 (secure 0 0 0) 0x7ff72000fa70 con 0x55b2815b5cc0 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff744adfb80 10 client.15930 fetch_fsmap finished waiting for FSMap version 67 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff744adfb80 10 client.15930 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:46:28.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff744adfb80 20 client.15930 populate_metadata read hostname 'smithi063' 2024-05-11T13:46:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff744adfb80 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b2815cf320 0x55b2815d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff744adfb80 4 client.15930 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:46:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b281582160 con 0x55b2815cf320 2024-05-11T13:46:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.357+0000 7ff733400700 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b2815cf320 0x55b2815d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.358+0000 7ff733400700 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b2815cf320 0x55b2815d1710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59435f27c72c5bb9 server_cookie=e96e41736560c777 in_seq=0 out_seq=0 2024-05-11T13:46:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.358+0000 7ff71d600700 10 client.15930 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:28.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff71d600700 1 -- 172.21.15.63:0/2128545939 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b281582160 con 0x55b2815cf320 2024-05-11T13:46:28.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff71d600700 10 client.15930 handle_command_reply: tid=0 2024-05-11T13:46:28.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 client.15930 shutdown 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 msgr2=0x55b2815c1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815c1730 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720009ca0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 shutdown_connections 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b2815cf320 0x55b2815d1710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815bf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815b5cc0 0x55b2815c1730 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 --2- 172.21.15.63:0/2128545939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815bb1a0 0x55b2815bead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 >> 172.21.15.63:0/2128545939 conn(0x55b2814e3c60 msgr2=0x55b2815b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:28.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.360+0000 7ff71d600700 0 client.15930 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:28.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.361+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 shutdown_connections 2024-05-11T13:46:28.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.361+0000 7ff744adfb80 1 -- 172.21.15.63:0/2128545939 wait complete. 2024-05-11T13:46:28.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.361+0000 7ff744adfb80 20 client.15930 trim_cache size 0 max 16384 2024-05-11T13:46:28.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.383+0000 7ff744adfb80 1 Processor -- start 2024-05-11T13:46:28.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.383+0000 7ff744adfb80 1 -- start start 2024-05-11T13:46:28.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bb1a0 0x55b2815bdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff744adfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b1860 con 0x55b2815bb1a0 2024-05-11T13:46:28.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff744adfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b33a0 con 0x55b2815bbac0 2024-05-11T13:46:28.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff744adfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b1f30 con 0x55b2815b5cc0 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff733400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bb1a0 0x55b2815bdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff733400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bb1a0 0x55b2815bdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39754/0 (socket says 172.21.15.63:39754) 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff733400700 1 -- 172.21.15.63:0/3293840278 learned_addr learned my addr 172.21.15.63:0/3293840278 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff732a00700 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff732a00700 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff733400700 1 -- 172.21.15.63:0/3293840278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 msgr2=0x55b2815b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff733400700 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.384+0000 7ff733400700 1 -- 172.21.15.63:0/3293840278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 msgr2=0x55b2815ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.385+0000 7ff733400700 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.385+0000 7ff733400700 1 -- 172.21.15.63:0/3293840278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2815b1460 con 0x55b2815bb1a0 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.385+0000 7ff733400700 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bb1a0 0x55b2815bdc90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff724009d00 tx=0x7ff724009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233a15b8e7ebadc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.385+0000 7ff71fe00700 1 -- 172.21.15.63:0/3293840278 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724030070 con 0x55b2815bb1a0 2024-05-11T13:46:28.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.385+0000 7ff71fe00700 1 -- 172.21.15.63:0/3293840278 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff724005040 con 0x55b2815bb1a0 2024-05-11T13:46:28.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.385+0000 7ff71fe00700 1 -- 172.21.15.63:0/3293840278 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72402a670 con 0x55b2815bb1a0 2024-05-11T13:46:28.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.386+0000 7ff744adfb80 1 -- 172.21.15.63:0/3293840278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bb1a0 msgr2=0x55b2815bdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.386+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bb1a0 0x55b2815bdc90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff724009d00 tx=0x7ff724009730 comp rx=0 tx=0).stop 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.386+0000 7ff744adfb80 1 -- 172.21.15.63:0/3293840278 shutdown_connections 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.386+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bbac0 0x55b2815ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.386+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.386+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3293840278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bb1a0 0x55b2815bdc90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.386+0000 7ff744adfb80 1 -- 172.21.15.63:0/3293840278 >> 172.21.15.63:0/3293840278 conn(0x55b2814e3c60 msgr2=0x55b2815b9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.387+0000 7ff744adfb80 1 -- 172.21.15.63:0/3293840278 shutdown_connections 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.387+0000 7ff744adfb80 1 -- 172.21.15.63:0/3293840278 wait complete. 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.387+0000 7ff744adfb80 1 -- start start 2024-05-11T13:46:28.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.387+0000 7ff744adfb80 1 Processor -- start 2024-05-11T13:46:28.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bb1a0 0x55b2815c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff744adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bbac0 0x55b2815c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff744adfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b2cd0 con 0x55b2815bbac0 2024-05-11T13:46:28.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff744adfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b2600 con 0x55b2815bb1a0 2024-05-11T13:46:28.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff744adfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341af820 con 0x55b2815b5cc0 2024-05-11T13:46:28.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff744adfb80 10 client.0 _ll_register_callbacks cb 0x55b28149b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff732a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bbac0 0x55b2815c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff733400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bb1a0 0x55b2815c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff732a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bbac0 0x55b2815c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39756/0 (socket says 172.21.15.63:39756) 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff732a00700 1 -- 172.21.15.63:0/3065495673 learned_addr learned my addr 172.21.15.63:0/3065495673 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.388+0000 7ff732a00700 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff732a00700 1 -- 172.21.15.63:0/3065495673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 msgr2=0x55b2815c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff732a00700 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff732a00700 1 -- 172.21.15.63:0/3065495673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bb1a0 msgr2=0x55b2815c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff732a00700 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bb1a0 0x55b2815c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff732a00700 1 -- 172.21.15.63:0/3065495673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b281639d40 con 0x55b2815bbac0 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff732a00700 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bbac0 0x55b2815c5470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff720005ae0 tx=0x7ff720005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9203c21fa16e9cec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff71d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:46:28.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.389+0000 7ff71d600700 1 -- 172.21.15.63:0/3065495673 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72002c960 con 0x55b2815bbac0 2024-05-11T13:46:28.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff71d600700 1 -- 172.21.15.63:0/3065495673 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7ff72002cb00 con 0x55b2815bbac0 2024-05-11T13:46:28.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff724005040 con 0x55b2815bbac0 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff71d600700 1 -- 172.21.15.63:0/3065495673 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff72002cd00 con 0x55b2815bbac0 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff744adfb80 10 client.15936 fetch_fsmap learned FSMap version 67 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55b28163b040 con 0x55b2815bbac0 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff71d600700 1 -- 172.21.15.63:0/3065495673 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff724005040 con 0x55b2815bbac0 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff71d600700 1 -- 172.21.15.63:0/3065495673 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 67) ==== 8362+0+0 (secure 0 0 0) 0x7ff72000f460 con 0x55b2815bbac0 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.390+0000 7ff744adfb80 10 client.15936 fetch_fsmap finished waiting for FSMap version 67 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.391+0000 7ff744adfb80 10 client.15936 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.391+0000 7ff744adfb80 20 client.15936 populate_metadata read hostname 'smithi063' 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.391+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b28163b670 0x55b28163da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.391+0000 7ff744adfb80 4 client.15936 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f3cd8d5d", "release": true, "await": true}] 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.391+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f3cd8d5d", "release": true, "await": true}) -- 0x55b281515500 con 0x55b28163b670 2024-05-11T13:46:28.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.391+0000 7ff733400700 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b28163b670 0x55b28163da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:28.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.392+0000 7ff733400700 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b28163b670 0x55b28163da90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8861d1567a681df1 server_cookie=90537381ec3d492a in_seq=0 out_seq=0 2024-05-11T13:46:28.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.392+0000 7ff71d600700 10 client.15936 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:28.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:28 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:46:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.458+0000 7ff71d600700 1 -- 172.21.15.63:0/3065495673 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b281515500 con 0x55b28163b670 2024-05-11T13:46:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.458+0000 7ff71d600700 10 client.15936 handle_command_reply: tid=0 2024-05-11T13:46:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.458+0000 7ff744adfb80 1 client.15936 shutdown 2024-05-11T13:46:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bbac0 msgr2=0x55b2815c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bbac0 0x55b2815c5470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff720005ae0 tx=0x7ff720005570 comp rx=0 tx=0).stop 2024-05-11T13:46:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 shutdown_connections 2024-05-11T13:46:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b2815bbac0 0x55b2815c5470 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2815b5cc0 0x55b2815c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b28163b670 0x55b28163da90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 --2- 172.21.15.63:0/3065495673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b2815bb1a0 0x55b2815c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.459+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 >> 172.21.15.63:0/3065495673 conn(0x55b2814e3c60 msgr2=0x55b2815bdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.460+0000 7ff71d600700 0 client.15936 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.460+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 shutdown_connections 2024-05-11T13:46:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.461+0000 7ff744adfb80 1 -- 172.21.15.63:0/3065495673 wait complete. 2024-05-11T13:46:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.461+0000 7ff744adfb80 20 client.15936 trim_cache size 0 max 16384 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 67, 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 36, 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "f3cd8d5d": { 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 36, 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:46:28.452 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 87.6, 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:28.454 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:28.454 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:28.454 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:46:28.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff7180b29f0 msgr2=0x7ff7180b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff7180b29f0 0x7ff7180b4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00d010 tx=0x7ff72c00cfa0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff73410b770 msgr2=0x7ff7341a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff73410b770 0x7ff7341a0610 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00ef30 tx=0x7ff72c00c3a0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 shutdown_connections 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff7180b29f0 0x7ff7180b4eb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff73410b770 0x7ff7341a0610 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.462+0000 7ff73a200700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff734105770 0x7ff73419bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.463+0000 7ff73a200700 1 --2- 172.21.15.63:0/4059016330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff734106130 0x7ff73419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.463+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 >> 172.21.15.63:0/4059016330 conn(0x7ff734101260 msgr2=0x7ff734101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.463+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 shutdown_connections 2024-05-11T13:46:28.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:28.463+0000 7ff73a200700 1 -- 172.21.15.63:0/4059016330 wait complete. 2024-05-11T13:46:29.265 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f3cd8d5d', seconds elapsed: 45.7 2024-05-11T13:46:29.265 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.6 2024-05-11T13:46:29.265 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:46:29.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:29 smithi165 ceph-mon[33235]: pgmap v395: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 287 KiB/s wr, 46 op/s 2024-05-11T13:46:29.403 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:29 smithi063 ceph-mon[29809]: pgmap v395: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 287 KiB/s wr, 46 op/s 2024-05-11T13:46:29.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:29 smithi071 ceph-mon[33293]: pgmap v395: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 287 KiB/s wr, 46 op/s 2024-05-11T13:46:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:46:30.590 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:30 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:46:30.591 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:30 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:46:30.591 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:30 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:46:30.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:30 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:46:30.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:30 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:46:30.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:30 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:46:30.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:30 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:46:30.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:30 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:46:30.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:30 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:46:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 305.0 s, 206.6 tps, lat 3684.866 ms stddev 10433.137 2024-05-11T13:46:32.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:32 smithi165 ceph-mon[33235]: pgmap v396: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 211 KiB/s wr, 33 op/s 2024-05-11T13:46:32.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.587+0000 7f0a62600700 1 -- 172.21.15.63:0/1982936942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c109de0 msgr2=0x7f0a5c10a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:32.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.587+0000 7f0a62600700 1 --2- 172.21.15.63:0/1982936942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c109de0 0x7f0a5c10a1c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0099c0 tx=0x7f0a4c009cd0 comp rx=0 tx=0).stop 2024-05-11T13:46:32.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.587+0000 7f0a62600700 1 -- 172.21.15.63:0/1982936942 shutdown_connections 2024-05-11T13:46:32.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.587+0000 7f0a62600700 1 --2- 172.21.15.63:0/1982936942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5c10eec0 0x7f0a5c116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:32.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.587+0000 7f0a62600700 1 --2- 172.21.15.63:0/1982936942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c109de0 0x7f0a5c10a1c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:32.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.587+0000 7f0a62600700 1 --2- 172.21.15.63:0/1982936942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c10a790 0x7f0a5c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:32.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.587+0000 7f0a62600700 1 -- 172.21.15.63:0/1982936942 >> 172.21.15.63:0/1982936942 conn(0x7f0a5c076d20 msgr2=0x7f0a5c108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:32.579 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:32 smithi063 ceph-mon[29809]: pgmap v396: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 211 KiB/s wr, 33 op/s 2024-05-11T13:46:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.588+0000 7f0a62600700 1 -- 172.21.15.63:0/1982936942 shutdown_connections 2024-05-11T13:46:32.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.588+0000 7f0a62600700 1 -- 172.21.15.63:0/1982936942 wait complete. 2024-05-11T13:46:32.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.590+0000 7f0a62600700 1 Processor -- start 2024-05-11T13:46:32.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.591+0000 7f0a62600700 1 -- start start 2024-05-11T13:46:32.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.592+0000 7f0a62600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c109de0 0x7f0a5c1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:32.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.592+0000 7f0a62600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5c10a790 0x7f0a5c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.592+0000 7f0a62600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c10eec0 0x7f0a5c1a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.592+0000 7f0a62600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c119c50 con 0x7f0a5c10eec0 2024-05-11T13:46:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.592+0000 7f0a62600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c119ad0 con 0x7f0a5c109de0 2024-05-11T13:46:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.592+0000 7f0a62600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c119dd0 con 0x7f0a5c10a790 2024-05-11T13:46:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.593+0000 7f0a5aa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5c10a790 0x7f0a5c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:32.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.593+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c109de0 0x7f0a5c1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:32.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.593+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c109de0 0x7f0a5c1a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:35650/0 (socket says 172.21.15.63:35650) 2024-05-11T13:46:32.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.593+0000 7f0a5b400700 1 -- 172.21.15.63:0/3668433564 learned_addr learned my addr 172.21.15.63:0/3668433564 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:32.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.593+0000 7f0a5be00700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c10eec0 0x7f0a5c1a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:32.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.594+0000 7f0a5b400700 1 -- 172.21.15.63:0/3668433564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5c10a790 msgr2=0x7f0a5c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.594+0000 7f0a5b400700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5c10a790 0x7f0a5c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.594+0000 7f0a5b400700 1 -- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c10eec0 msgr2=0x7f0a5c1a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.594+0000 7f0a5b400700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c10eec0 0x7f0a5c1a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.594+0000 7f0a5b400700 1 -- 172.21.15.63:0/3668433564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c1a5190 con 0x7f0a5c109de0 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.594+0000 7f0a5b400700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c109de0 0x7f0a5c1a0180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006010 tx=0x7f0a4c005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.594+0000 7f0a53e00700 1 -- 172.21.15.63:0/3668433564 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c026070 con 0x7f0a5c109de0 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.595+0000 7f0a53e00700 1 -- 172.21.15.63:0/3668433564 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a4c003c50 con 0x7f0a5c109de0 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.595+0000 7f0a53e00700 1 -- 172.21.15.63:0/3668433564 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00fa70 con 0x7f0a5c109de0 2024-05-11T13:46:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.595+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c1a5420 con 0x7f0a5c109de0 2024-05-11T13:46:32.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.595+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c1acd00 con 0x7f0a5c109de0 2024-05-11T13:46:32.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.596+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5c06a240 con 0x7f0a5c109de0 2024-05-11T13:46:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.601+0000 7f0a53e00700 1 -- 172.21.15.63:0/3668433564 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0a4c029070 con 0x7f0a5c109de0 2024-05-11T13:46:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.603+0000 7f0a53e00700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0a400b29f0 0x7f0a400b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.604+0000 7f0a53e00700 1 -- 172.21.15.63:0/3668433564 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f0a4c0e3640 con 0x7f0a5c109de0 2024-05-11T13:46:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.604+0000 7f0a5aa00700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0a400b29f0 0x7f0a400b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.605+0000 7f0a5aa00700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0a400b29f0 0x7f0a400b4eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c077c20 tx=0x7f0a48009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:32.607+0000 7f0a53e00700 1 -- 172.21.15.63:0/3668433564 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a4c013070 con 0x7f0a5c109de0 2024-05-11T13:46:32.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:32 smithi071 ceph-mon[33293]: pgmap v396: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 211 KiB/s wr, 33 op/s 2024-05-11T13:46:33.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.067+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a5c02e570 con 0x7f0a5c109de0 2024-05-11T13:46:33.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.069+0000 7f0a53e00700 1 -- 172.21.15.63:0/3668433564 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11935 (secure 0 0 0) 0x7f0a4c0aacc0 con 0x7f0a5c109de0 2024-05-11T13:46:33.060 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:46:33.060 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:45:56.474021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34451,34472,34427,34403,24472]},"id":1}]} 2024-05-11T13:46:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0a400b29f0 msgr2=0x7f0a400b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0a400b29f0 0x7f0a400b4eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c077c20 tx=0x7f0a48009260 comp rx=0 tx=0).stop 2024-05-11T13:46:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c109de0 msgr2=0x7f0a5c1a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c109de0 0x7f0a5c1a0180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006010 tx=0x7f0a4c005280 comp rx=0 tx=0).stop 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 shutdown_connections 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f0a400b29f0 0x7f0a400b4eb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a5c10eec0 0x7f0a5c1a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a5c109de0 0x7f0a5c1a0180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 --2- 172.21.15.63:0/3668433564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5c10a790 0x7f0a5c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 >> 172.21.15.63:0/3668433564 conn(0x7f0a5c076d20 msgr2=0x7f0a5c109200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.073+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 shutdown_connections 2024-05-11T13:46:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:33.074+0000 7f0a62600700 1 -- 172.21.15.63:0/3668433564 wait complete. 2024-05-11T13:46:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 67 2024-05-11T13:46:33.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:33 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3668433564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:46:33.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:33 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3668433564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:46:33.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:33 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3668433564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:46:33.906 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 87.6 --await-for 42.0' 2024-05-11T13:46:33.906 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 87.6 --await-for 42.0 2024-05-11T13:46:34.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:34 smithi063 ceph-mon[29809]: pgmap v397: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 211 KiB/s wr, 33 op/s 2024-05-11T13:46:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:46:34.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:34 smithi071 ceph-mon[33293]: pgmap v397: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 211 KiB/s wr, 33 op/s 2024-05-11T13:46:34.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:34 smithi165 ceph-mon[33235]: pgmap v397: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 211 KiB/s wr, 33 op/s 2024-05-11T13:46:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 310.0 s, 468.2 tps, lat 213.962 ms stddev 39.394 2024-05-11T13:46:36.515 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:36 smithi063 ceph-mon[29809]: pgmap v398: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-05-11T13:46:36.608 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:36 smithi165 ceph-mon[33235]: pgmap v398: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-05-11T13:46:36.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:36 smithi071 ceph-mon[33293]: pgmap v398: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-05-11T13:46:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 -- 172.21.15.63:0/1922777608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b00760a0 msgr2=0x7f78b010fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 --2- 172.21.15.63:0/1922777608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b00760a0 0x7f78b010fa40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098f0 tx=0x7f789c009c00 comp rx=0 tx=0).stop 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 -- 172.21.15.63:0/1922777608 shutdown_connections 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 --2- 172.21.15.63:0/1922777608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b010ff80 0x7f78b0112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 --2- 172.21.15.63:0/1922777608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b0075780 0x7f78b0075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 --2- 172.21.15.63:0/1922777608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b00760a0 0x7f78b010fa40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 -- 172.21.15.63:0/1922777608 >> 172.21.15.63:0/1922777608 conn(0x7f78b007a700 msgr2=0x7f78b007ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 -- 172.21.15.63:0/1922777608 shutdown_connections 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.000+0000 7f78b7200700 1 -- 172.21.15.63:0/1922777608 wait complete. 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.002+0000 7f78b7200700 1 Processor -- start 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.002+0000 7f78b7200700 1 -- start start 2024-05-11T13:46:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78b7200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b0075780 0x7f78b0108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78b7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b00760a0 0x7f78b01094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78b7200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b010ff80 0x7f78b01a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78b7200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0115870 con 0x7f78b00760a0 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78b7200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01156f0 con 0x7f78b0075780 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78b7200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01159f0 con 0x7f78b010ff80 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b0075780 0x7f78b0108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.003+0000 7f78afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b0075780 0x7f78b0108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:35668/0 (socket says 172.21.15.63:35668) 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78afe00700 1 -- 172.21.15.63:0/1483002300 learned_addr learned my addr 172.21.15.63:0/1483002300 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78af400700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b00760a0 0x7f78b01094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78b5200700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b010ff80 0x7f78b01a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78afe00700 1 -- 172.21.15.63:0/1483002300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b010ff80 msgr2=0x7f78b01a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78afe00700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b010ff80 0x7f78b01a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78afe00700 1 -- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b00760a0 msgr2=0x7f78b01094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78afe00700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b00760a0 0x7f78b01094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.004+0000 7f78afe00700 1 -- 172.21.15.63:0/1483002300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b01a2680 con 0x7f78b0075780 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.005+0000 7f78afe00700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b0075780 0x7f78b0108f40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d980 tx=0x7f78a000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.005+0000 7f78acc00700 1 -- 172.21.15.63:0/1483002300 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0016070 con 0x7f78b0075780 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.006+0000 7f78acc00700 1 -- 172.21.15.63:0/1483002300 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78a0004830 con 0x7f78b0075780 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.006+0000 7f78acc00700 1 -- 172.21.15.63:0/1483002300 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00054e0 con 0x7f78b0075780 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.006+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b01a2970 con 0x7f78b0075780 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.006+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b010f010 con 0x7f78b0075780 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.012+0000 7f78c1bedb80 1 Processor -- start 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.013+0000 7f78c1bedb80 1 -- start start 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.013+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204ee1060 0x559204ee1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.013+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204edbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.013+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edbff0 0x559204ee7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.013+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559204e2dbb0 con 0x559204edb630 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.013+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559204dcb8f0 con 0x559204ee1060 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.013+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559204e29b70 con 0x559204edbff0 2024-05-11T13:46:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.014+0000 7f78b5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204ee1060 0x559204ee1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.014+0000 7f78b5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204ee1060 0x559204ee1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:35680/0 (socket says 172.21.15.63:35680) 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.014+0000 7f78b5200700 1 -- 172.21.15.63:0/165875562 learned_addr learned my addr 172.21.15.63:0/165875562 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.014+0000 7f78af400700 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edbff0 0x559204ee7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.015+0000 7f78acc00700 1 -- 172.21.15.63:0/1483002300 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f78a0004220 con 0x7f78b0075780 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.015+0000 7f78afe00700 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204edbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.015+0000 7f78b5200700 1 -- 172.21.15.63:0/165875562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edbff0 msgr2=0x559204ee7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.015+0000 7f78b5200700 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edbff0 0x559204ee7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.015+0000 7f78b5200700 1 -- 172.21.15.63:0/165875562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 msgr2=0x559204edbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.015+0000 7f78b5200700 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204edbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.015+0000 7f78b5200700 1 -- 172.21.15.63:0/165875562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559204d7aa60 con 0x559204ee1060 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.016+0000 7f78b5200700 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204ee1060 0x559204ee1440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78a80093a0 tx=0x7f78a80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e893e7d0eadd7f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.016+0000 7f78a5600700 1 -- 172.21.15.63:0/165875562 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a801b030 con 0x559204ee1060 2024-05-11T13:46:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.017+0000 7f78a5600700 1 -- 172.21.15.63:0/165875562 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78a8004020 con 0x559204ee1060 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.017+0000 7f78a5600700 1 -- 172.21.15.63:0/165875562 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a80176a0 con 0x559204ee1060 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.018+0000 7f78acc00700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f78940b2b20 0x7f78940b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.018+0000 7f78acc00700 1 -- 172.21.15.63:0/1483002300 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f78a00e2830 con 0x7f78b0075780 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.018+0000 7f78c1bedb80 1 -- 172.21.15.63:0/165875562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204ee1060 msgr2=0x559204ee1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.018+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204ee1060 0x559204ee1440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78a80093a0 tx=0x7f78a80099e0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78afe00700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f78940b2b20 0x7f78940b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78c1bedb80 1 -- 172.21.15.63:0/165875562 shutdown_connections 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edbff0 0x559204ee7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204ee1060 0x559204ee1440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/165875562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204edbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78c1bedb80 1 -- 172.21.15.63:0/165875562 >> 172.21.15.63:0/165875562 conn(0x559204e09cf0 msgr2=0x559204ed5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78c1bedb80 1 -- 172.21.15.63:0/165875562 shutdown_connections 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.019+0000 7f78c1bedb80 1 -- 172.21.15.63:0/165875562 wait complete. 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.020+0000 7f78afe00700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f78940b2b20 0x7f78940b4fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f78a00a5740 tx=0x7f78a0004f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.020+0000 7f78c1bedb80 1 -- start start 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.021+0000 7f78c1bedb80 1 Processor -- start 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edb630 0x559204ee7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204ee4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204ee1060 0x559204ee5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559204ee83c0 con 0x559204ee1060 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559204d93cb0 con 0x559204edbff0 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559204ee8540 con 0x559204edb630 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78c1bedb80 10 client.0 _ll_register_callbacks cb 0x559204dd4e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78af400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204ee4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78af400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204ee4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:35694/0 (socket says 172.21.15.63:35694) 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.023+0000 7f78af400700 1 -- 172.21.15.63:0/3535666527 learned_addr learned my addr 172.21.15.63:0/3535666527 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.024+0000 7f78af400700 1 -- 172.21.15.63:0/3535666527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edb630 msgr2=0x559204ee7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.024+0000 7f78af400700 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edb630 0x559204ee7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.024+0000 7f78af400700 1 -- 172.21.15.63:0/3535666527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204ee1060 msgr2=0x559204ee5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.024+0000 7f78af400700 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204ee1060 0x559204ee5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.024+0000 7f78af400700 1 -- 172.21.15.63:0/3535666527 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559204df35b0 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.024+0000 7f78af400700 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204ee4df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098f0 tx=0x7f789c00f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bad8b3c6e6ae517 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.024+0000 7f789a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.025+0000 7f789a000700 1 -- 172.21.15.63:0/3535666527 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c005040 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.025+0000 7f789a000700 1 -- 172.21.15.63:0/3535666527 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f789c0051e0 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.025+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a8004020 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.026+0000 7f789a000700 1 -- 172.21.15.63:0/3535666527 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f789c0053e0 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.026+0000 7f78c1bedb80 10 client.35540 fetch_fsmap learned FSMap version 67 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.026+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x559204ef29d0 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.026+0000 7f789a000700 1 -- 172.21.15.63:0/3535666527 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78a8004020 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.026+0000 7f789a000700 1 -- 172.21.15.63:0/3535666527 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) ==== 8362+0+0 (secure 0 0 0) 0x7f789c00b560 con 0x559204edbff0 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.026+0000 7f78c1bedb80 10 client.35540 fetch_fsmap finished waiting for FSMap version 67 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.026+0000 7f78c1bedb80 10 client.35540 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.027+0000 7f78c1bedb80 20 client.35540 populate_metadata read hostname 'smithi063' 2024-05-11T13:46:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.027+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204ef5060 0x559204ef74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.027+0000 7f78c1bedb80 4 client.35540 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.027+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559204ea8340 con 0x559204ef5060 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.028+0000 7f78b5200700 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204ef5060 0x559204ef74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.028+0000 7f78b5200700 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204ef5060 0x559204ef74b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46582d4df4ad514d server_cookie=39d6cf00a40617eb in_seq=0 out_seq=0 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.028+0000 7f789a000700 10 client.35540 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.029+0000 7f789a000700 1 -- 172.21.15.63:0/3535666527 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559204ea8340 con 0x559204ef5060 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.029+0000 7f789a000700 10 client.35540 handle_command_reply: tid=0 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.030+0000 7f78c1bedb80 1 client.35540 shutdown 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.030+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 msgr2=0x559204ee4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.030+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204ee4df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098f0 tx=0x7f789c00f510 comp rx=0 tx=0).stop 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 shutdown_connections 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204ee1060 0x559204ee5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204edb630 0x559204ee7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204ef5060 0x559204ef74b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/3535666527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204ee4df0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 >> 172.21.15.63:0/3535666527 conn(0x559204e09cf0 msgr2=0x559204ed09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f789a000700 0 client.35540 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 shutdown_connections 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.031+0000 7f78c1bedb80 1 -- 172.21.15.63:0/3535666527 wait complete. 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.032+0000 7f78c1bedb80 20 client.35540 trim_cache size 0 max 16384 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.072+0000 7f78c1bedb80 1 Processor -- start 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.072+0000 7f78c1bedb80 1 -- start start 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.073+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edb630 0x559204ef1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.073+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edbff0 0x559204edf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.073+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204efcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.073+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01be3f0 con 0x559204edbff0 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.073+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01bff30 con 0x559204edb630 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.073+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01beac0 con 0x559204ee1060 2024-05-11T13:46:37.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edb630 0x559204ef1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edb630 0x559204ef1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:35704/0 (socket says 172.21.15.63:35704) 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 -- 172.21.15.63:0/517039512 learned_addr learned my addr 172.21.15.63:0/517039512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78af400700 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edbff0 0x559204edf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204efcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 -- 172.21.15.63:0/517039512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 msgr2=0x559204efcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204efcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 -- 172.21.15.63:0/517039512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edbff0 msgr2=0x559204edf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edbff0 0x559204edf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.074+0000 7f78b5200700 1 -- 172.21.15.63:0/517039512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559204ed2b20 con 0x559204edb630 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.075+0000 7f78b5200700 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edb630 0x559204ef1410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x559204ea84c0 tx=0x7f78a8009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac094c467cd440c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.076+0000 7f78a4c00700 1 -- 172.21.15.63:0/517039512 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a80178b0 con 0x559204edb630 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.076+0000 7f78a4c00700 1 -- 172.21.15.63:0/517039512 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78a8027640 con 0x559204edb630 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.076+0000 7f78a4c00700 1 -- 172.21.15.63:0/517039512 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a80219e0 con 0x559204edb630 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.077+0000 7f78c1bedb80 1 -- 172.21.15.63:0/517039512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edb630 msgr2=0x559204ef1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.077+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edb630 0x559204ef1410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x559204ea84c0 tx=0x7f78a8009c60 comp rx=0 tx=0).stop 2024-05-11T13:46:37.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.078+0000 7f78c1bedb80 1 -- 172.21.15.63:0/517039512 shutdown_connections 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.078+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204efcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.078+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edb630 0x559204ef1410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.078+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/517039512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edbff0 0x559204edf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.078+0000 7f78c1bedb80 1 -- 172.21.15.63:0/517039512 >> 172.21.15.63:0/517039512 conn(0x559204e09cf0 msgr2=0x559204edea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.079+0000 7f78c1bedb80 1 -- 172.21.15.63:0/517039512 shutdown_connections 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.079+0000 7f78c1bedb80 1 -- 172.21.15.63:0/517039512 wait complete. 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.080+0000 7f78c1bedb80 1 -- start start 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.080+0000 7f78c1bedb80 1 Processor -- start 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.082+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204eec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.083+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204eeca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.083+0000 7f78b5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204eec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.084+0000 7f78b5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204eec510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44570/0 (socket says 172.21.15.63:44570) 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.084+0000 7f78c1bedb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204eecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.084+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01bf860 con 0x559204edb630 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.085+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01bf190 con 0x559204edbff0 2024-05-11T13:46:37.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.085+0000 7f78c1bedb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01bc3b0 con 0x559204ee1060 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.085+0000 7f78c1bedb80 10 client.0 _ll_register_callbacks cb 0x559204dd4e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.085+0000 7f78b5200700 1 -- 172.21.15.63:0/540897133 learned_addr learned my addr 172.21.15.63:0/540897133 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.085+0000 7f78b5200700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204eecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.085+0000 7f78af400700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204eeca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.086+0000 7f78af400700 1 -- 172.21.15.63:0/540897133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 msgr2=0x559204eecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.086+0000 7f78af400700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204eecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.086+0000 7f78af400700 1 -- 172.21.15.63:0/540897133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 msgr2=0x559204eec510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.087+0000 7f78af400700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204eec510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.087+0000 7f78af400700 1 -- 172.21.15.63:0/540897133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559204f5fbb0 con 0x559204edbff0 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.087+0000 7f78b5200700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204eec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.088+0000 7f78af400700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204eeca50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f789c000c00 tx=0x7f789c0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f287448553470b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.088+0000 7f789a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.088+0000 7f789a000700 1 -- 172.21.15.63:0/540897133 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c00bdb0 con 0x559204edbff0 2024-05-11T13:46:37.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.088+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a8027640 con 0x559204edbff0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.089+0000 7f789a000700 1 -- 172.21.15.63:0/540897133 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f789c034450 con 0x559204edbff0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.089+0000 7f789a000700 1 -- 172.21.15.63:0/540897133 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f789c03d850 con 0x559204edbff0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.089+0000 7f78c1bedb80 10 client.35546 fetch_fsmap learned FSMap version 67 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.089+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x559204f60e90 con 0x559204edbff0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.090+0000 7f789a000700 1 -- 172.21.15.63:0/540897133 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78a8027640 con 0x559204edbff0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.090+0000 7f789a000700 1 -- 172.21.15.63:0/540897133 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) ==== 8362+0+0 (secure 0 0 0) 0x7f789c034670 con 0x559204edbff0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.091+0000 7f78c1bedb80 10 client.35546 fetch_fsmap finished waiting for FSMap version 67 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.091+0000 7f78c1bedb80 10 client.35546 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.091+0000 7f78c1bedb80 20 client.35546 populate_metadata read hostname 'smithi063' 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.091+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204f61270 0x559204f63690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.092+0000 7f78c1bedb80 4 client.35546 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 87.6, "await_for": 42.0}] 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.092+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 87.6, "await_for": 42.0}) -- 0x559204ed6480 con 0x559204f61270 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.093+0000 7f78b5200700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204f61270 0x559204f63690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.094+0000 7f78b5200700 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204f61270 0x559204f63690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c77565e6bef0c1 server_cookie=404ceb287292ba1d in_seq=0 out_seq=0 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.095+0000 7f789a000700 10 client.35546 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.369+0000 7f789a000700 1 -- 172.21.15.63:0/540897133 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559204ed6480 con 0x559204f61270 2024-05-11T13:46:37.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.369+0000 7f789a000700 10 client.35546 handle_command_reply: tid=0 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.369+0000 7f78c1bedb80 1 client.35546 shutdown 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 msgr2=0x559204eeca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204eeca50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f789c000c00 tx=0x7f789c0057b0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 shutdown_connections 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559204ee1060 0x559204eecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559204edb630 0x559204eec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x559204f61270 0x559204f63690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 --2- 172.21.15.63:0/540897133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559204edbff0 0x559204eeca50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f789a000700 0 client.35546 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 >> 172.21.15.63:0/540897133 conn(0x559204e09cf0 msgr2=0x559204edf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 shutdown_connections 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 1 -- 172.21.15.63:0/540897133 wait complete. 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.370+0000 7f78c1bedb80 20 client.35546 trim_cache size 0 max 16384 2024-05-11T13:46:37.584 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 67, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 38, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "98517938": { 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 38, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 87.6, 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:46:37.585 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:46:37.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.371+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f78940b2b20 msgr2=0x7f78940b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.371+0000 7f78b7200700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f78940b2b20 0x7f78940b4fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f78a00a5740 tx=0x7f78a0004f90 comp rx=0 tx=0).stop 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.371+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b0075780 msgr2=0x7f78b0108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.371+0000 7f78b7200700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b0075780 0x7f78b0108f40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d980 tx=0x7f78a000dc90 comp rx=0 tx=0).stop 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 shutdown_connections 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f78940b2b20 0x7f78940b4fe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b010ff80 0x7f78b01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78b0075780 0x7f78b0108f40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 --2- 172.21.15.63:0/1483002300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b00760a0 0x7f78b01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 >> 172.21.15.63:0/1483002300 conn(0x7f78b007a700 msgr2=0x7f78b0103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 shutdown_connections 2024-05-11T13:46:37.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:46:37.372+0000 7f78b7200700 1 -- 172.21.15.63:0/1483002300 wait complete. 2024-05-11T13:46:37.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:37 smithi063 ceph-mon[29809]: pgmap v399: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-05-11T13:46:37.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:37 smithi071 ceph-mon[33293]: pgmap v399: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-05-11T13:46:37.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:37 smithi165 ceph-mon[33235]: pgmap v399: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-05-11T13:46:38.495 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '98517938', quiesce took 9.2 seconds. Will release after: 27.6 2024-05-11T13:46:39.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:39 smithi063 ceph-mon[29809]: pgmap v400: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.5 MiB/s wr, 346 op/s 2024-05-11T13:46:39.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:39 smithi071 ceph-mon[33293]: pgmap v400: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.5 MiB/s wr, 346 op/s 2024-05-11T13:46:39.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:39 smithi165 ceph-mon[33235]: pgmap v400: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.5 MiB/s wr, 346 op/s 2024-05-11T13:46:41.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 315.0 s, 93.7 tps, lat 250.122 ms stddev 42.496 2024-05-11T13:46:41.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:41 smithi063 ceph-mon[29809]: pgmap v401: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-05-11T13:46:41.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:41 smithi071 ceph-mon[33293]: pgmap v401: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-05-11T13:46:41.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:41 smithi165 ceph-mon[33235]: pgmap v401: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-05-11T13:46:43.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:43 smithi063 ceph-mon[29809]: pgmap v402: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-05-11T13:46:43.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:43 smithi071 ceph-mon[33293]: pgmap v402: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-05-11T13:46:43.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:43 smithi165 ceph-mon[33235]: pgmap v402: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-05-11T13:46:45.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:45 smithi063 ceph-mon[29809]: pgmap v403: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 5.5 MiB/s wr, 340 op/s 2024-05-11T13:46:45.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:45 smithi071 ceph-mon[33293]: pgmap v403: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 5.5 MiB/s wr, 340 op/s 2024-05-11T13:46:45.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:45 smithi165 ceph-mon[33235]: pgmap v403: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 5.5 MiB/s wr, 340 op/s 2024-05-11T13:46:46.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 320.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:47.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:47 smithi063 ceph-mon[29809]: pgmap v404: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-05-11T13:46:47.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:47 smithi071 ceph-mon[33293]: pgmap v404: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-05-11T13:46:47.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:47 smithi165 ceph-mon[33235]: pgmap v404: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-05-11T13:46:49.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:49 smithi063 ceph-mon[29809]: pgmap v405: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.8 MiB/s wr, 188 op/s 2024-05-11T13:46:49.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:49 smithi071 ceph-mon[33293]: pgmap v405: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.8 MiB/s wr, 188 op/s 2024-05-11T13:46:49.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:49 smithi165 ceph-mon[33235]: pgmap v405: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.8 MiB/s wr, 188 op/s 2024-05-11T13:46:51.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 325.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:51.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:51 smithi063 ceph-mon[29809]: pgmap v406: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 157 KiB/s wr, 26 op/s 2024-05-11T13:46:51.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:51 smithi071 ceph-mon[33293]: pgmap v406: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 157 KiB/s wr, 26 op/s 2024-05-11T13:46:51.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:51 smithi165 ceph-mon[33235]: pgmap v406: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 157 KiB/s wr, 26 op/s 2024-05-11T13:46:53.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:53 smithi063 ceph-mon[29809]: pgmap v407: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 157 KiB/s wr, 26 op/s 2024-05-11T13:46:53.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:53 smithi071 ceph-mon[33293]: pgmap v407: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 157 KiB/s wr, 26 op/s 2024-05-11T13:46:53.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:53 smithi165 ceph-mon[33235]: pgmap v407: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 157 KiB/s wr, 26 op/s 2024-05-11T13:46:55.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:55 smithi063 ceph-mon[29809]: pgmap v408: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 692 KiB/s wr, 91 op/s 2024-05-11T13:46:55.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:55 smithi071 ceph-mon[33293]: pgmap v408: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 692 KiB/s wr, 91 op/s 2024-05-11T13:46:55.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:55 smithi165 ceph-mon[33235]: pgmap v408: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 692 KiB/s wr, 91 op/s 2024-05-11T13:46:56.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 330.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:46:57.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:57 smithi063 ceph-mon[29809]: pgmap v409: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 624 KiB/s wr, 78 op/s 2024-05-11T13:46:57.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:57 smithi071 ceph-mon[33293]: pgmap v409: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 624 KiB/s wr, 78 op/s 2024-05-11T13:46:57.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:57 smithi165 ceph-mon[33235]: pgmap v409: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 624 KiB/s wr, 78 op/s 2024-05-11T13:46:59.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:46:59 smithi063 ceph-mon[29809]: pgmap v410: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 629 KiB/s wr, 84 op/s 2024-05-11T13:46:59.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:46:59 smithi071 ceph-mon[33293]: pgmap v410: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 629 KiB/s wr, 84 op/s 2024-05-11T13:46:59.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:46:59 smithi165 ceph-mon[33235]: pgmap v410: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 629 KiB/s wr, 84 op/s 2024-05-11T13:47:01.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 335.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:47:01.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:01 smithi063 ceph-mon[29809]: pgmap v411: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 541 KiB/s wr, 70 op/s 2024-05-11T13:47:01.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:01 smithi071 ceph-mon[33293]: pgmap v411: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 541 KiB/s wr, 70 op/s 2024-05-11T13:47:01.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:01 smithi165 ceph-mon[33235]: pgmap v411: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 541 KiB/s wr, 70 op/s 2024-05-11T13:47:03.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:03 smithi063 ceph-mon[29809]: pgmap v412: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 541 KiB/s wr, 70 op/s 2024-05-11T13:47:03.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:03 smithi071 ceph-mon[33293]: pgmap v412: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 541 KiB/s wr, 70 op/s 2024-05-11T13:47:03.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:03 smithi165 ceph-mon[33235]: pgmap v412: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 541 KiB/s wr, 70 op/s 2024-05-11T13:47:05.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:05 smithi063 ceph-mon[29809]: pgmap v413: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 591 KiB/s wr, 92 op/s 2024-05-11T13:47:05.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:05 smithi071 ceph-mon[33293]: pgmap v413: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 591 KiB/s wr, 92 op/s 2024-05-11T13:47:05.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:05 smithi165 ceph-mon[33235]: pgmap v413: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 591 KiB/s wr, 92 op/s 2024-05-11T13:47:06.097 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:47:06.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 340.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:47:06.566 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:47:07.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:07 smithi071 ceph-mon[33293]: pgmap v414: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-05-11T13:47:07.759 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:07 smithi063 ceph-mon[29809]: pgmap v414: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-05-11T13:47:07.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:07 smithi165 ceph-mon[33235]: pgmap v414: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-05-11T13:47:09.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.015+0000 7f5e50a00700 1 -- 172.21.15.63:0/2345623935 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40003b40 con 0x7f5e54105660 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.015+0000 7f5e53200700 1 -- 172.21.15.63:0/2345623935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e54105660 msgr2=0x7f5e54105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.015+0000 7f5e53200700 1 --2- 172.21.15.63:0/2345623935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e54105660 0x7f5e54105ae0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009990 tx=0x7f5e40009ca0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.016+0000 7f5e53200700 1 -- 172.21.15.63:0/2345623935 shutdown_connections 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.016+0000 7f5e53200700 1 --2- 172.21.15.63:0/2345623935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e54106020 0x7f5e54112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.016+0000 7f5e53200700 1 --2- 172.21.15.63:0/2345623935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5410b660 0x7f5e5410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.016+0000 7f5e53200700 1 --2- 172.21.15.63:0/2345623935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e54105660 0x7f5e54105ae0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.016+0000 7f5e53200700 1 -- 172.21.15.63:0/2345623935 >> 172.21.15.63:0/2345623935 conn(0x7f5e54101130 msgr2=0x7f5e54103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.016+0000 7f5e53200700 1 -- 172.21.15.63:0/2345623935 shutdown_connections 2024-05-11T13:47:09.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.016+0000 7f5e53200700 1 -- 172.21.15.63:0/2345623935 wait complete. 2024-05-11T13:47:09.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.017+0000 7f5e53200700 1 Processor -- start 2024-05-11T13:47:09.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.017+0000 7f5e53200700 1 -- start start 2024-05-11T13:47:09.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e53200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e54105660 0x7f5e541a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:09.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e53200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e54106020 0x7f5e541a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:09.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e53200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5410b660 0x7f5e5419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:09.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e53200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54115770 con 0x7f5e5410b660 2024-05-11T13:47:09.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e53200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541155f0 con 0x7f5e54105660 2024-05-11T13:47:09.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e53200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541158f0 con 0x7f5e54106020 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e52800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5410b660 0x7f5e5419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e52800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5410b660 0x7f5e5419bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50464/0 (socket says 172.21.15.63:50464) 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e52800700 1 -- 172.21.15.63:0/1236053301 learned_addr learned my addr 172.21.15.63:0/1236053301 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.018+0000 7f5e51e00700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e54105660 0x7f5e541a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e51400700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e54106020 0x7f5e541a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e52800700 1 -- 172.21.15.63:0/1236053301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e54106020 msgr2=0x7f5e541a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e52800700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e54106020 0x7f5e541a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e52800700 1 -- 172.21.15.63:0/1236053301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e54105660 msgr2=0x7f5e541a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:09.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e52800700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e54105660 0x7f5e541a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e52800700 1 -- 172.21.15.63:0/1236053301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5419c210 con 0x7f5e5410b660 2024-05-11T13:47:09.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e51e00700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e54105660 0x7f5e541a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:47:09.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e52800700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5410b660 0x7f5e5419bab0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c007ee0 tx=0x7f5e4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:09.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e4aa00700 1 -- 172.21.15.63:0/1236053301 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00dbd0 con 0x7f5e5410b660 2024-05-11T13:47:09.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.019+0000 7f5e4aa00700 1 -- 172.21.15.63:0/1236053301 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e4c00dd70 con 0x7f5e5410b660 2024-05-11T13:47:09.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.020+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5419c500 con 0x7f5e5410b660 2024-05-11T13:47:09.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.020+0000 7f5e4aa00700 1 -- 172.21.15.63:0/1236053301 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0058d0 con 0x7f5e5410b660 2024-05-11T13:47:09.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.020+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e541a87c0 con 0x7f5e5410b660 2024-05-11T13:47:09.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.021+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5410dbf0 con 0x7f5e5410b660 2024-05-11T13:47:09.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.023+0000 7f5e4aa00700 1 -- 172.21.15.63:0/1236053301 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5e4c01a020 con 0x7f5e5410b660 2024-05-11T13:47:09.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.025+0000 7f5e4aa00700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5e380b29f0 0x7f5e380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:09.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.025+0000 7f5e4aa00700 1 -- 172.21.15.63:0/1236053301 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f5e4c0e22d0 con 0x7f5e5410b660 2024-05-11T13:47:09.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.025+0000 7f5e51e00700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5e380b29f0 0x7f5e380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:09.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.026+0000 7f5e51e00700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5e380b29f0 0x7f5e380b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005e10 tx=0x7f5e44005da0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:09.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.030+0000 7f5e4aa00700 1 -- 172.21.15.63:0/1236053301 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e4c010070 con 0x7f5e5410b660 2024-05-11T13:47:09.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.323+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e5419cfe0 con 0x7f5e5410b660 2024-05-11T13:47:09.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.325+0000 7f5e4aa00700 1 -- 172.21.15.63:0/1236053301 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11935 (secure 0 0 0) 0x7f5e4c0a9860 con 0x7f5e5410b660 2024-05-11T13:47:09.316 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:47:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:45:56.474021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34451,34472,34427,34403,24472]},"id":1}]} 2024-05-11T13:47:09.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5e380b29f0 msgr2=0x7f5e380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:09.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5e380b29f0 0x7f5e380b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005e10 tx=0x7f5e44005da0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5410b660 msgr2=0x7f5e5419bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:09.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5410b660 0x7f5e5419bab0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c007ee0 tx=0x7f5e4c00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 shutdown_connections 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5e380b29f0 0x7f5e380b4eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5410b660 0x7f5e5419bab0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e54105660 0x7f5e541a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 --2- 172.21.15.63:0/1236053301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e54106020 0x7f5e541a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 >> 172.21.15.63:0/1236053301 conn(0x7f5e54101130 msgr2=0x7f5e54101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 shutdown_connections 2024-05-11T13:47:09.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:09.328+0000 7f5e53200700 1 -- 172.21.15.63:0/1236053301 wait complete. 2024-05-11T13:47:09.321 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 67 2024-05-11T13:47:09.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:09 smithi063 ceph-mon[29809]: pgmap v415: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 32 op/s 2024-05-11T13:47:09.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:09 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1236053301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:47:09.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:09 smithi165 ceph-mon[33235]: pgmap v415: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 32 op/s 2024-05-11T13:47:09.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:09 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1236053301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:47:09.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:09 smithi071 ceph-mon[33293]: pgmap v415: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 32 op/s 2024-05-11T13:47:09.944 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:09 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1236053301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:47:10.481 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:10 smithi063 ceph-mon[29809]: osdmap e106: 12 total, 12 up, 12 in 2024-05-11T13:47:10.544 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 98517938 --release --await' 2024-05-11T13:47:10.544 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 98517938 --release --await 2024-05-11T13:47:10.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:10 smithi165 ceph-mon[33235]: osdmap e106: 12 total, 12 up, 12 in 2024-05-11T13:47:10.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:10 smithi071 ceph-mon[33293]: osdmap e106: 12 total, 12 up, 12 in 2024-05-11T13:47:11.029 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:47:11.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 345.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:47:11.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:11 smithi165 ceph-mon[33235]: pgmap v417: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-05-11T13:47:11.924 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:11 smithi063 ceph-mon[29809]: pgmap v417: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-05-11T13:47:11.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:11 smithi071 ceph-mon[33293]: pgmap v417: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 -- 172.21.15.63:0/4132150544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f242406bbf0 msgr2=0x7f24241103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 --2- 172.21.15.63:0/4132150544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f242406bbf0 0x7f24241103e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f241c00b0a0 tx=0x7f241c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 -- 172.21.15.63:0/4132150544 shutdown_connections 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 --2- 172.21.15.63:0/4132150544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f242406bbf0 0x7f24241103e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 --2- 172.21.15.63:0/4132150544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24241095c0 0x7f24241099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 --2- 172.21.15.63:0/4132150544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242406b230 0x7f242406b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 -- 172.21.15.63:0/4132150544 >> 172.21.15.63:0/4132150544 conn(0x7f2424079400 msgr2=0x7f2424079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 -- 172.21.15.63:0/4132150544 shutdown_connections 2024-05-11T13:47:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.357+0000 7f2429400700 1 -- 172.21.15.63:0/4132150544 wait complete. 2024-05-11T13:47:13.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.358+0000 7f2429400700 1 Processor -- start 2024-05-11T13:47:13.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.358+0000 7f2429400700 1 -- start start 2024-05-11T13:47:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2429400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242406b230 0x7f242410a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2429400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f242406bbf0 0x7f242410a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2429400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24241095c0 0x7f242410ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2429400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24241135e0 con 0x7f24241095c0 2024-05-11T13:47:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2429400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2424113460 con 0x7f242406b230 2024-05-11T13:47:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2429400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2424113760 con 0x7f242406bbf0 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2423200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24241095c0 0x7f242410ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2423200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24241095c0 0x7f242410ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54276/0 (socket says 172.21.15.63:54276) 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.359+0000 7f2423200700 1 -- 172.21.15.63:0/1912733320 learned_addr learned my addr 172.21.15.63:0/1912733320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2422800700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242406b230 0x7f242410a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2421e00700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f242406bbf0 0x7f242410a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2423200700 1 -- 172.21.15.63:0/1912733320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f242406bbf0 msgr2=0x7f242410a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2423200700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f242406bbf0 0x7f242410a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2423200700 1 -- 172.21.15.63:0/1912733320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242406b230 msgr2=0x7f242410a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2423200700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242406b230 0x7f242410a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2423200700 1 -- 172.21.15.63:0/1912733320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24241ac400 con 0x7f24241095c0 2024-05-11T13:47:13.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.360+0000 7f2423200700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24241095c0 0x7f242410ae50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f241c00b070 tx=0x7f241c004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.361+0000 7f241b400700 1 -- 172.21.15.63:0/1912733320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c021070 con 0x7f24241095c0 2024-05-11T13:47:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.361+0000 7f241b400700 1 -- 172.21.15.63:0/1912733320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f241c007c00 con 0x7f24241095c0 2024-05-11T13:47:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.361+0000 7f241b400700 1 -- 172.21.15.63:0/1912733320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c004770 con 0x7f24241095c0 2024-05-11T13:47:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.361+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24241ac690 con 0x7f24241095c0 2024-05-11T13:47:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.361+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241aca90 con 0x7f24241095c0 2024-05-11T13:47:13.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.365+0000 7f241b400700 1 -- 172.21.15.63:0/1912733320 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f241c003a00 con 0x7f24241095c0 2024-05-11T13:47:13.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.366+0000 7f2433df7b80 1 Processor -- start 2024-05-11T13:47:13.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.366+0000 7f2433df7b80 1 -- start start 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b00afd0 0x562c4b00b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f241b400700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f24080b2a70 0x7f24080b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b005a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b005f60 0x562c4b011d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2433df7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c4af57b20 con 0x562c4b00afd0 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2433df7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562c4aef5860 con 0x562c4b0055a0 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2433df7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c4af53ae0 con 0x562c4b005f60 2024-05-11T13:47:13.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f241b400700 1 -- 172.21.15.63:0/1912733320 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f241c00f020 con 0x7f24241095c0 2024-05-11T13:47:13.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2421e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b00afd0 0x562c4b00b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2421e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b00afd0 0x562c4b00b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54288/0 (socket says 172.21.15.63:54288) 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2421e00700 1 -- 172.21.15.63:0/1454028511 learned_addr learned my addr 172.21.15.63:0/1454028511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.367+0000 7f2422800700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f24080b2a70 0x7f24080b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2421e00700 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2423200700 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b005f60 0x562c4b011d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2421e00700 1 -- 172.21.15.63:0/1454028511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b005f60 msgr2=0x562c4b011d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2421e00700 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b005f60 0x562c4b011d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2421e00700 1 -- 172.21.15.63:0/1454028511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 msgr2=0x562c4b005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2421e00700 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b005a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2421e00700 1 -- 172.21.15.63:0/1454028511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c4aea49d0 con 0x562c4b00afd0 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2421e00700 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b00afd0 0x562c4b00b3b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f241000bfd0 tx=0x7f241000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8ee5af1eda56fa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.368+0000 7f2422800700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f24080b2a70 0x7f24080b4f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2414005fd0 tx=0x7f2414005db0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:13.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.369+0000 7f240fe00700 1 -- 172.21.15.63:0/1454028511 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410019020 con 0x562c4b00afd0 2024-05-11T13:47:13.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.369+0000 7f240fe00700 1 -- 172.21.15.63:0/1454028511 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2410010af0 con 0x562c4b00afd0 2024-05-11T13:47:13.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.369+0000 7f240fe00700 1 -- 172.21.15.63:0/1454028511 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241000f7b0 con 0x562c4b00afd0 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.369+0000 7f2433df7b80 1 -- 172.21.15.63:0/1454028511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b00afd0 msgr2=0x562c4b00b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.369+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b00afd0 0x562c4b00b3b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f241000bfd0 tx=0x7f241000b710 comp rx=0 tx=0).stop 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 -- 172.21.15.63:0/1454028511 shutdown_connections 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b005f60 0x562c4b011d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b00afd0 0x562c4b00b3b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1454028511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b005a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 -- 172.21.15.63:0/1454028511 >> 172.21.15.63:0/1454028511 conn(0x562c4af33c60 msgr2=0x562c4afffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 -- 172.21.15.63:0/1454028511 shutdown_connections 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 -- 172.21.15.63:0/1454028511 wait complete. 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.370+0000 7f2433df7b80 1 -- start start 2024-05-11T13:47:13.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 1 Processor -- start 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b0119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b00ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b00f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c4b012330 con 0x562c4b0055a0 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562c4aebdc20 con 0x562c4b005f60 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c4b0124b0 con 0x562c4b00afd0 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.371+0000 7f2433df7b80 10 client.0 _ll_register_callbacks cb 0x562c4aefed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b0119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b0119d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54302/0 (socket says 172.21.15.63:54302) 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 -- 172.21.15.63:0/2626932720 learned_addr learned my addr 172.21.15.63:0/2626932720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2421e00700 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b00ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2423200700 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b00f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 -- 172.21.15.63:0/2626932720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 msgr2=0x562c4b00f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b00f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 -- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 msgr2=0x562c4b00ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b00ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 -- 172.21.15.63:0/2626932720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4af1d520 con 0x562c4b0055a0 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.372+0000 7f2422800700 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b0119d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f24140106b0 tx=0x7f2414004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e71ce4589b15ec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.373+0000 7f240cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.373+0000 7f240cc00700 1 -- 172.21.15.63:0/2626932720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414021070 con 0x562c4b0055a0 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.373+0000 7f240cc00700 1 -- 172.21.15.63:0/2626932720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f241401d860 con 0x562c4b0055a0 2024-05-11T13:47:13.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.373+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2410010af0 con 0x562c4b0055a0 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.373+0000 7f240cc00700 1 -- 172.21.15.63:0/2626932720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f241402d050 con 0x562c4b0055a0 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.373+0000 7f2433df7b80 10 client.15972 fetch_fsmap learned FSMap version 67 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.373+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x562c4b01cc20 con 0x562c4b0055a0 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f240cc00700 1 -- 172.21.15.63:0/2626932720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2410010af0 con 0x562c4b0055a0 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f240cc00700 1 -- 172.21.15.63:0/2626932720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 67) ==== 8362+0+0 (secure 0 0 0) 0x7f241400fab0 con 0x562c4b0055a0 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f2433df7b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 67 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f2433df7b80 10 client.15972 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f2433df7b80 20 client.15972 populate_metadata read hostname 'smithi063' 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f2433df7b80 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b01f1b0 0x562c4b0215e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f2433df7b80 4 client.15972 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:47:13.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.374+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c4afd22b0 con 0x562c4b01f1b0 2024-05-11T13:47:13.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.375+0000 7f2421e00700 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b01f1b0 0x562c4b0215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.376+0000 7f240cc00700 10 client.15972 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:47:13.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.376+0000 7f2421e00700 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b01f1b0 0x562c4b0215e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f6a86c137e4519 server_cookie=707647e088fa63ff in_seq=0 out_seq=0 2024-05-11T13:47:13.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.377+0000 7f240cc00700 1 -- 172.21.15.63:0/2626932720 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c4afd22b0 con 0x562c4b01f1b0 2024-05-11T13:47:13.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.377+0000 7f240cc00700 10 client.15972 handle_command_reply: tid=0 2024-05-11T13:47:13.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.377+0000 7f2433df7b80 1 client.15972 shutdown 2024-05-11T13:47:13.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 msgr2=0x562c4b0119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b0119d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f24140106b0 tx=0x7f2414004750 comp rx=0 tx=0).stop 2024-05-11T13:47:13.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 shutdown_connections 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b00f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b0119d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b01f1b0 0x562c4b0215e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 --2- 172.21.15.63:0/2626932720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b00ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 >> 172.21.15.63:0/2626932720 conn(0x562c4af33c60 msgr2=0x562c4affa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f240cc00700 0 client.15972 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 shutdown_connections 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 1 -- 172.21.15.63:0/2626932720 wait complete. 2024-05-11T13:47:13.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.378+0000 7f2433df7b80 20 client.15972 trim_cache size 0 max 16384 2024-05-11T13:47:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.401+0000 7f2433df7b80 1 Processor -- start 2024-05-11T13:47:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.401+0000 7f2433df7b80 1 -- start start 2024-05-11T13:47:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.401+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b01b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b026eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b0273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2433df7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24241b5bd0 con 0x562c4b0055a0 2024-05-11T13:47:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2433df7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24241b7710 con 0x562c4b005f60 2024-05-11T13:47:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2433df7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24241b62a0 con 0x562c4b00afd0 2024-05-11T13:47:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b01b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b01b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54310/0 (socket says 172.21.15.63:54310) 2024-05-11T13:47:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 -- 172.21.15.63:0/553736655 learned_addr learned my addr 172.21.15.63:0/553736655 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:47:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2421e00700 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b026eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 -- 172.21.15.63:0/553736655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 msgr2=0x562c4b0273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b0273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 -- 172.21.15.63:0/553736655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 msgr2=0x562c4b026eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b026eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2422800700 1 -- 172.21.15.63:0/553736655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c4b021c60 con 0x562c4b0055a0 2024-05-11T13:47:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.402+0000 7f2421e00700 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b026eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:47:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.403+0000 7f2422800700 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b01b3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2414010f90 tx=0x7f2414005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2681944570551763 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.403+0000 7f240f400700 1 -- 172.21.15.63:0/553736655 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414035030 con 0x562c4b0055a0 2024-05-11T13:47:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.403+0000 7f240f400700 1 -- 172.21.15.63:0/553736655 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2414025d30 con 0x562c4b0055a0 2024-05-11T13:47:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.403+0000 7f240f400700 1 -- 172.21.15.63:0/553736655 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414021050 con 0x562c4b0055a0 2024-05-11T13:47:13.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 -- 172.21.15.63:0/553736655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 msgr2=0x562c4b01b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b01b3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2414010f90 tx=0x7f2414005ae0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 -- 172.21.15.63:0/553736655 shutdown_connections 2024-05-11T13:47:13.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b0273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b0055a0 0x562c4b01b3c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 --2- 172.21.15.63:0/553736655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b005f60 0x562c4b026eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 -- 172.21.15.63:0/553736655 >> 172.21.15.63:0/553736655 conn(0x562c4af33c60 msgr2=0x562c4b004e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 -- 172.21.15.63:0/553736655 shutdown_connections 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.404+0000 7f2433df7b80 1 -- 172.21.15.63:0/553736655 wait complete. 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.405+0000 7f2433df7b80 1 -- start start 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.405+0000 7f2433df7b80 1 Processor -- start 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.405+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b014710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.405+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b005f60 0x562c4b014c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2433df7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b015190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2433df7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24241b7040 con 0x562c4b005f60 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2433df7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24241b6970 con 0x562c4b0055a0 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2433df7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24241b3b90 con 0x562c4b00afd0 2024-05-11T13:47:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2433df7b80 10 client.0 _ll_register_callbacks cb 0x562c4aefed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b005f60 0x562c4b014c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b005f60 0x562c4b014c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54324/0 (socket says 172.21.15.63:54324) 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 -- 172.21.15.63:0/1029641363 learned_addr learned my addr 172.21.15.63:0/1029641363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2422800700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b014710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 -- 172.21.15.63:0/1029641363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 msgr2=0x562c4b015190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2423200700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b015190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b015190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 -- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 msgr2=0x562c4b014710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b014710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2421e00700 1 -- 172.21.15.63:0/1029641363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4b089b40 con 0x562c4b005f60 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.406+0000 7f2422800700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b014710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f2421e00700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b005f60 0x562c4b014c50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f241000bfd0 tx=0x7f2410013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c13f28cd31b903 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:47:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f240cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f240cc00700 1 -- 172.21.15.63:0/1029641363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410030060 con 0x562c4b005f60 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f240cc00700 1 -- 172.21.15.63:0/1029641363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f2410010a10 con 0x562c4b005f60 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2414025d30 con 0x562c4b005f60 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f240cc00700 1 -- 172.21.15.63:0/1029641363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2410037050 con 0x562c4b005f60 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f2433df7b80 10 client.15978 fetch_fsmap learned FSMap version 67 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.407+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x562c4b08adb0 con 0x562c4b005f60 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f240cc00700 1 -- 172.21.15.63:0/1029641363 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2414025d30 con 0x562c4b005f60 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f240cc00700 1 -- 172.21.15.63:0/1029641363 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 67) ==== 8362+0+0 (secure 0 0 0) 0x7f2410027070 con 0x562c4b005f60 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f2433df7b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 67 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f2433df7b80 10 client.15978 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f2433df7b80 20 client.15978 populate_metadata read hostname 'smithi063' 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b08b210 0x562c4b08d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f2433df7b80 4 client.15978 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "98517938", "release": true, "await": true}] 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.408+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "98517938", "release": true, "await": true}) -- 0x562c4af65500 con 0x562c4b08b210 2024-05-11T13:47:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.409+0000 7f2422800700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b08b210 0x562c4b08d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:47:13.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.410+0000 7f2422800700 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b08b210 0x562c4b08d630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecdfce0cff29d4a server_cookie=24ce8a4d547a0a4f in_seq=0 out_seq=0 2024-05-11T13:47:13.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.410+0000 7f240cc00700 10 client.15978 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:47:13.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f240cc00700 1 -- 172.21.15.63:0/1029641363 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562c4af65500 con 0x562c4b08b210 2024-05-11T13:47:13.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f240cc00700 10 client.15978 handle_command_reply: tid=0 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f2433df7b80 1 client.15978 shutdown 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b005f60 msgr2=0x562c4b014c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b005f60 0x562c4b014c50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f241000bfd0 tx=0x7f2410013f40 comp rx=0 tx=0).stop 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 shutdown_connections 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c4b00afd0 0x562c4b015190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.467+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562c4b08b210 0x562c4b08d630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.468+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c4b0055a0 0x562c4b014710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.468+0000 7f2433df7b80 1 --2- 172.21.15.63:0/1029641363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c4b005f60 0x562c4b014c50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.468+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 >> 172.21.15.63:0/1029641363 conn(0x562c4af33c60 msgr2=0x562c4b008a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:13.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.468+0000 7f240cc00700 0 client.15978 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:47:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.468+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 shutdown_connections 2024-05-11T13:47:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.468+0000 7f2433df7b80 1 -- 172.21.15.63:0/1029641363 wait complete. 2024-05-11T13:47:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.468+0000 7f2433df7b80 20 client.15978 trim_cache size 0 max 16384 2024-05-11T13:47:13.460 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:47:13.460 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 67, 2024-05-11T13:47:13.460 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:47:13.460 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 40, 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "98517938": { 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 40, 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 87.6, 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:47:13.461 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:47:13.462 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:47:13.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:47:13.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:47:13.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:47:13.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:47:13.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:47:13.462 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.469+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f24080b2a70 msgr2=0x7f24080b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.469+0000 7f2429400700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f24080b2a70 0x7f24080b4f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2414005fd0 tx=0x7f2414005db0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.469+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24241095c0 msgr2=0x7f242410ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.469+0000 7f2429400700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24241095c0 0x7f242410ae50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f241c00b070 tx=0x7f241c004e00 comp rx=0 tx=0).stop 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.471+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 shutdown_connections 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.471+0000 7f2429400700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f24080b2a70 0x7f24080b4f30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.471+0000 7f2429400700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24241095c0 0x7f242410ae50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.471+0000 7f2429400700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242406b230 0x7f242410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.471+0000 7f2429400700 1 --2- 172.21.15.63:0/1912733320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f242406bbf0 0x7f242410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:47:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.471+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 >> 172.21.15.63:0/1912733320 conn(0x7f2424079400 msgr2=0x7f242410e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:47:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.471+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 shutdown_connections 2024-05-11T13:47:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:47:13.472+0000 7f2429400700 1 -- 172.21.15.63:0/1912733320 wait complete. 2024-05-11T13:47:13.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:13 smithi165 ceph-mon[33235]: pgmap v418: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-05-11T13:47:13.838 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:13 smithi063 ceph-mon[29809]: pgmap v418: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-05-11T13:47:13.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:13 smithi071 ceph-mon[33293]: pgmap v418: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-05-11T13:47:14.406 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '98517938', seconds elapsed: 45.1 2024-05-11T13:47:16.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 350.0 s, 209.8 tps, lat 3698.238 ms stddev 10585.379 2024-05-11T13:47:16.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:16 smithi063 ceph-mon[29809]: pgmap v419: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-05-11T13:47:16.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:16 smithi063 ceph-mon[29809]: osdmap e107: 12 total, 12 up, 12 in 2024-05-11T13:47:16.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:16 smithi071 ceph-mon[33293]: pgmap v419: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-05-11T13:47:16.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:16 smithi071 ceph-mon[33293]: osdmap e107: 12 total, 12 up, 12 in 2024-05-11T13:47:16.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:16 smithi165 ceph-mon[33235]: pgmap v419: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-05-11T13:47:16.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:16 smithi165 ceph-mon[33235]: osdmap e107: 12 total, 12 up, 12 in 2024-05-11T13:47:18.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:18 smithi063 ceph-mon[29809]: pgmap v421: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-05-11T13:47:18.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:18 smithi071 ceph-mon[33293]: pgmap v421: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-05-11T13:47:18.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:18 smithi165 ceph-mon[33235]: pgmap v421: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-05-11T13:47:20.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:20 smithi063 ceph-mon[29809]: pgmap v422: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2024-05-11T13:47:20.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:20 smithi071 ceph-mon[33293]: pgmap v422: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2024-05-11T13:47:20.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:20 smithi165 ceph-mon[33235]: pgmap v422: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2024-05-11T13:47:21.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 355.0 s, 457.0 tps, lat 222.109 ms stddev 45.785 2024-05-11T13:47:22.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:22 smithi063 ceph-mon[29809]: pgmap v423: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 258 op/s 2024-05-11T13:47:22.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:22 smithi071 ceph-mon[33293]: pgmap v423: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 258 op/s 2024-05-11T13:47:22.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:22 smithi165 ceph-mon[33235]: pgmap v423: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 258 op/s 2024-05-11T13:47:24.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:24 smithi071 ceph-mon[33293]: pgmap v424: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 258 op/s 2024-05-11T13:47:24.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:24 smithi165 ceph-mon[33235]: pgmap v424: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 258 op/s 2024-05-11T13:47:24.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:24 smithi063 ceph-mon[29809]: pgmap v424: 129 pgs: 129 active+clean; 2.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 258 op/s 2024-05-11T13:47:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 360.0 s, 427.2 tps, lat 234.661 ms stddev 76.231 2024-05-11T13:47:26.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:26 smithi071 ceph-mon[33293]: pgmap v425: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 7.8 MiB/s wr, 540 op/s 2024-05-11T13:47:26.509 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:26 smithi063 ceph-mon[29809]: pgmap v425: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 7.8 MiB/s wr, 540 op/s 2024-05-11T13:47:26.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:26 smithi165 ceph-mon[33235]: pgmap v425: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 7.8 MiB/s wr, 540 op/s 2024-05-11T13:47:28.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:28 smithi071 ceph-mon[33293]: pgmap v426: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 7.0 MiB/s wr, 482 op/s 2024-05-11T13:47:28.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:28 smithi165 ceph-mon[33235]: pgmap v426: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 7.0 MiB/s wr, 482 op/s 2024-05-11T13:47:28.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:28 smithi063 ceph-mon[29809]: pgmap v426: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 7.0 MiB/s wr, 482 op/s 2024-05-11T13:47:30.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:30 smithi071 ceph-mon[33293]: pgmap v427: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 9.8 MiB/s wr, 690 op/s 2024-05-11T13:47:30.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:30 smithi165 ceph-mon[33235]: pgmap v427: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 9.8 MiB/s wr, 690 op/s 2024-05-11T13:47:30.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:30 smithi063 ceph-mon[29809]: pgmap v427: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 9.8 MiB/s wr, 690 op/s 2024-05-11T13:47:31.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 365.0 s, 463.2 tps, lat 212.617 ms stddev 38.808 2024-05-11T13:47:31.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:31 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:47:31.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:31 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:47:31.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:31 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:47:32.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:32 smithi165 ceph-mon[33235]: pgmap v428: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 480 op/s 2024-05-11T13:47:32.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:32 smithi063 ceph-mon[29809]: pgmap v428: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 480 op/s 2024-05-11T13:47:32.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:32 smithi071 ceph-mon[33293]: pgmap v428: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 480 op/s 2024-05-11T13:47:33.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:33 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:47:33.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:33 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:47:33.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:33 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:47:33.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:33 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:47:33.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:33 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:47:33.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:33 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:47:33.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:33 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:47:33.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:33 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:47:33.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:33 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:47:34.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:34 smithi165 ceph-mon[33235]: pgmap v429: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 480 op/s 2024-05-11T13:47:34.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:34 smithi063 ceph-mon[29809]: pgmap v429: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 480 op/s 2024-05-11T13:47:34.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:34 smithi071 ceph-mon[33293]: pgmap v429: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 480 op/s 2024-05-11T13:47:36.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 370.0 s, 434.8 tps, lat 231.804 ms stddev 39.716 2024-05-11T13:47:36.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:36 smithi165 ceph-mon[33235]: pgmap v430: 129 pgs: 129 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 10 MiB/s wr, 726 op/s 2024-05-11T13:47:36.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:36 smithi063 ceph-mon[29809]: pgmap v430: 129 pgs: 129 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 10 MiB/s wr, 726 op/s 2024-05-11T13:47:36.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:36 smithi071 ceph-mon[33293]: pgmap v430: 129 pgs: 129 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 10 MiB/s wr, 726 op/s 2024-05-11T13:47:38.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:38 smithi071 ceph-mon[33293]: pgmap v431: 129 pgs: 129 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.7 MiB/s wr, 486 op/s 2024-05-11T13:47:38.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:38 smithi165 ceph-mon[33235]: pgmap v431: 129 pgs: 129 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.7 MiB/s wr, 486 op/s 2024-05-11T13:47:38.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:38 smithi063 ceph-mon[29809]: pgmap v431: 129 pgs: 129 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.7 MiB/s wr, 486 op/s 2024-05-11T13:47:40.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:40 smithi165 ceph-mon[33235]: pgmap v432: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 11 MiB/s wr, 749 op/s 2024-05-11T13:47:40.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:40 smithi063 ceph-mon[29809]: pgmap v432: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 11 MiB/s wr, 749 op/s 2024-05-11T13:47:40.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:40 smithi071 ceph-mon[33293]: pgmap v432: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 11 MiB/s wr, 749 op/s 2024-05-11T13:47:41.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 375.0 s, 346.2 tps, lat 289.413 ms stddev 320.000 2024-05-11T13:47:42.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:42 smithi165 ceph-mon[33235]: pgmap v433: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 509 op/s 2024-05-11T13:47:42.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:42 smithi063 ceph-mon[29809]: pgmap v433: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 509 op/s 2024-05-11T13:47:42.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:42 smithi071 ceph-mon[33293]: pgmap v433: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 509 op/s 2024-05-11T13:47:44.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:44 smithi165 ceph-mon[33235]: pgmap v434: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 509 op/s 2024-05-11T13:47:44.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:44 smithi063 ceph-mon[29809]: pgmap v434: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 509 op/s 2024-05-11T13:47:44.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:44 smithi071 ceph-mon[33293]: pgmap v434: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 509 op/s 2024-05-11T13:47:46.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 380.0 s, 350.6 tps, lat 272.427 ms stddev 299.492 2024-05-11T13:47:46.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:46 smithi165 ceph-mon[33235]: pgmap v435: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 11 MiB/s wr, 688 op/s 2024-05-11T13:47:46.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:46 smithi063 ceph-mon[29809]: pgmap v435: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 11 MiB/s wr, 688 op/s 2024-05-11T13:47:46.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:46 smithi071 ceph-mon[33293]: pgmap v435: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 11 MiB/s wr, 688 op/s 2024-05-11T13:47:48.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:48 smithi063 ceph-mon[29809]: pgmap v436: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 7.2 MiB/s wr, 442 op/s 2024-05-11T13:47:48.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:48 smithi165 ceph-mon[33235]: pgmap v436: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 7.2 MiB/s wr, 442 op/s 2024-05-11T13:47:48.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:48 smithi071 ceph-mon[33293]: pgmap v436: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 7.2 MiB/s wr, 442 op/s 2024-05-11T13:47:50.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:50 smithi165 ceph-mon[33235]: pgmap v437: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-05-11T13:47:50.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:50 smithi063 ceph-mon[29809]: pgmap v437: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-05-11T13:47:50.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:50 smithi071 ceph-mon[33293]: pgmap v437: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-05-11T13:47:51.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 385.0 s, 393.8 tps, lat 230.045 ms stddev 70.106 2024-05-11T13:47:52.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:52 smithi165 ceph-mon[33235]: pgmap v438: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-05-11T13:47:52.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:52 smithi063 ceph-mon[29809]: pgmap v438: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-05-11T13:47:52.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:52 smithi071 ceph-mon[33293]: pgmap v438: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-05-11T13:47:54.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:54 smithi165 ceph-mon[33235]: pgmap v439: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-05-11T13:47:54.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:54 smithi063 ceph-mon[29809]: pgmap v439: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-05-11T13:47:54.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:54 smithi071 ceph-mon[33293]: pgmap v439: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.6 MiB/s wr, 386 op/s 2024-05-11T13:47:56.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 390.0 s, 402.7 tps, lat 275.812 ms stddev 352.370 2024-05-11T13:47:56.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:56 smithi165 ceph-mon[33235]: pgmap v440: 129 pgs: 129 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 11 MiB/s wr, 620 op/s 2024-05-11T13:47:56.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:56 smithi063 ceph-mon[29809]: pgmap v440: 129 pgs: 129 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 11 MiB/s wr, 620 op/s 2024-05-11T13:47:56.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:56 smithi071 ceph-mon[33293]: pgmap v440: 129 pgs: 129 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 11 MiB/s wr, 620 op/s 2024-05-11T13:47:58.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:47:58 smithi165 ceph-mon[33235]: pgmap v441: 129 pgs: 129 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 8.1 MiB/s wr, 441 op/s 2024-05-11T13:47:58.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:47:58 smithi063 ceph-mon[29809]: pgmap v441: 129 pgs: 129 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 8.1 MiB/s wr, 441 op/s 2024-05-11T13:47:58.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:47:58 smithi071 ceph-mon[33293]: pgmap v441: 129 pgs: 129 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 8.1 MiB/s wr, 441 op/s 2024-05-11T13:48:00.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:00 smithi165 ceph-mon[33235]: pgmap v442: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-05-11T13:48:00.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:00 smithi063 ceph-mon[29809]: pgmap v442: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-05-11T13:48:00.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:00 smithi071 ceph-mon[33293]: pgmap v442: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-05-11T13:48:01.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 395.0 s, 382.2 tps, lat 269.407 ms stddev 270.387 2024-05-11T13:48:02.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:02 smithi165 ceph-mon[33235]: pgmap v443: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.8 MiB/s wr, 511 op/s 2024-05-11T13:48:02.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:02 smithi063 ceph-mon[29809]: pgmap v443: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.8 MiB/s wr, 511 op/s 2024-05-11T13:48:02.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:02 smithi071 ceph-mon[33293]: pgmap v443: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.8 MiB/s wr, 511 op/s 2024-05-11T13:48:04.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:04 smithi063 ceph-mon[29809]: pgmap v444: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.8 MiB/s wr, 511 op/s 2024-05-11T13:48:04.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:04 smithi071 ceph-mon[33293]: pgmap v444: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.8 MiB/s wr, 511 op/s 2024-05-11T13:48:04.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:04 smithi165 ceph-mon[33235]: pgmap v444: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 7.8 MiB/s wr, 511 op/s 2024-05-11T13:48:06.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 400.0 s, 350.8 tps, lat 249.650 ms stddev 113.139 2024-05-11T13:48:06.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:06 smithi063 ceph-mon[29809]: pgmap v445: 129 pgs: 129 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-05-11T13:48:06.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:06 smithi071 ceph-mon[33293]: pgmap v445: 129 pgs: 129 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-05-11T13:48:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:06 smithi165 ceph-mon[33235]: pgmap v445: 129 pgs: 129 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-05-11T13:48:08.608 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:08 smithi165 ceph-mon[33235]: pgmap v446: 129 pgs: 129 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.8 MiB/s wr, 435 op/s 2024-05-11T13:48:08.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:08 smithi063 ceph-mon[29809]: pgmap v446: 129 pgs: 129 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.8 MiB/s wr, 435 op/s 2024-05-11T13:48:08.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:08 smithi071 ceph-mon[33293]: pgmap v446: 129 pgs: 129 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.8 MiB/s wr, 435 op/s 2024-05-11T13:48:10.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:10 smithi063 ceph-mon[29809]: pgmap v447: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 11 MiB/s wr, 665 op/s 2024-05-11T13:48:10.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:10 smithi063 ceph-mon[29809]: osdmap e108: 12 total, 12 up, 12 in 2024-05-11T13:48:10.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:10 smithi071 ceph-mon[33293]: pgmap v447: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 11 MiB/s wr, 665 op/s 2024-05-11T13:48:10.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:10 smithi071 ceph-mon[33293]: osdmap e108: 12 total, 12 up, 12 in 2024-05-11T13:48:10.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:10 smithi165 ceph-mon[33235]: pgmap v447: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 11 MiB/s wr, 665 op/s 2024-05-11T13:48:10.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:10 smithi165 ceph-mon[33235]: osdmap e108: 12 total, 12 up, 12 in 2024-05-11T13:48:11.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 405.0 s, 400.0 tps, lat 280.281 ms stddev 268.532 2024-05-11T13:48:12.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:12 smithi063 ceph-mon[29809]: pgmap v449: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.6 MiB/s wr, 466 op/s 2024-05-11T13:48:12.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:12 smithi071 ceph-mon[33293]: pgmap v449: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.6 MiB/s wr, 466 op/s 2024-05-11T13:48:12.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:12 smithi165 ceph-mon[33235]: pgmap v449: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.6 MiB/s wr, 466 op/s 2024-05-11T13:48:13.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:13 smithi063 ceph-mon[29809]: pgmap v450: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.6 MiB/s wr, 466 op/s 2024-05-11T13:48:13.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:13 smithi071 ceph-mon[33293]: pgmap v450: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.6 MiB/s wr, 466 op/s 2024-05-11T13:48:13.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:13 smithi165 ceph-mon[33235]: pgmap v450: 129 pgs: 129 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.6 MiB/s wr, 466 op/s 2024-05-11T13:48:16.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 410.0 s, 441.6 tps, lat 227.346 ms stddev 97.248 2024-05-11T13:48:16.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:16 smithi063 ceph-mon[29809]: pgmap v451: 129 pgs: 129 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 7.4 MiB/s wr, 473 op/s 2024-05-11T13:48:16.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:16 smithi063 ceph-mon[29809]: osdmap e109: 12 total, 12 up, 12 in 2024-05-11T13:48:16.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:16 smithi071 ceph-mon[33293]: pgmap v451: 129 pgs: 129 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 7.4 MiB/s wr, 473 op/s 2024-05-11T13:48:16.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:16 smithi071 ceph-mon[33293]: osdmap e109: 12 total, 12 up, 12 in 2024-05-11T13:48:16.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:16 smithi165 ceph-mon[33235]: pgmap v451: 129 pgs: 129 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 7.4 MiB/s wr, 473 op/s 2024-05-11T13:48:16.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:16 smithi165 ceph-mon[33235]: osdmap e109: 12 total, 12 up, 12 in 2024-05-11T13:48:18.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:18 smithi071 ceph-mon[33293]: pgmap v453: 129 pgs: 129 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-05-11T13:48:18.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:18 smithi165 ceph-mon[33235]: pgmap v453: 129 pgs: 129 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-05-11T13:48:18.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:18 smithi063 ceph-mon[29809]: pgmap v453: 129 pgs: 129 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-05-11T13:48:20.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:20 smithi071 ceph-mon[33293]: pgmap v454: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 6.9 MiB/s wr, 555 op/s 2024-05-11T13:48:20.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:20 smithi165 ceph-mon[33235]: pgmap v454: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 6.9 MiB/s wr, 555 op/s 2024-05-11T13:48:20.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:20 smithi063 ceph-mon[29809]: pgmap v454: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 6.9 MiB/s wr, 555 op/s 2024-05-11T13:48:21.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 415.0 s, 449.0 tps, lat 222.285 ms stddev 48.461 2024-05-11T13:48:22.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:22 smithi071 ceph-mon[33293]: pgmap v455: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 6.3 MiB/s wr, 501 op/s 2024-05-11T13:48:22.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:22 smithi165 ceph-mon[33235]: pgmap v455: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 6.3 MiB/s wr, 501 op/s 2024-05-11T13:48:22.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:22 smithi063 ceph-mon[29809]: pgmap v455: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 6.3 MiB/s wr, 501 op/s 2024-05-11T13:48:24.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:24 smithi165 ceph-mon[33235]: pgmap v456: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 6.3 MiB/s wr, 501 op/s 2024-05-11T13:48:24.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:24 smithi063 ceph-mon[29809]: pgmap v456: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 6.3 MiB/s wr, 501 op/s 2024-05-11T13:48:24.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:24 smithi071 ceph-mon[33293]: pgmap v456: 129 pgs: 129 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 6.3 MiB/s wr, 501 op/s 2024-05-11T13:48:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 420.0 s, 418.8 tps, lat 237.745 ms stddev 109.547 2024-05-11T13:48:26.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:26 smithi165 ceph-mon[33235]: pgmap v457: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-05-11T13:48:26.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:26 smithi063 ceph-mon[29809]: pgmap v457: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-05-11T13:48:26.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:26 smithi071 ceph-mon[33293]: pgmap v457: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-05-11T13:48:28.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:28 smithi165 ceph-mon[33235]: pgmap v458: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 6.3 MiB/s wr, 521 op/s 2024-05-11T13:48:28.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:28 smithi063 ceph-mon[29809]: pgmap v458: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 6.3 MiB/s wr, 521 op/s 2024-05-11T13:48:28.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:28 smithi071 ceph-mon[33293]: pgmap v458: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 6.3 MiB/s wr, 521 op/s 2024-05-11T13:48:30.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:30 smithi165 ceph-mon[33235]: pgmap v459: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 8.3 MiB/s wr, 694 op/s 2024-05-11T13:48:30.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:30 smithi063 ceph-mon[29809]: pgmap v459: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 8.3 MiB/s wr, 694 op/s 2024-05-11T13:48:30.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:30 smithi071 ceph-mon[33293]: pgmap v459: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 8.3 MiB/s wr, 694 op/s 2024-05-11T13:48:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 425.0 s, 426.4 tps, lat 233.358 ms stddev 39.950 2024-05-11T13:48:32.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:32 smithi165 ceph-mon[33235]: pgmap v460: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-05-11T13:48:32.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:32 smithi063 ceph-mon[29809]: pgmap v460: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-05-11T13:48:32.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:32 smithi071 ceph-mon[33293]: pgmap v460: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-05-11T13:48:33.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:33 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:48:33.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:33 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:48:33.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:33 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:48:34.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:34 smithi165 ceph-mon[33235]: pgmap v461: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-05-11T13:48:34.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:34 smithi063 ceph-mon[29809]: pgmap v461: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-05-11T13:48:34.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:34 smithi071 ceph-mon[33293]: pgmap v461: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-05-11T13:48:35.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:35 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:48:35.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:35 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:48:35.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:35 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:48:35.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:35 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:48:35.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:35 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:48:35.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:35 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:48:35.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:35 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:48:35.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:35 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:48:35.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:35 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:48:36.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 430.0 s, 445.8 tps, lat 217.017 ms stddev 39.636 2024-05-11T13:48:36.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:36 smithi165 ceph-mon[33235]: pgmap v462: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 671 op/s 2024-05-11T13:48:36.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:36 smithi063 ceph-mon[29809]: pgmap v462: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 671 op/s 2024-05-11T13:48:36.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:36 smithi071 ceph-mon[33293]: pgmap v462: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.9 MiB/s wr, 671 op/s 2024-05-11T13:48:38.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:38 smithi165 ceph-mon[33235]: pgmap v463: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.1 MiB/s wr, 438 op/s 2024-05-11T13:48:38.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:38 smithi063 ceph-mon[29809]: pgmap v463: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.1 MiB/s wr, 438 op/s 2024-05-11T13:48:38.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:38 smithi071 ceph-mon[33293]: pgmap v463: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.1 MiB/s wr, 438 op/s 2024-05-11T13:48:40.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:40 smithi165 ceph-mon[33235]: pgmap v464: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 7.8 MiB/s wr, 677 op/s 2024-05-11T13:48:40.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:40 smithi063 ceph-mon[29809]: pgmap v464: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 7.8 MiB/s wr, 677 op/s 2024-05-11T13:48:40.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:40 smithi071 ceph-mon[33293]: pgmap v464: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 7.8 MiB/s wr, 677 op/s 2024-05-11T13:48:41.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 435.0 s, 388.8 tps, lat 266.497 ms stddev 82.361 2024-05-11T13:48:42.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:42 smithi165 ceph-mon[33235]: pgmap v465: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-05-11T13:48:42.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:42 smithi063 ceph-mon[29809]: pgmap v465: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-05-11T13:48:42.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:42 smithi071 ceph-mon[33293]: pgmap v465: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-05-11T13:48:44.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:44 smithi165 ceph-mon[33235]: pgmap v466: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-05-11T13:48:44.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:44 smithi063 ceph-mon[29809]: pgmap v466: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-05-11T13:48:44.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:44 smithi071 ceph-mon[33293]: pgmap v466: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-05-11T13:48:46.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 440.0 s, 388.2 tps, lat 257.936 ms stddev 43.131 2024-05-11T13:48:46.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:46 smithi165 ceph-mon[33235]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 7.7 MiB/s wr, 677 op/s 2024-05-11T13:48:46.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:46 smithi063 ceph-mon[29809]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 7.7 MiB/s wr, 677 op/s 2024-05-11T13:48:46.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:46 smithi071 ceph-mon[33293]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 7.7 MiB/s wr, 677 op/s 2024-05-11T13:48:48.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:48 smithi165 ceph-mon[33235]: pgmap v468: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-05-11T13:48:48.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:48 smithi063 ceph-mon[29809]: pgmap v468: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-05-11T13:48:48.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:48 smithi071 ceph-mon[33293]: pgmap v468: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-05-11T13:48:50.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:50 smithi165 ceph-mon[33235]: pgmap v469: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 7.2 MiB/s wr, 635 op/s 2024-05-11T13:48:50.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:50 smithi063 ceph-mon[29809]: pgmap v469: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 7.2 MiB/s wr, 635 op/s 2024-05-11T13:48:50.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:50 smithi071 ceph-mon[33293]: pgmap v469: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 7.2 MiB/s wr, 635 op/s 2024-05-11T13:48:51.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 445.0 s, 393.4 tps, lat 255.404 ms stddev 85.553 2024-05-11T13:48:52.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:52 smithi165 ceph-mon[33235]: pgmap v470: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-05-11T13:48:52.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:52 smithi063 ceph-mon[29809]: pgmap v470: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-05-11T13:48:52.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:52 smithi071 ceph-mon[33293]: pgmap v470: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-05-11T13:48:54.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:54 smithi063 ceph-mon[29809]: pgmap v471: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-05-11T13:48:54.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:54 smithi071 ceph-mon[33293]: pgmap v471: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-05-11T13:48:54.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:54 smithi165 ceph-mon[33235]: pgmap v471: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-05-11T13:48:56.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 450.0 s, 430.6 tps, lat 230.882 ms stddev 45.836 2024-05-11T13:48:56.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:56 smithi063 ceph-mon[29809]: pgmap v472: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-05-11T13:48:56.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:56 smithi071 ceph-mon[33293]: pgmap v472: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-05-11T13:48:56.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:56 smithi165 ceph-mon[33235]: pgmap v472: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-05-11T13:48:56.921 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-11T13:48:56.922 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.9 2024-05-11T13:48:56.922 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:48:57.459 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:48:58.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:58 smithi063 ceph-mon[29809]: pgmap v473: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 4.6 MiB/s wr, 413 op/s 2024-05-11T13:48:58.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:48:58 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:48:58.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:58 smithi071 ceph-mon[33293]: pgmap v473: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 4.6 MiB/s wr, 413 op/s 2024-05-11T13:48:58.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:48:58 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:48:58.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:58 smithi165 ceph-mon[33235]: pgmap v473: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 4.6 MiB/s wr, 413 op/s 2024-05-11T13:48:58.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:48:58 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:00.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 -- 172.21.15.63:0/2582277738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103610 msgr2=0x7f07e8103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:00.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2582277738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103610 0x7f07e8103a90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07d00099c0 tx=0x7f07d0009cd0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 -- 172.21.15.63:0/2582277738 shutdown_connections 2024-05-11T13:49:00.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2582277738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103fd0 0x7f07e81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2582277738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07e81095f0 0x7f07e81099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2582277738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103610 0x7f07e8103a90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 -- 172.21.15.63:0/2582277738 >> 172.21.15.63:0/2582277738 conn(0x7f07e807a710 msgr2=0x7f07e807ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:00.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 -- 172.21.15.63:0/2582277738 shutdown_connections 2024-05-11T13:49:00.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.331+0000 7f07ecc00700 1 -- 172.21.15.63:0/2582277738 wait complete. 2024-05-11T13:49:00.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.333+0000 7f07ecc00700 1 Processor -- start 2024-05-11T13:49:00.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.333+0000 7f07ecc00700 1 -- start start 2024-05-11T13:49:00.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07ecc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103610 0x7f07e81a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:00.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07ecc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103fd0 0x7f07e81a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07ecc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07e81095f0 0x7f07e810f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07ecc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8113650 con 0x7f07e8103610 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07ecc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f07e81134d0 con 0x7f07e8103fd0 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07ecc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07e81137d0 con 0x7f07e81095f0 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103610 0x7f07e81a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103610 0x7f07e81a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34300/0 (socket says 172.21.15.63:34300) 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07e5e00700 1 -- 172.21.15.63:0/2424222225 learned_addr learned my addr 172.21.15.63:0/2424222225 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07e5400700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103fd0 0x7f07e81a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.334+0000 7f07e6800700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07e81095f0 0x7f07e810f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.335+0000 7f07e5e00700 1 -- 172.21.15.63:0/2424222225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07e81095f0 msgr2=0x7f07e810f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.335+0000 7f07e5e00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07e81095f0 0x7f07e810f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.335+0000 7f07e5e00700 1 -- 172.21.15.63:0/2424222225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103fd0 msgr2=0x7f07e81a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.335+0000 7f07e5e00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103fd0 0x7f07e81a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.335+0000 7f07e5e00700 1 -- 172.21.15.63:0/2424222225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e810fd60 con 0x7f07e8103610 2024-05-11T13:49:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.335+0000 7f07e5e00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103610 0x7f07e81a5d90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f07e00060b0 tx=0x7f07e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.336+0000 7f07dea00700 1 -- 172.21.15.63:0/2424222225 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e001a070 con 0x7f07e8103610 2024-05-11T13:49:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.336+0000 7f07dea00700 1 -- 172.21.15.63:0/2424222225 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07e000ddd0 con 0x7f07e8103610 2024-05-11T13:49:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.336+0000 7f07dea00700 1 -- 172.21.15.63:0/2424222225 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0014690 con 0x7f07e8103610 2024-05-11T13:49:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.337+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e8110050 con 0x7f07e8103610 2024-05-11T13:49:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.337+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e81acbf0 con 0x7f07e8103610 2024-05-11T13:49:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.339+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e8104c60 con 0x7f07e8103610 2024-05-11T13:49:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.343+0000 7f07dea00700 1 -- 172.21.15.63:0/2424222225 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f07e001d070 con 0x7f07e8103610 2024-05-11T13:49:00.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.347+0000 7f07dea00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f07cc0b2a70 0x7f07cc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.347+0000 7f07dea00700 1 -- 172.21.15.63:0/2424222225 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f07e00e26c0 con 0x7f07e8103610 2024-05-11T13:49:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.348+0000 7f07e5400700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f07cc0b2a70 0x7f07cc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.350+0000 7f07dea00700 1 -- 172.21.15.63:0/2424222225 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07e0010070 con 0x7f07e8103610 2024-05-11T13:49:00.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.351+0000 7f07e5400700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f07cc0b2a70 0x7f07cc0b4f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07e81a16a0 tx=0x7f07d0005fb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:00.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:00 smithi063 ceph-mon[29809]: pgmap v474: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-11T13:49:00.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:00 smithi071 ceph-mon[33293]: pgmap v474: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-11T13:49:00.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.777+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07e806a240 con 0x7f07e8103610 2024-05-11T13:49:00.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.780+0000 7f07dea00700 1 -- 172.21.15.63:0/2424222225 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+11936 (secure 0 0 0) 0x7f07e00a9cb0 con 0x7f07e8103610 2024-05-11T13:49:00.780 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:49:00.780 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:48:56.547217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24472,34403,34427,34451,34472]},"id":1}]} 2024-05-11T13:49:00.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.791+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f07cc0b2a70 msgr2=0x7f07cc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:00.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.791+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f07cc0b2a70 0x7f07cc0b4f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07e81a16a0 tx=0x7f07d0005fb0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.791+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103610 msgr2=0x7f07e81a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:00.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.791+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103610 0x7f07e81a5d90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f07e00060b0 tx=0x7f07e000d3b0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.796+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 shutdown_connections 2024-05-11T13:49:00.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.796+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f07cc0b2a70 0x7f07cc0b4f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.796+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07e81095f0 0x7f07e810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.796+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e8103610 0x7f07e81a5d90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.796+0000 7f07ecc00700 1 --2- 172.21.15.63:0/2424222225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07e8103fd0 0x7f07e81a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:00.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.796+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 >> 172.21.15.63:0/2424222225 conn(0x7f07e807a710 msgr2=0x7f07e81021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:00.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.797+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 shutdown_connections 2024-05-11T13:49:00.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:00.797+0000 7f07ecc00700 1 -- 172.21.15.63:0/2424222225 wait complete. 2024-05-11T13:49:00.791 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 68 2024-05-11T13:49:00.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:00 smithi165 ceph-mon[33235]: pgmap v474: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-11T13:49:01.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 455.0 s, 360.2 tps, lat 277.398 ms stddev 135.376 2024-05-11T13:49:01.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:01 smithi063 ceph-mon[29809]: pgmap v475: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-05-11T13:49:01.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:01 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2424222225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:01.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:01 smithi071 ceph-mon[33293]: pgmap v475: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-05-11T13:49:01.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:01 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2424222225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:01.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:01 smithi165 ceph-mon[33235]: pgmap v475: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-05-11T13:49:01.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:01 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2424222225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:02.854 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 79.9 --await-for 42.0' 2024-05-11T13:49:02.855 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 79.9 --await-for 42.0 2024-05-11T13:49:03.623 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:49:03.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:03 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:03.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:03 smithi063 ceph-mon[29809]: pgmap v476: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-05-11T13:49:03.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:03 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:03.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:03 smithi071 ceph-mon[33293]: pgmap v476: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-05-11T13:49:03.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:03 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:03.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:03 smithi165 ceph-mon[33235]: pgmap v476: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-05-11T13:49:05.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:05 smithi063 ceph-mon[29809]: pgmap v477: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.9 MiB/s wr, 646 op/s 2024-05-11T13:49:05.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:05 smithi071 ceph-mon[33293]: pgmap v477: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.9 MiB/s wr, 646 op/s 2024-05-11T13:49:05.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:05 smithi165 ceph-mon[33235]: pgmap v477: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.9 MiB/s wr, 646 op/s 2024-05-11T13:49:06.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 460.0 s, 362.4 tps, lat 275.810 ms stddev 92.717 2024-05-11T13:49:07.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.342+0000 7f1ad5c00700 1 -- 172.21.15.63:0/4195187449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01035d0 msgr2=0x7f1ad0103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.342+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/4195187449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01035d0 0x7f1ad0103a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009990 tx=0x7f1ab8009ca0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.343+0000 7f1ad5c00700 1 -- 172.21.15.63:0/4195187449 shutdown_connections 2024-05-11T13:49:07.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.343+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/4195187449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad0103f90 0x7f1ad0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.343+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/4195187449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad01096e0 0x7f1ad0109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.343+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/4195187449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01035d0 0x7f1ad0103a50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.343+0000 7f1ad5c00700 1 -- 172.21.15.63:0/4195187449 >> 172.21.15.63:0/4195187449 conn(0x7f1ad007a700 msgr2=0x7f1ad007ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:07.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.343+0000 7f1ad5c00700 1 -- 172.21.15.63:0/4195187449 shutdown_connections 2024-05-11T13:49:07.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.343+0000 7f1ad5c00700 1 -- 172.21.15.63:0/4195187449 wait complete. 2024-05-11T13:49:07.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.345+0000 7f1ad5c00700 1 Processor -- start 2024-05-11T13:49:07.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.346+0000 7f1ad5c00700 1 -- start start 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.346+0000 7f1ad5c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad01035d0 0x7f1ad010f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.346+0000 7f1ad5c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0103f90 0x7f1ad010fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.346+0000 7f1ad5c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01096e0 0x7f1ad01a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.346+0000 7f1ad5c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01136a0 con 0x7f1ad01035d0 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.346+0000 7f1ad5c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0113520 con 0x7f1ad0103f90 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.346+0000 7f1ad5c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0113820 con 0x7f1ad01096e0 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.347+0000 7f1ac6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0103f90 0x7f1ad010fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.347+0000 7f1ac6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0103f90 0x7f1ad010fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:55976/0 (socket says 172.21.15.63:55976) 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.347+0000 7f1ac6000700 1 -- 172.21.15.63:0/2215416812 learned_addr learned my addr 172.21.15.63:0/2215416812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.347+0000 7f1acea00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad01035d0 0x7f1ad010f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.347+0000 7f1acf400700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01096e0 0x7f1ad01a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.348+0000 7f1ac6000700 1 -- 172.21.15.63:0/2215416812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01096e0 msgr2=0x7f1ad01a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.348+0000 7f1ac6000700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01096e0 0x7f1ad01a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.348+0000 7f1ac6000700 1 -- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad01035d0 msgr2=0x7f1ad010f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.348+0000 7f1ac6000700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad01035d0 0x7f1ad010f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.348+0000 7f1ac6000700 1 -- 172.21.15.63:0/2215416812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad01a50d0 con 0x7f1ad0103f90 2024-05-11T13:49:07.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.348+0000 7f1ac6000700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0103f90 0x7f1ad010fcf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800b650 tx=0x7f1ab80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:07.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.349+0000 7f1ac7e00700 1 -- 172.21.15.63:0/2215416812 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8026070 con 0x7f1ad0103f90 2024-05-11T13:49:07.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.349+0000 7f1ac7e00700 1 -- 172.21.15.63:0/2215416812 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ab8021410 con 0x7f1ad0103f90 2024-05-11T13:49:07.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.349+0000 7f1ac7e00700 1 -- 172.21.15.63:0/2215416812 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab800fe60 con 0x7f1ad0103f90 2024-05-11T13:49:07.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.349+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad01a5360 con 0x7f1ad0103f90 2024-05-11T13:49:07.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.350+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad01acc40 con 0x7f1ad0103f90 2024-05-11T13:49:07.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.355+0000 7f1ac7e00700 1 -- 172.21.15.63:0/2215416812 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1ab8029070 con 0x7f1ad0103f90 2024-05-11T13:49:07.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.357+0000 7f1ae05d7b80 1 Processor -- start 2024-05-11T13:49:07.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.357+0000 7f1ac7e00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ab40b2bf0 0x7f1ab40b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.358+0000 7f1ac7e00700 1 -- 172.21.15.63:0/2215416812 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f1ab80e3640 con 0x7f1ad0103f90 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.358+0000 7f1ae05d7b80 1 -- start start 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.358+0000 7f1acf400700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ab40b2bf0 0x7f1ab40b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02523f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e025239730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e02523a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e02518bbb0 con 0x55e025239d50 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e0251298f0 con 0x55e02523f230 2024-05-11T13:49:07.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e025187b70 con 0x55e02523fb50 2024-05-11T13:49:07.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ac6000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e025239730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ac6000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e025239730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:54738/0 (socket says 172.21.15.63:54738) 2024-05-11T13:49:07.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.359+0000 7f1ac6000700 1 -- 172.21.15.63:0/1913619408 learned_addr learned my addr 172.21.15.63:0/1913619408 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:07.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.360+0000 7f1acea00700 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02523f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.360+0000 7f1acea00700 1 -- 172.21.15.63:0/1913619408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 msgr2=0x55e025239730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:07.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.360+0000 7f1acea00700 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e025239730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.360+0000 7f1acea00700 1 -- 172.21.15.63:0/1913619408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 msgr2=0x55e02523a1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T13:49:07.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.360+0000 7f1acf400700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ab40b2bf0 0x7f1ab40b50b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8007bd0 tx=0x7f1ac80072a0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:07.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.360+0000 7f1acea00700 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e02523a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.361+0000 7f1acea00700 1 -- 172.21.15.63:0/1913619408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0250d8a60 con 0x55e02523f230 2024-05-11T13:49:07.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.361+0000 7f1acea00700 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02523f610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0011580 tx=0x7f1ac0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c57b770149128 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:07.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.361+0000 7f1abfe00700 1 -- 172.21.15.63:0/1913619408 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0019030 con 0x55e02523f230 2024-05-11T13:49:07.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.362+0000 7f1abfe00700 1 -- 172.21.15.63:0/1913619408 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac00092e0 con 0x55e02523f230 2024-05-11T13:49:07.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.362+0000 7f1abfe00700 1 -- 172.21.15.63:0/1913619408 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004320 con 0x55e02523f230 2024-05-11T13:49:07.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1913619408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 msgr2=0x55e02523f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02523f610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0011580 tx=0x7f1ac0009920 comp rx=0 tx=0).stop 2024-05-11T13:49:07.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1913619408 shutdown_connections 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e02523a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02523f610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1913619408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e025239730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1913619408 >> 172.21.15.63:0/1913619408 conn(0x55e025167cf0 msgr2=0x55e025234350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.363+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1913619408 shutdown_connections 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.364+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1913619408 wait complete. 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.364+0000 7f1ae05d7b80 1 -- start start 2024-05-11T13:49:07.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.365+0000 7f1ae05d7b80 1 Processor -- start 2024-05-11T13:49:07.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025245910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e025242cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e0252431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e025246670 con 0x55e025239d50 2024-05-11T13:49:07.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e0250f1cb0 con 0x55e02523f230 2024-05-11T13:49:07.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0252467f0 con 0x55e02523fb50 2024-05-11T13:49:07.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ae05d7b80 10 client.0 _ll_register_callbacks cb 0x55e02511f510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:49:07.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ac6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e025242cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.366+0000 7f1ac6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e025242cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:55992/0 (socket says 172.21.15.63:55992) 2024-05-11T13:49:07.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1ac6000700 1 -- 172.21.15.63:0/1780904115 learned_addr learned my addr 172.21.15.63:0/1780904115 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1acea00700 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025245910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1acf400700 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e0252431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1ac6000700 1 -- 172.21.15.63:0/1780904115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 msgr2=0x55e0252431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1ac6000700 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e0252431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1ac6000700 1 -- 172.21.15.63:0/1780904115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 msgr2=0x55e025245910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1ac6000700 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025245910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.367+0000 7f1ac6000700 1 -- 172.21.15.63:0/1780904115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0251515b0 con 0x55e02523f230 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.368+0000 7f1ac6000700 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e025242cb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80a77d0 tx=0x7f1ab8021ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0453c6541edd5c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.368+0000 7f1abcc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:49:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.368+0000 7f1abcc00700 1 -- 172.21.15.63:0/1780904115 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80aa560 con 0x55e02523f230 2024-05-11T13:49:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.368+0000 7f1abcc00700 1 -- 172.21.15.63:0/1780904115 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f1ab80aa9d0 con 0x55e02523f230 2024-05-11T13:49:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.369+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac00092e0 con 0x55e02523f230 2024-05-11T13:49:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.369+0000 7f1abcc00700 1 -- 172.21.15.63:0/1780904115 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1ab80ae910 con 0x55e02523f230 2024-05-11T13:49:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.369+0000 7f1ae05d7b80 10 client.35585 fetch_fsmap learned FSMap version 69 2024-05-11T13:49:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.369+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e025250f70 con 0x55e02523f230 2024-05-11T13:49:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1abcc00700 1 -- 172.21.15.63:0/1780904115 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac00092e0 con 0x55e02523f230 2024-05-11T13:49:07.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1abcc00700 1 -- 172.21.15.63:0/1780904115 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 69) ==== 8378+0+0 (secure 0 0 0) 0x7f1ab80b2fa0 con 0x55e02523f230 2024-05-11T13:49:07.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1ae05d7b80 10 client.35585 fetch_fsmap finished waiting for FSMap version 69 2024-05-11T13:49:07.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1ae05d7b80 10 client.35585 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:49:07.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1ae05d7b80 20 client.35585 populate_metadata read hostname 'smithi063' 2024-05-11T13:49:07.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e025253490 0x55e0252558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1ae05d7b80 4 client.35585 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:49:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.370+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0252061f0 con 0x55e025253490 2024-05-11T13:49:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.371+0000 7f1acea00700 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e025253490 0x55e0252558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.372+0000 7f1acea00700 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e025253490 0x55e0252558b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62225829f13673c server_cookie=e83d3d68f8243bb5 in_seq=0 out_seq=0 2024-05-11T13:49:07.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.372+0000 7f1abcc00700 10 client.35585 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:07.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.373+0000 7f1abcc00700 1 -- 172.21.15.63:0/1780904115 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0252061f0 con 0x55e025253490 2024-05-11T13:49:07.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.373+0000 7f1abcc00700 10 client.35585 handle_command_reply: tid=0 2024-05-11T13:49:07.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.373+0000 7f1ae05d7b80 1 client.35585 shutdown 2024-05-11T13:49:07.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.377+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 msgr2=0x55e025242cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.377+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e025242cb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80a77d0 tx=0x7f1ab8021ba0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 shutdown_connections 2024-05-11T13:49:07.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e0252431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025245910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e025253490 0x55e0252558b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1780904115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e025242cb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 >> 172.21.15.63:0/1780904115 conn(0x55e025167cf0 msgr2=0x55e025238720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:07.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1abcc00700 0 client.35585 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:07.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 shutdown_connections 2024-05-11T13:49:07.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1780904115 wait complete. 2024-05-11T13:49:07.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.378+0000 7f1ae05d7b80 20 client.35585 trim_cache size 0 max 16384 2024-05-11T13:49:07.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.411+0000 7f1ae05d7b80 1 Processor -- start 2024-05-11T13:49:07.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.411+0000 7f1ae05d7b80 1 -- start start 2024-05-11T13:49:07.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025239730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02524f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e02523e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01b5a40 con 0x55e025239d50 2024-05-11T13:49:07.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01b7640 con 0x55e02523f230 2024-05-11T13:49:07.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01b6110 con 0x55e02523fb50 2024-05-11T13:49:07.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1acea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025239730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1acea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025239730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34350/0 (socket says 172.21.15.63:34350) 2024-05-11T13:49:07.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.412+0000 7f1acea00700 1 -- 172.21.15.63:0/2318001534 learned_addr learned my addr 172.21.15.63:0/2318001534 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:07.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.413+0000 7f1ac6000700 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02524f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.413+0000 7f1acf400700 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e02523e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.413+0000 7f1acea00700 1 -- 172.21.15.63:0/2318001534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 msgr2=0x55e02523e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.414+0000 7f1acea00700 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e02523e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.414+0000 7f1acea00700 1 -- 172.21.15.63:0/2318001534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 msgr2=0x55e02524f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.414+0000 7f1acea00700 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02524f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.414+0000 7f1acea00700 1 -- 172.21.15.63:0/2318001534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e025235c90 con 0x55e025239d50 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.414+0000 7f1acf400700 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e02523e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.414+0000 7f1acea00700 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025239730 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000f80 tx=0x7f1ac000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddaa132b1afc40f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:07.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.415+0000 7f1abf400700 1 -- 172.21.15.63:0/2318001534 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00285f0 con 0x55e025239d50 2024-05-11T13:49:07.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.415+0000 7f1abf400700 1 -- 172.21.15.63:0/2318001534 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac0007500 con 0x55e025239d50 2024-05-11T13:49:07.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.415+0000 7f1abf400700 1 -- 172.21.15.63:0/2318001534 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac001f6b0 con 0x55e025239d50 2024-05-11T13:49:07.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.416+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/2318001534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 msgr2=0x55e025239730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.416+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025239730 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000f80 tx=0x7f1ac000b040 comp rx=0 tx=0).stop 2024-05-11T13:49:07.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.416+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/2318001534 shutdown_connections 2024-05-11T13:49:07.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.416+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523fb50 0x55e02523e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.416+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025239730 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.416+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/2318001534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523f230 0x55e02524f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.416+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/2318001534 >> 172.21.15.63:0/2318001534 conn(0x55e025167cf0 msgr2=0x55e02523de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:07.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.417+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/2318001534 shutdown_connections 2024-05-11T13:49:07.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.417+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/2318001534 wait complete. 2024-05-11T13:49:07.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.418+0000 7f1ae05d7b80 1 -- start start 2024-05-11T13:49:07.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.418+0000 7f1ae05d7b80 1 Processor -- start 2024-05-11T13:49:07.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.419+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025248da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.419+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523f230 0x55e0252492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.419+0000 7f1ae05d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523fb50 0x55e0252bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.420+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01b6f70 con 0x55e025239d50 2024-05-11T13:49:07.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.420+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01b67e0 con 0x55e02523fb50 2024-05-11T13:49:07.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.420+0000 7f1ae05d7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01b3a00 con 0x55e02523f230 2024-05-11T13:49:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.420+0000 7f1ae05d7b80 10 client.0 _ll_register_callbacks cb 0x55e02511f510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:49:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.420+0000 7f1acea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025248da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1ac6000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523f230 0x55e0252492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523fb50 0x55e0252bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523fb50 0x55e0252bdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:55996/0 (socket says 172.21.15.63:55996) 2024-05-11T13:49:07.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 -- 172.21.15.63:0/1363636492 learned_addr learned my addr 172.21.15.63:0/1363636492 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 -- 172.21.15.63:0/1363636492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523f230 msgr2=0x55e0252492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523f230 0x55e0252492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 -- 172.21.15.63:0/1363636492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 msgr2=0x55e025248da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025248da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acf400700 1 -- 172.21.15.63:0/1363636492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0252be2b0 con 0x55e02523fb50 2024-05-11T13:49:07.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.421+0000 7f1acea00700 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025248da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:49:07.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.422+0000 7f1acf400700 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523fb50 0x55e0252bdd40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80040d0 tx=0x7f1ac8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51415d77efbac6a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:07.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.422+0000 7f1abcc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:49:07.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.423+0000 7f1abcc00700 1 -- 172.21.15.63:0/1363636492 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8025070 con 0x55e02523fb50 2024-05-11T13:49:07.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.423+0000 7f1abcc00700 1 -- 172.21.15.63:0/1363636492 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f1ac8003c90 con 0x55e02523fb50 2024-05-11T13:49:07.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.423+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac0007500 con 0x55e02523fb50 2024-05-11T13:49:07.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.423+0000 7f1abcc00700 1 -- 172.21.15.63:0/1363636492 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1ac8031050 con 0x55e02523fb50 2024-05-11T13:49:07.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.424+0000 7f1ae05d7b80 10 client.35591 fetch_fsmap learned FSMap version 69 2024-05-11T13:49:07.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.424+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e0252bf560 con 0x55e02523fb50 2024-05-11T13:49:07.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1abcc00700 1 -- 172.21.15.63:0/1363636492 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac0007500 con 0x55e02523fb50 2024-05-11T13:49:07.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1abcc00700 1 -- 172.21.15.63:0/1363636492 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 69) ==== 8378+0+0 (secure 0 0 0) 0x7f1ac80214a0 con 0x55e02523fb50 2024-05-11T13:49:07.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1ae05d7b80 10 client.35591 fetch_fsmap finished waiting for FSMap version 69 2024-05-11T13:49:07.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1ae05d7b80 10 client.35591 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:49:07.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1ae05d7b80 20 client.35591 populate_metadata read hostname 'smithi063' 2024-05-11T13:49:07.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e0252bfd60 0x55e0252c2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:07.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1ae05d7b80 4 client.35591 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.9, "await_for": 42.0}] 2024-05-11T13:49:07.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.425+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.9, "await_for": 42.0}) -- 0x55e025199590 con 0x55e0252bfd60 2024-05-11T13:49:07.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.426+0000 7f1acea00700 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e0252bfd60 0x55e0252c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:07.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.428+0000 7f1acea00700 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e0252bfd60 0x55e0252c2150 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b937fcf830a51483 server_cookie=8aa7ffe40c83fdad in_seq=0 out_seq=0 2024-05-11T13:49:07.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.428+0000 7f1abcc00700 10 client.35591 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:07.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.677+0000 7f1abcc00700 1 -- 172.21.15.63:0/1363636492 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e025199590 con 0x55e0252bfd60 2024-05-11T13:49:07.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.677+0000 7f1abcc00700 10 client.35591 handle_command_reply: tid=0 2024-05-11T13:49:07.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.677+0000 7f1ae05d7b80 1 client.35591 shutdown 2024-05-11T13:49:07.667 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:07 smithi063 ceph-mon[29809]: pgmap v478: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-05-11T13:49:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523fb50 msgr2=0x55e0252bdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523fb50 0x55e0252bdd40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80040d0 tx=0x7f1ac8004750 comp rx=0 tx=0).stop 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 shutdown_connections 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55e0252bfd60 0x55e0252c2150 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e02523fb50 0x55e0252bdd40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e025239d50 0x55e025248da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 --2- 172.21.15.63:0/1363636492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02523f230 0x55e0252492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 >> 172.21.15.63:0/1363636492 conn(0x55e025167cf0 msgr2=0x55e02523e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1abcc00700 0 client.35591 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 shutdown_connections 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.678+0000 7f1ae05d7b80 1 -- 172.21.15.63:0/1363636492 wait complete. 2024-05-11T13:49:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.679+0000 7f1ae05d7b80 20 client.35591 trim_cache size 0 max 16384 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 69, 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 42, 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "592de27": { 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 42, 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:49:07.670 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 79.9, 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:07.671 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:49:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.680+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ab40b2bf0 msgr2=0x7f1ab40b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.680+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ab40b2bf0 0x7f1ab40b50b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8007bd0 tx=0x7f1ac80072a0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.680+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0103f90 msgr2=0x7f1ad010fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.680+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0103f90 0x7f1ad010fcf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800b650 tx=0x7f1ab80057c0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.680+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 shutdown_connections 2024-05-11T13:49:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.680+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ab40b2bf0 0x7f1ab40b50b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.681+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ad01096e0 0x7f1ad01a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.681+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad01035d0 0x7f1ad010f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.681+0000 7f1ad5c00700 1 --2- 172.21.15.63:0/2215416812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0103f90 0x7f1ad010fcf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:07.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.681+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 >> 172.21.15.63:0/2215416812 conn(0x7f1ad007a700 msgr2=0x7f1ad0101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:07.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.681+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 shutdown_connections 2024-05-11T13:49:07.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:07.681+0000 7f1ad5c00700 1 -- 172.21.15.63:0/2215416812 wait complete. 2024-05-11T13:49:07.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:07 smithi165 ceph-mon[33235]: pgmap v478: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-05-11T13:49:07.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:07 smithi071 ceph-mon[33293]: pgmap v478: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-05-11T13:49:09.409 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '592de27', quiesce took 12.5 seconds. Will release after: 19.9 2024-05-11T13:49:09.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:09 smithi063 ceph-mon[29809]: pgmap v479: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-05-11T13:49:09.745 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:09 smithi071 ceph-mon[33293]: pgmap v479: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-05-11T13:49:09.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:09 smithi165 ceph-mon[33235]: pgmap v479: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-05-11T13:49:10.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:10 smithi165 ceph-mon[33235]: osdmap e110: 12 total, 12 up, 12 in 2024-05-11T13:49:10.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:10 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:10.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:10 smithi063 ceph-mon[29809]: osdmap e110: 12 total, 12 up, 12 in 2024-05-11T13:49:10.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:10 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:10.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:10 smithi071 ceph-mon[33293]: osdmap e110: 12 total, 12 up, 12 in 2024-05-11T13:49:10.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:10 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:11.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 465.0 s, 97.7 tps, lat 267.110 ms stddev 46.407 2024-05-11T13:49:11.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:11 smithi165 ceph-mon[33235]: pgmap v481: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.8 MiB/s wr, 449 op/s 2024-05-11T13:49:11.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:11 smithi063 ceph-mon[29809]: pgmap v481: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.8 MiB/s wr, 449 op/s 2024-05-11T13:49:11.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:11 smithi071 ceph-mon[33293]: pgmap v481: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.8 MiB/s wr, 449 op/s 2024-05-11T13:49:13.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:13 smithi165 ceph-mon[33235]: pgmap v482: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.8 MiB/s wr, 449 op/s 2024-05-11T13:49:13.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:13 smithi063 ceph-mon[29809]: pgmap v482: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.8 MiB/s wr, 449 op/s 2024-05-11T13:49:13.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:13 smithi071 ceph-mon[33293]: pgmap v482: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.8 MiB/s wr, 449 op/s 2024-05-11T13:49:16.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 470.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:49:16.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:16 smithi165 ceph-mon[33235]: pgmap v483: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-05-11T13:49:16.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:16 smithi165 ceph-mon[33235]: osdmap e111: 12 total, 12 up, 12 in 2024-05-11T13:49:16.627 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:16 smithi063 ceph-mon[29809]: pgmap v483: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-05-11T13:49:16.627 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:16 smithi063 ceph-mon[29809]: osdmap e111: 12 total, 12 up, 12 in 2024-05-11T13:49:16.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:16 smithi071 ceph-mon[33293]: pgmap v483: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-05-11T13:49:16.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:16 smithi071 ceph-mon[33293]: osdmap e111: 12 total, 12 up, 12 in 2024-05-11T13:49:18.364 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:18 smithi063 ceph-mon[29809]: pgmap v485: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 132 KiB/s wr, 23 op/s 2024-05-11T13:49:18.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:18 smithi165 ceph-mon[33235]: pgmap v485: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 132 KiB/s wr, 23 op/s 2024-05-11T13:49:18.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:18 smithi071 ceph-mon[33293]: pgmap v485: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 132 KiB/s wr, 23 op/s 2024-05-11T13:49:20.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:20 smithi165 ceph-mon[33235]: pgmap v486: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-05-11T13:49:20.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:20 smithi063 ceph-mon[29809]: pgmap v486: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-05-11T13:49:20.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:20 smithi071 ceph-mon[33293]: pgmap v486: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-05-11T13:49:21.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 475.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:49:22.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:22 smithi165 ceph-mon[33235]: pgmap v487: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 170 KiB/s wr, 29 op/s 2024-05-11T13:49:22.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:22 smithi063 ceph-mon[29809]: pgmap v487: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 170 KiB/s wr, 29 op/s 2024-05-11T13:49:22.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:22 smithi071 ceph-mon[33293]: pgmap v487: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 170 KiB/s wr, 29 op/s 2024-05-11T13:49:24.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:24 smithi165 ceph-mon[33235]: pgmap v488: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 170 KiB/s wr, 29 op/s 2024-05-11T13:49:24.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:24 smithi063 ceph-mon[29809]: pgmap v488: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 170 KiB/s wr, 29 op/s 2024-05-11T13:49:24.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:24 smithi071 ceph-mon[33293]: pgmap v488: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 170 KiB/s wr, 29 op/s 2024-05-11T13:49:26.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 480.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:49:26.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:26 smithi165 ceph-mon[33235]: pgmap v489: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 778 KiB/s wr, 103 op/s 2024-05-11T13:49:26.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:26 smithi063 ceph-mon[29809]: pgmap v489: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 778 KiB/s wr, 103 op/s 2024-05-11T13:49:26.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:26 smithi071 ceph-mon[33293]: pgmap v489: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 778 KiB/s wr, 103 op/s 2024-05-11T13:49:28.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:28 smithi165 ceph-mon[33235]: pgmap v490: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 695 KiB/s wr, 92 op/s 2024-05-11T13:49:28.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:28 smithi063 ceph-mon[29809]: pgmap v490: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 695 KiB/s wr, 92 op/s 2024-05-11T13:49:28.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:28 smithi071 ceph-mon[33293]: pgmap v490: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 695 KiB/s wr, 92 op/s 2024-05-11T13:49:29.310 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:49:29.811 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:49:30.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:30 smithi165 ceph-mon[33235]: pgmap v491: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 651 KiB/s wr, 91 op/s 2024-05-11T13:49:30.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:30 smithi063 ceph-mon[29809]: pgmap v491: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 651 KiB/s wr, 91 op/s 2024-05-11T13:49:30.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:30 smithi071 ceph-mon[33293]: pgmap v491: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 651 KiB/s wr, 91 op/s 2024-05-11T13:49:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 485.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:49:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 -- 172.21.15.63:0/2692330061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec0760a0 msgr2=0x7f9eec10fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 --2- 172.21.15.63:0/2692330061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec0760a0 0x7f9eec10fa40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009990 tx=0x7f9ed8009ca0 comp rx=0 tx=0).stop 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 -- 172.21.15.63:0/2692330061 shutdown_connections 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 --2- 172.21.15.63:0/2692330061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec10ff80 0x7f9eec112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 --2- 172.21.15.63:0/2692330061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec075780 0x7f9eec075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 --2- 172.21.15.63:0/2692330061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec0760a0 0x7f9eec10fa40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 -- 172.21.15.63:0/2692330061 >> 172.21.15.63:0/2692330061 conn(0x7f9eec07a700 msgr2=0x7f9eec07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 -- 172.21.15.63:0/2692330061 shutdown_connections 2024-05-11T13:49:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.002+0000 7f9ef1400700 1 -- 172.21.15.63:0/2692330061 wait complete. 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.003+0000 7f9ef1400700 1 Processor -- start 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.003+0000 7f9ef1400700 1 -- start start 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9ef1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec075780 0x7f9eec19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9ef1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec0760a0 0x7f9eec19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9ef1400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec10ff80 0x7f9eec1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9ef1400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec115870 con 0x7f9eec075780 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9ef1400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1156f0 con 0x7f9eec0760a0 2024-05-11T13:49:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9ef1400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1159f0 con 0x7f9eec10ff80 2024-05-11T13:49:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9eea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec075780 0x7f9eec19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9eea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec075780 0x7f9eec19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50570/0 (socket says 172.21.15.63:50570) 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.004+0000 7f9eea800700 1 -- 172.21.15.63:0/3620803082 learned_addr learned my addr 172.21.15.63:0/3620803082 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9eeb200700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec10ff80 0x7f9eec1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9ee9e00700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec0760a0 0x7f9eec19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9eea800700 1 -- 172.21.15.63:0/3620803082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec10ff80 msgr2=0x7f9eec1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9eea800700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec10ff80 0x7f9eec1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9eea800700 1 -- 172.21.15.63:0/3620803082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec0760a0 msgr2=0x7f9eec19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9eea800700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec0760a0 0x7f9eec19c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9eea800700 1 -- 172.21.15.63:0/3620803082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec1a0df0 con 0x7f9eec075780 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9ee9e00700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec0760a0 0x7f9eec19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:49:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.005+0000 7f9eea800700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec075780 0x7f9eec19bde0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00d950 tx=0x7f9edc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.006+0000 7f9ee3400700 1 -- 172.21.15.63:0/3620803082 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc016070 con 0x7f9eec075780 2024-05-11T13:49:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.006+0000 7f9ee3400700 1 -- 172.21.15.63:0/3620803082 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9edc009dd0 con 0x7f9eec075780 2024-05-11T13:49:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.006+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec1a10e0 con 0x7f9eec075780 2024-05-11T13:49:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.006+0000 7f9ee3400700 1 -- 172.21.15.63:0/3620803082 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00f3c0 con 0x7f9eec075780 2024-05-11T13:49:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.006+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec1a8960 con 0x7f9eec075780 2024-05-11T13:49:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.007+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec06a240 con 0x7f9eec075780 2024-05-11T13:49:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.010+0000 7f9ee3400700 1 -- 172.21.15.63:0/3620803082 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9edc019070 con 0x7f9eec075780 2024-05-11T13:49:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.012+0000 7f9ee3400700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9ed00b29f0 0x7f9ed00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.012+0000 7f9ee3400700 1 -- 172.21.15.63:0/3620803082 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f9edc0e26a0 con 0x7f9eec075780 2024-05-11T13:49:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.013+0000 7f9ee9e00700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9ed00b29f0 0x7f9ed00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.013+0000 7f9ee9e00700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9ed00b29f0 0x7f9ed00b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006010 tx=0x7f9ed801a040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:32.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.015+0000 7f9ee3400700 1 -- 172.21.15.63:0/3620803082 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9edc0a9c30 con 0x7f9eec075780 2024-05-11T13:49:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.303+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eec02e570 con 0x7f9eec075780 2024-05-11T13:49:32.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.305+0000 7f9ee3400700 1 -- 172.21.15.63:0/3620803082 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+11943 (secure 0 0 0) 0x7f9edc0a9530 con 0x7f9eec075780 2024-05-11T13:49:32.295 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:49:32.295 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:49:08.852236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:49:32.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.308+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9ed00b29f0 msgr2=0x7f9ed00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.308+0000 7f9ef1400700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9ed00b29f0 0x7f9ed00b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006010 tx=0x7f9ed801a040 comp rx=0 tx=0).stop 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.308+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec075780 msgr2=0x7f9eec19bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.308+0000 7f9ef1400700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec075780 0x7f9eec19bde0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00d950 tx=0x7f9edc00dc60 comp rx=0 tx=0).stop 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 shutdown_connections 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9ed00b29f0 0x7f9ed00b4eb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec10ff80 0x7f9eec1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eec075780 0x7f9eec19bde0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 --2- 172.21.15.63:0/3620803082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec0760a0 0x7f9eec19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 >> 172.21.15.63:0/3620803082 conn(0x7f9eec07a700 msgr2=0x7f9eec100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:32.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 shutdown_connections 2024-05-11T13:49:32.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:32.309+0000 7f9ef1400700 1 -- 172.21.15.63:0/3620803082 wait complete. 2024-05-11T13:49:32.299 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 70 2024-05-11T13:49:32.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:32 smithi165 ceph-mon[33235]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 560 KiB/s wr, 77 op/s 2024-05-11T13:49:32.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:32 smithi063 ceph-mon[29809]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 560 KiB/s wr, 77 op/s 2024-05-11T13:49:32.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:32 smithi071 ceph-mon[33293]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 560 KiB/s wr, 77 op/s 2024-05-11T13:49:33.143 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 592de27 --release --await' 2024-05-11T13:49:33.143 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 592de27 --release --await 2024-05-11T13:49:33.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:33 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3620803082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:33.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:33 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3620803082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:33.608 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:49:33.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:33 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3620803082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:34.398 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:34 smithi063 ceph-mon[29809]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 560 KiB/s wr, 77 op/s 2024-05-11T13:49:34.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:34 smithi165 ceph-mon[33235]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 560 KiB/s wr, 77 op/s 2024-05-11T13:49:34.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:34 smithi071 ceph-mon[33293]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 560 KiB/s wr, 77 op/s 2024-05-11T13:49:35.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:35 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:49:35.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:35 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:49:35.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:35 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 -- 172.21.15.63:0/1291347156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c10ffb0 msgr2=0x7f041c1123a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 --2- 172.21.15.63:0/1291347156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c10ffb0 0x7f041c1123a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f041400b0a0 tx=0x7f041400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 -- 172.21.15.63:0/1291347156 shutdown_connections 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 --2- 172.21.15.63:0/1291347156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c10ffb0 0x7f041c1123a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 --2- 172.21.15.63:0/1291347156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041c075780 0x7f041c075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 --2- 172.21.15.63:0/1291347156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c0760a0 0x7f041c10fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 -- 172.21.15.63:0/1291347156 >> 172.21.15.63:0/1291347156 conn(0x7f041c07a6a0 msgr2=0x7f041c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 -- 172.21.15.63:0/1291347156 shutdown_connections 2024-05-11T13:49:35.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.982+0000 7f0421c00700 1 -- 172.21.15.63:0/1291347156 wait complete. 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.983+0000 7f0421c00700 1 Processor -- start 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.983+0000 7f0421c00700 1 -- start start 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f0421c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c075780 0x7f041c19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f0421c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041c0760a0 0x7f041c19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f0421c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c10ffb0 0x7f041c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f0421c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1158f0 con 0x7f041c075780 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f0421c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f041c115770 con 0x7f041c10ffb0 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f0421c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f041c115a70 con 0x7f041c0760a0 2024-05-11T13:49:35.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f041aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c075780 0x7f041c19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f041aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c075780 0x7f041c19be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50588/0 (socket says 172.21.15.63:50588) 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f041a000700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041c0760a0 0x7f041c19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.984+0000 7f041aa00700 1 -- 172.21.15.63:0/111992671 learned_addr learned my addr 172.21.15.63:0/111992671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.985+0000 7f041b400700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c10ffb0 0x7f041c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.985+0000 7f041aa00700 1 -- 172.21.15.63:0/111992671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041c0760a0 msgr2=0x7f041c19c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.985+0000 7f041aa00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041c0760a0 0x7f041c19c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.985+0000 7f041aa00700 1 -- 172.21.15.63:0/111992671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c10ffb0 msgr2=0x7f041c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.985+0000 7f041aa00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c10ffb0 0x7f041c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.985+0000 7f041aa00700 1 -- 172.21.15.63:0/111992671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1a0e00 con 0x7f041c075780 2024-05-11T13:49:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.985+0000 7f041aa00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c075780 0x7f041c19be40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9c0 tx=0x7f040c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.986+0000 7f0413400700 1 -- 172.21.15.63:0/111992671 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00cbd0 con 0x7f041c075780 2024-05-11T13:49:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.986+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1a10f0 con 0x7f041c075780 2024-05-11T13:49:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.986+0000 7f0413400700 1 -- 172.21.15.63:0/111992671 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f040c00cd70 con 0x7f041c075780 2024-05-11T13:49:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.986+0000 7f0413400700 1 -- 172.21.15.63:0/111992671 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c005540 con 0x7f041c075780 2024-05-11T13:49:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.986+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a8970 con 0x7f041c075780 2024-05-11T13:49:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.990+0000 7f0413400700 1 -- 172.21.15.63:0/111992671 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f040c004750 con 0x7f041c075780 2024-05-11T13:49:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 Processor -- start 2024-05-11T13:49:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 -- start start 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc441256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc44126150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc44077b20 con 0x55dc4412b1a0 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dc44015860 con 0x55dc4412bac0 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.991+0000 7f042c58bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dc44073ae0 con 0x55dc44125cc0 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50602/0 (socket says 172.21.15.63:50602) 2024-05-11T13:49:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 -- 172.21.15.63:0/1139879460 learned_addr learned my addr 172.21.15.63:0/1139879460 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:35.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041aa00700 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc441256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041a000700 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc44126150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f0413400700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f04000b29f0 0x7f04000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 -- 172.21.15.63:0/1139879460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 msgr2=0x55dc44126150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:35.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc44126150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 -- 172.21.15.63:0/1139879460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 msgr2=0x55dc441256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc441256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.992+0000 7f041b400700 1 -- 172.21.15.63:0/1139879460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc43fc49d0 con 0x55dc4412b1a0 2024-05-11T13:49:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.993+0000 7f0413400700 1 -- 172.21.15.63:0/111992671 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f040c0e2d20 con 0x7f041c075780 2024-05-11T13:49:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.993+0000 7f041b400700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f04000b29f0 0x7f04000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.993+0000 7f041b400700 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412b580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f041400b0a0 tx=0x7f0414009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b2fa1b87bae26 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.994+0000 7f0407e00700 1 -- 172.21.15.63:0/1139879460 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414004a50 con 0x55dc4412b1a0 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.994+0000 7f0407e00700 1 -- 172.21.15.63:0/1139879460 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0414022040 con 0x55dc4412b1a0 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.994+0000 7f0407e00700 1 -- 172.21.15.63:0/1139879460 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041402f050 con 0x55dc4412b1a0 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 -- 172.21.15.63:0/1139879460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 msgr2=0x55dc4412b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412b580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f041400b0a0 tx=0x7f0414009d90 comp rx=0 tx=0).stop 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f041b400700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f04000b29f0 0x7f04000b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0414028000 tx=0x7f041402b000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 -- 172.21.15.63:0/1139879460 shutdown_connections 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc44126150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412b580 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 --2- 172.21.15.63:0/1139879460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc441256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 -- 172.21.15.63:0/1139879460 >> 172.21.15.63:0/1139879460 conn(0x55dc44053c60 msgr2=0x55dc441202c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 -- 172.21.15.63:0/1139879460 shutdown_connections 2024-05-11T13:49:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.995+0000 7f042c58bb80 1 -- 172.21.15.63:0/1139879460 wait complete. 2024-05-11T13:49:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 -- start start 2024-05-11T13:49:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 Processor -- start 2024-05-11T13:49:35.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc44125cc0 0x55dc44131960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412bac0 0x55dc4412f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc441325e0 con 0x55dc4412b1a0 2024-05-11T13:49:35.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dc43fddc20 con 0x55dc44125cc0 2024-05-11T13:49:35.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dc44132760 con 0x55dc4412bac0 2024-05-11T13:49:35.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.996+0000 7f042c58bb80 10 client.0 _ll_register_callbacks cb 0x55dc4400b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:49:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50606/0 (socket says 172.21.15.63:50606) 2024-05-11T13:49:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 -- 172.21.15.63:0/549819234 learned_addr learned my addr 172.21.15.63:0/549819234 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041b400700 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412bac0 0x55dc4412f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041aa00700 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc44125cc0 0x55dc44131960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 -- 172.21.15.63:0/549819234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412bac0 msgr2=0x55dc4412f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412bac0 0x55dc4412f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 -- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc44125cc0 msgr2=0x55dc44131960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc44125cc0 0x55dc44131960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.997+0000 7f041a000700 1 -- 172.21.15.63:0/549819234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4403d520 con 0x55dc4412b1a0 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.998+0000 7f041a000700 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412ed00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0408007f50 tx=0x7f0408011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0a1c11914bbfd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.998+0000 7f0404c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.998+0000 7f0404c00700 1 -- 172.21.15.63:0/549819234 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408004830 con 0x55dc4412b1a0 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.998+0000 7f0404c00700 1 -- 172.21.15.63:0/549819234 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f04080049d0 con 0x55dc4412b1a0 2024-05-11T13:49:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.998+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0414022040 con 0x55dc4412b1a0 2024-05-11T13:49:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f0404c00700 1 -- 172.21.15.63:0/549819234 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0408004bd0 con 0x55dc4412b1a0 2024-05-11T13:49:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 10 client.16017 fetch_fsmap learned FSMap version 70 2024-05-11T13:49:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55dc4413d040 con 0x55dc4412b1a0 2024-05-11T13:49:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f0404c00700 1 -- 172.21.15.63:0/549819234 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0414022040 con 0x55dc4412b1a0 2024-05-11T13:49:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f0404c00700 1 -- 172.21.15.63:0/549819234 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7f04080050c0 con 0x55dc4412b1a0 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 10 client.16017 fetch_fsmap finished waiting for FSMap version 70 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 10 client.16017 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 20 client.16017 populate_metadata read hostname 'smithi063' 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc4413f680 0x55dc44141a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 4 client.16017 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:35.999+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc440f2160 con 0x55dc4413f680 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.000+0000 7f041aa00700 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc4413f680 0x55dc44141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.001+0000 7f041aa00700 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc4413f680 0x55dc44141a70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105b358c66c4b28b server_cookie=4f98ec93c3457707 in_seq=0 out_seq=0 2024-05-11T13:49:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.001+0000 7f0404c00700 10 client.16017 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:35.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.002+0000 7f0404c00700 1 -- 172.21.15.63:0/549819234 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc440f2160 con 0x55dc4413f680 2024-05-11T13:49:35.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.002+0000 7f0404c00700 10 client.16017 handle_command_reply: tid=0 2024-05-11T13:49:35.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.002+0000 7f042c58bb80 1 client.16017 shutdown 2024-05-11T13:49:35.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 msgr2=0x55dc4412ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:35.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412ed00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0408007f50 tx=0x7f0408011720 comp rx=0 tx=0).stop 2024-05-11T13:49:35.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 shutdown_connections 2024-05-11T13:49:35.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412bac0 0x55dc4412f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc4413f680 0x55dc44141a70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc44125cc0 0x55dc44131960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 --2- 172.21.15.63:0/549819234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4412ed00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:35.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 >> 172.21.15.63:0/549819234 conn(0x55dc44053c60 msgr2=0x55dc44124690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:35.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f0404c00700 0 client.16017 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:35.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 shutdown_connections 2024-05-11T13:49:35.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 1 -- 172.21.15.63:0/549819234 wait complete. 2024-05-11T13:49:35.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.003+0000 7f042c58bb80 20 client.16017 trim_cache size 0 max 16384 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.025+0000 7f042c58bb80 1 Processor -- start 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.025+0000 7f042c58bb80 1 -- start start 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.025+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc441256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.025+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4413b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.025+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44147340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f042c58bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b1770 con 0x55dc4412b1a0 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f042c58bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b3370 con 0x55dc4412bac0 2024-05-11T13:49:36.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f042c58bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b1e40 con 0x55dc44125cc0 2024-05-11T13:49:36.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4413b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:36.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4413b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50608/0 (socket says 172.21.15.63:50608) 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 -- 172.21.15.63:0/2456587779 learned_addr learned my addr 172.21.15.63:0/2456587779 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041aa00700 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc441256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041b400700 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 -- 172.21.15.63:0/2456587779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 msgr2=0x55dc441256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc441256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 -- 172.21.15.63:0/2456587779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 msgr2=0x55dc44147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.026+0000 7f041a000700 1 -- 172.21.15.63:0/2456587779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4412f840 con 0x55dc4412b1a0 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.027+0000 7f041a000700 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4413b6c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0408000c00 tx=0x7f0408005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b3decf9f0558ec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.027+0000 7f0407400700 1 -- 172.21.15.63:0/2456587779 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040802c070 con 0x55dc4412b1a0 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.027+0000 7f0407400700 1 -- 172.21.15.63:0/2456587779 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0408005200 con 0x55dc4412b1a0 2024-05-11T13:49:36.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.027+0000 7f0407400700 1 -- 172.21.15.63:0/2456587779 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408010e20 con 0x55dc4412b1a0 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 -- 172.21.15.63:0/2456587779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 msgr2=0x55dc4413b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4413b6c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0408000c00 tx=0x7f0408005960 comp rx=0 tx=0).stop 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 -- 172.21.15.63:0/2456587779 shutdown_connections 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc44125cc0 0x55dc441256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 --2- 172.21.15.63:0/2456587779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc4412b1a0 0x55dc4413b6c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 -- 172.21.15.63:0/2456587779 >> 172.21.15.63:0/2456587779 conn(0x55dc44053c60 msgr2=0x55dc4411fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 -- 172.21.15.63:0/2456587779 shutdown_connections 2024-05-11T13:49:36.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.028+0000 7f042c58bb80 1 -- 172.21.15.63:0/2456587779 wait complete. 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 -- start start 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 Processor -- start 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc44125cc0 0x55dc44137c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412b1a0 0x55dc44138160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44134c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b2ca0 con 0x55dc44125cc0 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b2510 con 0x55dc4412bac0 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.029+0000 7f042c58bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1af730 con 0x55dc4412b1a0 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f042c58bb80 10 client.0 _ll_register_callbacks cb 0x55dc4411b190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc44125cc0 0x55dc44137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc44125cc0 0x55dc44137c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50624/0 (socket says 172.21.15.63:50624) 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 -- 172.21.15.63:0/3641322374 learned_addr learned my addr 172.21.15.63:0/3641322374 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:36.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041a000700 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412b1a0 0x55dc44138160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041b400700 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44134c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 -- 172.21.15.63:0/3641322374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412b1a0 msgr2=0x55dc44138160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412b1a0 0x55dc44138160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 -- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 msgr2=0x55dc44134c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44134c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.030+0000 7f041aa00700 1 -- 172.21.15.63:0/3641322374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc44135450 con 0x55dc44125cc0 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f041aa00700 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc44125cc0 0x55dc44137c20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f040c0a6e20 tx=0x7f040c0e67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3109b7322605d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f0404c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f0404c00700 1 -- 172.21.15.63:0/3641322374 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0cae60 con 0x55dc44125cc0 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f0404c00700 1 -- 172.21.15.63:0/3641322374 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f040c0aa020 con 0x55dc44125cc0 2024-05-11T13:49:36.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0408005200 con 0x55dc44125cc0 2024-05-11T13:49:36.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f0404c00700 1 -- 172.21.15.63:0/3641322374 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f040c0a9980 con 0x55dc44125cc0 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f042c58bb80 10 client.16023 fetch_fsmap learned FSMap version 70 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.031+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55dc441aac10 con 0x55dc44125cc0 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f0404c00700 1 -- 172.21.15.63:0/3641322374 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0408005200 con 0x55dc44125cc0 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f0404c00700 1 -- 172.21.15.63:0/3641322374 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7f040c0c4120 con 0x55dc44125cc0 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f042c58bb80 10 client.16023 fetch_fsmap finished waiting for FSMap version 70 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f042c58bb80 10 client.16023 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f042c58bb80 20 client.16023 populate_metadata read hostname 'smithi063' 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f042c58bb80 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc441ab2a0 0x55dc441ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f042c58bb80 4 client.16023 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "592de27", "release": true, "await": true}] 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "592de27", "release": true, "await": true}) -- 0x55dc44085500 con 0x55dc441ab2a0 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.032+0000 7f041a000700 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc441ab2a0 0x55dc441ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.033+0000 7f041a000700 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc441ab2a0 0x55dc441ad6c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e2f226e38708f8 server_cookie=12f43cb4f5aa806a in_seq=0 out_seq=0 2024-05-11T13:49:36.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.033+0000 7f0404c00700 10 client.16023 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:36.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.071+0000 7f0404c00700 1 -- 172.21.15.63:0/3641322374 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55dc44085500 con 0x55dc441ab2a0 2024-05-11T13:49:36.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.071+0000 7f0404c00700 10 client.16023 handle_command_reply: tid=0 2024-05-11T13:49:36.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.071+0000 7f042c58bb80 1 client.16023 shutdown 2024-05-11T13:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.071+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc44125cc0 msgr2=0x55dc44137c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.071+0000 7f042c58bb80 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc44125cc0 0x55dc44137c20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f040c0a6e20 tx=0x7f040c0e67d0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 shutdown_connections 2024-05-11T13:49:36.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55dc441ab2a0 0x55dc441ad6c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4412bac0 0x55dc44134c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc44125cc0 0x55dc44137c20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 --2- 172.21.15.63:0/3641322374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc4412b1a0 0x55dc44138160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 >> 172.21.15.63:0/3641322374 conn(0x55dc44053c60 msgr2=0x55dc44128cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f0404c00700 0 client.16023 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 shutdown_connections 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.072+0000 7f042c58bb80 1 -- 172.21.15.63:0/3641322374 wait complete. 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.073+0000 7f042c58bb80 20 client.16023 trim_cache size 0 max 16384 2024-05-11T13:49:36.063 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 70, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 44, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "592de27": { 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 44, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 79.9, 2024-05-11T13:49:36.064 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:36.065 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.074+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f04000b29f0 msgr2=0x7f04000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.074+0000 7f0421c00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f04000b29f0 0x7f04000b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0414028000 tx=0x7f041402b000 comp rx=0 tx=0).stop 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.074+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c075780 msgr2=0x7f041c19be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.074+0000 7f0421c00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c075780 0x7f041c19be40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9c0 tx=0x7f040c00ecd0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 shutdown_connections 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f04000b29f0 0x7f04000b4eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c10ffb0 0x7f041c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f041c075780 0x7f041c19be40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 --2- 172.21.15.63:0/111992671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041c0760a0 0x7f041c19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 >> 172.21.15.63:0/111992671 conn(0x7f041c07a6a0 msgr2=0x7f041c100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 shutdown_connections 2024-05-11T13:49:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:36.076+0000 7f0421c00700 1 -- 172.21.15.63:0/111992671 wait complete. 2024-05-11T13:49:36.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 490.0 s, 18.8 tps, lat 27559.674 ms stddev 5798.788 2024-05-11T13:49:36.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:36 smithi063 ceph-mon[29809]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 568 KiB/s wr, 82 op/s 2024-05-11T13:49:36.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:36 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:36.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:36 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:36.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:36 smithi165 ceph-mon[33235]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 568 KiB/s wr, 82 op/s 2024-05-11T13:49:36.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:36 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:36.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:36 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:36.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:36 smithi071 ceph-mon[33293]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 568 KiB/s wr, 82 op/s 2024-05-11T13:49:36.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:36 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:36.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:36 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:36.927 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '592de27', seconds elapsed: 40.0 2024-05-11T13:49:36.928 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.9 2024-05-11T13:49:36.928 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:49:37.595 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:49:38.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:38 smithi063 ceph-mon[29809]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-11T13:49:38.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:38 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:49:38.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:38 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:49:38.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:38 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:38.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:38 smithi165 ceph-mon[33235]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-11T13:49:38.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:38 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:49:38.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:38 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:49:38.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:38 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:38.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:38 smithi071 ceph-mon[33293]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-11T13:49:38.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:38 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:49:38.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:38 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:49:38.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:38 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:49:40.557 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:40 smithi165 ceph-mon[33235]: pgmap v496: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 483 KiB/s wr, 42 op/s 2024-05-11T13:49:40.626 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:40 smithi063 ceph-mon[29809]: pgmap v496: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 483 KiB/s wr, 42 op/s 2024-05-11T13:49:40.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:40 smithi071 ceph-mon[33293]: pgmap v496: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 483 KiB/s wr, 42 op/s 2024-05-11T13:49:40.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 -- 172.21.15.63:0/1292015871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 msgr2=0x7f55d410ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:40.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 --2- 172.21.15.63:0/1292015871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 0x7f55d410ec00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f55c00074e0 tx=0x7f55c000af90 comp rx=0 tx=0).stop 2024-05-11T13:49:40.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 -- 172.21.15.63:0/1292015871 shutdown_connections 2024-05-11T13:49:40.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 --2- 172.21.15.63:0/1292015871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d410f2e0 0x7f55d4116eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:40.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 --2- 172.21.15.63:0/1292015871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 0x7f55d410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:40.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 --2- 172.21.15.63:0/1292015871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 0x7f55d410ec00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:40.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 -- 172.21.15.63:0/1292015871 >> 172.21.15.63:0/1292015871 conn(0x7f55d406e6c0 msgr2=0x7f55d406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:40.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 -- 172.21.15.63:0/1292015871 shutdown_connections 2024-05-11T13:49:40.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.877+0000 7f55d8e00700 1 -- 172.21.15.63:0/1292015871 wait complete. 2024-05-11T13:49:40.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.879+0000 7f55d8e00700 1 Processor -- start 2024-05-11T13:49:40.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.879+0000 7f55d8e00700 1 -- start start 2024-05-11T13:49:40.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.879+0000 7f55d8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 0x7f55d41a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 0x7f55d41a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d8e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d410f2e0 0x7f55d41a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d8e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4119be0 con 0x7f55d410b160 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d8e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4119a60 con 0x7f55d410a7b0 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d8e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4119d60 con 0x7f55d410f2e0 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 0x7f55d41a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 0x7f55d41a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:44264/0 (socket says 172.21.15.63:44264) 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d1e00700 1 -- 172.21.15.63:0/2622583315 learned_addr learned my addr 172.21.15.63:0/2622583315 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:40.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.880+0000 7f55d1400700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 0x7f55d41a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:40.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.881+0000 7f55d1e00700 1 -- 172.21.15.63:0/2622583315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d410f2e0 msgr2=0x7f55d41a6c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T13:49:40.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.881+0000 7f55d2800700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d410f2e0 0x7f55d41a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:40.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.881+0000 7f55d1e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d410f2e0 0x7f55d41a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:40.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.881+0000 7f55d1e00700 1 -- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 msgr2=0x7f55d41a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:40.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.881+0000 7f55d1e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 0x7f55d41a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:40.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.881+0000 7f55d1e00700 1 -- 172.21.15.63:0/2622583315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d41a7380 con 0x7f55d410a7b0 2024-05-11T13:49:40.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.881+0000 7f55d1e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 0x7f55d41a2370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55c400e9a0 tx=0x7f55c400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:40.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.882+0000 7f55caa00700 1 -- 172.21.15.63:0/2622583315 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c400cbd0 con 0x7f55d410a7b0 2024-05-11T13:49:40.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.882+0000 7f55caa00700 1 -- 172.21.15.63:0/2622583315 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c400cd70 con 0x7f55d410a7b0 2024-05-11T13:49:40.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.882+0000 7f55caa00700 1 -- 172.21.15.63:0/2622583315 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4018680 con 0x7f55d410a7b0 2024-05-11T13:49:40.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.882+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d41a7670 con 0x7f55d410a7b0 2024-05-11T13:49:40.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.883+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41aeef0 con 0x7f55d410a7b0 2024-05-11T13:49:40.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.885+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d410b770 con 0x7f55d410a7b0 2024-05-11T13:49:40.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.887+0000 7f55caa00700 1 -- 172.21.15.63:0/2622583315 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f55c4010430 con 0x7f55d410a7b0 2024-05-11T13:49:40.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.891+0000 7f55caa00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f55b80b2a70 0x7f55b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:40.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.891+0000 7f55caa00700 1 -- 172.21.15.63:0/2622583315 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f55c40e2c30 con 0x7f55d410a7b0 2024-05-11T13:49:40.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.892+0000 7f55d1400700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f55b80b2a70 0x7f55b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:40.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.893+0000 7f55d1400700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f55b80b2a70 0x7f55b80b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55c0006010 tx=0x7f55c000c330 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:40.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:40.897+0000 7f55caa00700 1 -- 172.21.15.63:0/2622583315 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55c40aa1c0 con 0x7f55d410a7b0 2024-05-11T13:49:41.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 495.0 s, 381.2 tps, lat 408.623 ms stddev 2060.719 2024-05-11T13:49:41.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.466+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55d41a36d0 con 0x7f55d410a7b0 2024-05-11T13:49:41.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.468+0000 7f55caa00700 1 -- 172.21.15.63:0/2622583315 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+11943 (secure 0 0 0) 0x7f55c40b2240 con 0x7f55d410a7b0 2024-05-11T13:49:41.458 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:49:41.459 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:49:08.852236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:49:41.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.472+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f55b80b2a70 msgr2=0x7f55b80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:41.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f55b80b2a70 0x7f55b80b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55c0006010 tx=0x7f55c000c330 comp rx=0 tx=0).stop 2024-05-11T13:49:41.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 msgr2=0x7f55d41a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:41.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 0x7f55d41a2370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55c400e9a0 tx=0x7f55c400ecb0 comp rx=0 tx=0).stop 2024-05-11T13:49:41.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 shutdown_connections 2024-05-11T13:49:41.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f55b80b2a70 0x7f55b80b4f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:41.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d410f2e0 0x7f55d41a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:41.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d410a7b0 0x7f55d41a2370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:41.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 --2- 172.21.15.63:0/2622583315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d410b160 0x7f55d41a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:41.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.473+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 >> 172.21.15.63:0/2622583315 conn(0x7f55d406e6c0 msgr2=0x7f55d4073c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:41.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.474+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 shutdown_connections 2024-05-11T13:49:41.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:41.474+0000 7f55d8e00700 1 -- 172.21.15.63:0/2622583315 wait complete. 2024-05-11T13:49:41.466 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 70 2024-05-11T13:49:42.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:42 smithi165 ceph-mon[33235]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 481 KiB/s wr, 37 op/s 2024-05-11T13:49:42.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2622583315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:42.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:42 smithi063 ceph-mon[29809]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 481 KiB/s wr, 37 op/s 2024-05-11T13:49:42.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:42 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2622583315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:42.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:42 smithi071 ceph-mon[33293]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 481 KiB/s wr, 37 op/s 2024-05-11T13:49:42.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:42 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2622583315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:49:42.811 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 79.9 --await-for 42.0' 2024-05-11T13:49:42.812 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 79.9 --await-for 42.0 2024-05-11T13:49:43.521 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:49:44.361 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:44 smithi063 ceph-mon[29809]: pgmap v498: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 481 KiB/s wr, 37 op/s 2024-05-11T13:49:44.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:44 smithi071 ceph-mon[33293]: pgmap v498: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 481 KiB/s wr, 37 op/s 2024-05-11T13:49:44.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:44 smithi165 ceph-mon[33235]: pgmap v498: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 481 KiB/s wr, 37 op/s 2024-05-11T13:49:46.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 500.0 s, 392.2 tps, lat 256.705 ms stddev 69.265 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.238+0000 7f9f46400700 1 -- 172.21.15.63:0/3419818077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 msgr2=0x7f9f40103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.238+0000 7f9f46400700 1 --2- 172.21.15.63:0/3419818077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f40103a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00faf0 tx=0x7f9f2c00fe00 comp rx=0 tx=0).stop 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.238+0000 7f9f46400700 1 -- 172.21.15.63:0/3419818077 shutdown_connections 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.238+0000 7f9f46400700 1 --2- 172.21.15.63:0/3419818077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f40103fa0 0x7f9f40110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.238+0000 7f9f46400700 1 --2- 172.21.15.63:0/3419818077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40108510 0x7f9f401088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.238+0000 7f9f46400700 1 --2- 172.21.15.63:0/3419818077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f40103a60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.238+0000 7f9f46400700 1 -- 172.21.15.63:0/3419818077 >> 172.21.15.63:0/3419818077 conn(0x7f9f4007a6b0 msgr2=0x7f9f4007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:46.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.239+0000 7f9f46400700 1 -- 172.21.15.63:0/3419818077 shutdown_connections 2024-05-11T13:49:46.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.239+0000 7f9f46400700 1 -- 172.21.15.63:0/3419818077 wait complete. 2024-05-11T13:49:46.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.240+0000 7f9f46400700 1 Processor -- start 2024-05-11T13:49:46.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f46400700 1 -- start start 2024-05-11T13:49:46.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f46400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f4019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f46400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f40103fa0 0x7f9f4019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f46400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40108510 0x7f9f40197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f46400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401114b0 con 0x7f9f40103fa0 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f46400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40111330 con 0x7f9f40108510 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f46400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40111630 con 0x7f9f40103600 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.241+0000 7f9f3f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f4019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f4019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:45674/0 (socket says 172.21.15.63:45674) 2024-05-11T13:49:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3f400700 1 -- 172.21.15.63:0/2929018941 learned_addr learned my addr 172.21.15.63:0/2929018941 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40108510 0x7f9f40197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3ea00700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f40103fa0 0x7f9f4019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3f400700 1 -- 172.21.15.63:0/2929018941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40108510 msgr2=0x7f9f40197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3f400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40108510 0x7f9f40197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3f400700 1 -- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f40103fa0 msgr2=0x7f9f4019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3f400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f40103fa0 0x7f9f4019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.242+0000 7f9f3f400700 1 -- 172.21.15.63:0/2929018941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f40198100 con 0x7f9f40103600 2024-05-11T13:49:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.243+0000 7f9f3f400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f4019e600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d980 tx=0x7f9f3000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.244+0000 7f9f37e00700 1 -- 172.21.15.63:0/2929018941 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30017070 con 0x7f9f40103600 2024-05-11T13:49:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.244+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f401983f0 con 0x7f9f40103600 2024-05-11T13:49:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.244+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4019c300 con 0x7f9f40103600 2024-05-11T13:49:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.244+0000 7f9f37e00700 1 -- 172.21.15.63:0/2929018941 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f30009dc0 con 0x7f9f40103600 2024-05-11T13:49:46.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.244+0000 7f9f37e00700 1 -- 172.21.15.63:0/2929018941 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000f3c0 con 0x7f9f40103600 2024-05-11T13:49:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.250+0000 7f9f37e00700 1 -- 172.21.15.63:0/2929018941 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9f30009930 con 0x7f9f40103600 2024-05-11T13:49:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.251+0000 7f9f50d52b80 1 Processor -- start 2024-05-11T13:49:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.251+0000 7f9f50d52b80 1 -- start start 2024-05-11T13:49:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.252+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c19410060 0x555c19410440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.252+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940a630 0x555c1940aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.252+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19416e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.252+0000 7f9f50d52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555c1935cbb0 con 0x555c19410060 2024-05-11T13:49:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.252+0000 7f9f50d52b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555c192fa8f0 con 0x555c1940aff0 2024-05-11T13:49:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.252+0000 7f9f50d52b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555c19358b70 con 0x555c1940a630 2024-05-11T13:49:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.252+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19416e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.253+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19416e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:44298/0 (socket says 172.21.15.63:44298) 2024-05-11T13:49:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.253+0000 7f9f3ea00700 1 -- 172.21.15.63:0/1174200571 learned_addr learned my addr 172.21.15.63:0/1174200571 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.253+0000 7f9f3f400700 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940a630 0x555c1940aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.253+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c19410060 0x555c19410440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.254+0000 7f9f3ea00700 1 -- 172.21.15.63:0/1174200571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940a630 msgr2=0x555c1940aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.254+0000 7f9f3ea00700 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940a630 0x555c1940aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.254+0000 7f9f3ea00700 1 -- 172.21.15.63:0/1174200571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c19410060 msgr2=0x555c19410440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.254+0000 7f9f3ea00700 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c19410060 0x555c19410440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.254+0000 7f9f3ea00700 1 -- 172.21.15.63:0/1174200571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c192a9a60 con 0x555c1940aff0 2024-05-11T13:49:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.255+0000 7f9f3ea00700 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19416e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00faf0 tx=0x7f9f2c00f860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f092675175345af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.256+0000 7f9f37e00700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9f240b2b20 0x7f9f240b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.256+0000 7f9f37e00700 1 -- 172.21.15.63:0/2929018941 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f9f300e2380 con 0x7f9f40103600 2024-05-11T13:49:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.256+0000 7f9f34c00700 1 -- 172.21.15.63:0/1174200571 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c026070 con 0x555c1940aff0 2024-05-11T13:49:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.257+0000 7f9f34c00700 1 -- 172.21.15.63:0/1174200571 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c003be0 con 0x555c1940aff0 2024-05-11T13:49:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.257+0000 7f9f34c00700 1 -- 172.21.15.63:0/1174200571 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00f270 con 0x555c1940aff0 2024-05-11T13:49:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.258+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9f240b2b20 0x7f9f240b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.258+0000 7f9f50d52b80 1 -- 172.21.15.63:0/1174200571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 msgr2=0x555c19416e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.258+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19416e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00faf0 tx=0x7f9f2c00f860 comp rx=0 tx=0).stop 2024-05-11T13:49:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.259+0000 7f9f50d52b80 1 -- 172.21.15.63:0/1174200571 shutdown_connections 2024-05-11T13:49:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.259+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19416e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.259+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c19410060 0x555c19410440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.259+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/1174200571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940a630 0x555c1940aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.259+0000 7f9f50d52b80 1 -- 172.21.15.63:0/1174200571 >> 172.21.15.63:0/1174200571 conn(0x555c19338cf0 msgr2=0x555c19404f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.260+0000 7f9f50d52b80 1 -- 172.21.15.63:0/1174200571 shutdown_connections 2024-05-11T13:49:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.261+0000 7f9f50d52b80 1 -- 172.21.15.63:0/1174200571 wait complete. 2024-05-11T13:49:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.262+0000 7f9f50d52b80 1 -- start start 2024-05-11T13:49:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.262+0000 7f9f50d52b80 1 Processor -- start 2024-05-11T13:49:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.263+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9f240b2b20 0x7f9f240b4fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f380099d0 tx=0x7f9f38009050 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.264+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c194169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.264+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940aff0 0x555c19413d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.264+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c19410060 0x555c19414290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.264+0000 7f9f3f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c194169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.264+0000 7f9f50d52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555c19417360 con 0x555c1940a630 2024-05-11T13:49:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.265+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940aff0 0x555c19413d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.265+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940aff0 0x555c19413d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:45696/0 (socket says 172.21.15.63:45696) 2024-05-11T13:49:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.265+0000 7f9f3ea00700 1 -- 172.21.15.63:0/35308220 learned_addr learned my addr 172.21.15.63:0/35308220 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.265+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555c192c2cb0 con 0x555c19410060 2024-05-11T13:49:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.265+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555c194174e0 con 0x555c1940aff0 2024-05-11T13:49:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.265+0000 7f9f50d52b80 10 client.0 _ll_register_callbacks cb 0x555c19303e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:49:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.265+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c19410060 0x555c19414290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.266+0000 7f9f3f400700 1 -- 172.21.15.63:0/35308220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940aff0 msgr2=0x555c19413d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.266+0000 7f9f3f400700 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940aff0 0x555c19413d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.266+0000 7f9f3f400700 1 -- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c19410060 msgr2=0x555c19414290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.266+0000 7f9f3f400700 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c19410060 0x555c19414290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.266+0000 7f9f3f400700 1 -- 172.21.15.63:0/35308220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c193225b0 con 0x555c1940a630 2024-05-11T13:49:46.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.266+0000 7f9f3f400700 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c194169b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f300df680 tx=0x7f9f300dfbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af4642199e3179a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:46.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.266+0000 7f9f29600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:49:46.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.267+0000 7f9f29600700 1 -- 172.21.15.63:0/35308220 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30020070 con 0x555c1940a630 2024-05-11T13:49:46.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.267+0000 7f9f29600700 1 -- 172.21.15.63:0/35308220 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f9f300b1df0 con 0x555c1940a630 2024-05-11T13:49:46.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.267+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c003be0 con 0x555c1940a630 2024-05-11T13:49:46.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.268+0000 7f9f29600700 1 -- 172.21.15.63:0/35308220 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f9f300a8fe0 con 0x555c1940a630 2024-05-11T13:49:46.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.268+0000 7f9f50d52b80 10 client.16041 fetch_fsmap learned FSMap version 70 2024-05-11T13:49:46.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.268+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x555c19421b40 con 0x555c1940a630 2024-05-11T13:49:46.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f29600700 1 -- 172.21.15.63:0/35308220 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c003be0 con 0x555c1940a630 2024-05-11T13:49:46.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f29600700 1 -- 172.21.15.63:0/35308220 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7f9f300ad0f0 con 0x555c1940a630 2024-05-11T13:49:46.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f50d52b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 70 2024-05-11T13:49:46.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f50d52b80 10 client.16041 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:49:46.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f50d52b80 20 client.16041 populate_metadata read hostname 'smithi063' 2024-05-11T13:49:46.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19424180 0x555c19426570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f50d52b80 4 client.16041 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:49:46.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.270+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c193d7340 con 0x555c19424180 2024-05-11T13:49:46.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.271+0000 7f9f3ea00700 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19424180 0x555c19426570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.274+0000 7f9f3ea00700 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19424180 0x555c19426570 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25086d7504732c1b server_cookie=2de248278e55c448 in_seq=0 out_seq=0 2024-05-11T13:49:46.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.274+0000 7f9f29600700 10 client.16041 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:46.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.275+0000 7f9f29600700 1 -- 172.21.15.63:0/35308220 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c193d7340 con 0x555c19424180 2024-05-11T13:49:46.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.275+0000 7f9f29600700 10 client.16041 handle_command_reply: tid=0 2024-05-11T13:49:46.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.275+0000 7f9f50d52b80 1 client.16041 shutdown 2024-05-11T13:49:46.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 msgr2=0x555c194169b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c194169b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f300df680 tx=0x7f9f300dfbe0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 shutdown_connections 2024-05-11T13:49:46.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19424180 0x555c19426570 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c19410060 0x555c19414290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c194169b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/35308220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1940aff0 0x555c19413d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f29600700 0 client.16041 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:46.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.276+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 >> 172.21.15.63:0/35308220 conn(0x555c19338cf0 msgr2=0x555c193ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:46.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.277+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 shutdown_connections 2024-05-11T13:49:46.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.277+0000 7f9f50d52b80 1 -- 172.21.15.63:0/35308220 wait complete. 2024-05-11T13:49:46.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.277+0000 7f9f50d52b80 20 client.16041 trim_cache size 0 max 16384 2024-05-11T13:49:46.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.320+0000 7f9f50d52b80 1 Processor -- start 2024-05-11T13:49:46.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.320+0000 7f9f50d52b80 1 -- start start 2024-05-11T13:49:46.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.320+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c19420450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.320+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19409d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.321+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c1942be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.321+0000 7f9f50d52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2000eed0 con 0x555c1940a630 2024-05-11T13:49:46.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.321+0000 7f9f50d52b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20010ad0 con 0x555c1940aff0 2024-05-11T13:49:46.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.321+0000 7f9f50d52b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2000f5a0 con 0x555c19410060 2024-05-11T13:49:46.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.321+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19409d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.322+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19409d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:44316/0 (socket says 172.21.15.63:44316) 2024-05-11T13:49:46.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.322+0000 7f9f3ea00700 1 -- 172.21.15.63:0/844347396 learned_addr learned my addr 172.21.15.63:0/844347396 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:46.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.322+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c1942be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.323+0000 7f9f3f400700 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c19420450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.323+0000 7f9f3fe00700 1 -- 172.21.15.63:0/844347396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 msgr2=0x555c19409d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.323+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19409d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.323+0000 7f9f3fe00700 1 -- 172.21.15.63:0/844347396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 msgr2=0x555c19420450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.323+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c19420450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.323+0000 7f9f3fe00700 1 -- 172.21.15.63:0/844347396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c19426bf0 con 0x555c19410060 2024-05-11T13:49:46.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.324+0000 7f9f3f400700 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c19420450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:49:46.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.325+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c1942be40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009fd0 tx=0x7f9f38016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd7e6a5af70004d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:46.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.326+0000 7f9f2be00700 1 -- 172.21.15.63:0/844347396 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38016890 con 0x555c19410060 2024-05-11T13:49:46.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.326+0000 7f9f2be00700 1 -- 172.21.15.63:0/844347396 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f3801c030 con 0x555c19410060 2024-05-11T13:49:46.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.326+0000 7f9f2be00700 1 -- 172.21.15.63:0/844347396 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3802cd50 con 0x555c19410060 2024-05-11T13:49:46.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.327+0000 7f9f50d52b80 1 -- 172.21.15.63:0/844347396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 msgr2=0x555c1942be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.327+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c1942be40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009fd0 tx=0x7f9f38016000 comp rx=0 tx=0).stop 2024-05-11T13:49:46.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.328+0000 7f9f50d52b80 1 -- 172.21.15.63:0/844347396 shutdown_connections 2024-05-11T13:49:46.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.328+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c1942be40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.328+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940a630 0x555c19420450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.328+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/844347396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940aff0 0x555c19409d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.328+0000 7f9f50d52b80 1 -- 172.21.15.63:0/844347396 >> 172.21.15.63:0/844347396 conn(0x555c19338cf0 msgr2=0x555c19409620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:46.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.329+0000 7f9f50d52b80 1 -- 172.21.15.63:0/844347396 shutdown_connections 2024-05-11T13:49:46.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.329+0000 7f9f50d52b80 1 -- 172.21.15.63:0/844347396 wait complete. 2024-05-11T13:49:46.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.330+0000 7f9f50d52b80 1 -- start start 2024-05-11T13:49:46.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.330+0000 7f9f50d52b80 1 Processor -- start 2024-05-11T13:49:46.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.331+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940a630 0x555c1941a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.331+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940aff0 0x555c1941a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.331+0000 7f9f50d52b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c19488a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.332+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940aff0 0x555c1941a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.332+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940aff0 0x555c1941a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45788/0 (socket says 172.21.15.63:45788) 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.332+0000 7f9f3fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c19488a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.332+0000 7f9f3fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c19488a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:45720/0 (socket says 172.21.15.63:45720) 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.332+0000 7f9f3ea00700 1 -- 172.21.15.63:0/2114792824 learned_addr learned my addr 172.21.15.63:0/2114792824 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.333+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20010400 con 0x555c1940aff0 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.333+0000 7f9f3f400700 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940a630 0x555c1941a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.333+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2000fc70 con 0x555c1940a630 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.333+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2000ce90 con 0x555c19410060 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.333+0000 7f9f50d52b80 10 client.0 _ll_register_callbacks cb 0x555c19303e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:49:46.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.334+0000 7f9f3fe00700 1 -- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940a630 msgr2=0x555c1941a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.334+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940a630 0x555c1941a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.334+0000 7f9f3fe00700 1 -- 172.21.15.63:0/2114792824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940aff0 msgr2=0x555c1941a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:49:46.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.334+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940aff0 0x555c1941a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.334+0000 7f9f3fe00700 1 -- 172.21.15.63:0/2114792824 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c19489140 con 0x555c19410060 2024-05-11T13:49:46.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.334+0000 7f9f3fe00700 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c19488a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f3802c8f0 tx=0x7f9f380042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff5ea0d93de4844 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:49:46.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.334+0000 7f9f29600700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-05-11T13:49:46.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.335+0000 7f9f29600700 1 -- 172.21.15.63:0/2114792824 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3803b060 con 0x555c19410060 2024-05-11T13:49:46.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.335+0000 7f9f29600700 1 -- 172.21.15.63:0/2114792824 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f9f3802ccd0 con 0x555c19410060 2024-05-11T13:49:46.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.336+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3801c030 con 0x555c19410060 2024-05-11T13:49:46.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.336+0000 7f9f29600700 1 -- 172.21.15.63:0/2114792824 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f9f38016890 con 0x555c19410060 2024-05-11T13:49:46.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.336+0000 7f9f50d52b80 10 client.25621 fetch_fsmap learned FSMap version 70 2024-05-11T13:49:46.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.336+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x555c19490490 con 0x555c19410060 2024-05-11T13:49:46.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.337+0000 7f9f29600700 1 -- 172.21.15.63:0/2114792824 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f3801c030 con 0x555c19410060 2024-05-11T13:49:46.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.338+0000 7f9f29600700 1 -- 172.21.15.63:0/2114792824 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7f9f380369e0 con 0x555c19410060 2024-05-11T13:49:46.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.338+0000 7f9f50d52b80 10 client.25621 fetch_fsmap finished waiting for FSMap version 70 2024-05-11T13:49:46.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.338+0000 7f9f50d52b80 10 client.25621 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:49:46.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.339+0000 7f9f50d52b80 20 client.25621 populate_metadata read hostname 'smithi063' 2024-05-11T13:49:46.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.339+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19490a40 0x555c19492e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:49:46.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.339+0000 7f9f50d52b80 4 client.25621 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.9, "await_for": 42.0}] 2024-05-11T13:49:46.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.339+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.9, "await_for": 42.0}) -- 0x555c1936a590 con 0x555c19490a40 2024-05-11T13:49:46.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.340+0000 7f9f3f400700 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19490a40 0x555c19492e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:49:46.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.342+0000 7f9f3f400700 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19490a40 0x555c19492e60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0149c8b5213d79 server_cookie=54e96de97eb99fb1 in_seq=0 out_seq=0 2024-05-11T13:49:46.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.342+0000 7f9f29600700 10 client.25621 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.614+0000 7f9f29600700 1 -- 172.21.15.63:0/2114792824 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555c1936a590 con 0x555c19490a40 2024-05-11T13:49:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.614+0000 7f9f29600700 10 client.25621 handle_command_reply: tid=0 2024-05-11T13:49:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.615+0000 7f9f50d52b80 1 client.25621 shutdown 2024-05-11T13:49:46.605 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:46 smithi063 ceph-mon[29809]: pgmap v499: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-05-11T13:49:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.615+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 msgr2=0x555c19488a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.615+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c19488a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f3802c8f0 tx=0x7f9f380042c0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 shutdown_connections 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c19410060 0x555c19488a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555c19490a40 0x555c19492e60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c1940a630 0x555c1941a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 --2- 172.21.15.63:0/2114792824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c1940aff0 0x555c1941a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f29600700 0 client.25621 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 >> 172.21.15.63:0/2114792824 conn(0x555c19338cf0 msgr2=0x555c19409eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 shutdown_connections 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 1 -- 172.21.15.63:0/2114792824 wait complete. 2024-05-11T13:49:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.616+0000 7f9f50d52b80 20 client.25621 trim_cache size 0 max 16384 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 70, 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 46, 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "62bacf55": { 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 46, 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:49:46.607 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 79.9, 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:49:46.608 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:49:46.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9f240b2b20 msgr2=0x7f9f240b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9f240b2b20 0x7f9f240b4fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f380099d0 tx=0x7f9f38009050 comp rx=0 tx=0).stop 2024-05-11T13:49:46.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 msgr2=0x7f9f4019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:49:46.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f4019e600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d980 tx=0x7f9f3000dc90 comp rx=0 tx=0).stop 2024-05-11T13:49:46.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 shutdown_connections 2024-05-11T13:49:46.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f9f240b2b20 0x7f9f240b4fe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40108510 0x7f9f40197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f40103600 0x7f9f4019e600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 --2- 172.21.15.63:0/2929018941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f40103fa0 0x7f9f4019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:49:46.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 >> 172.21.15.63:0/2929018941 conn(0x7f9f4007a6b0 msgr2=0x7f9f401016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:49:46.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 shutdown_connections 2024-05-11T13:49:46.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:49:46.618+0000 7f9f46400700 1 -- 172.21.15.63:0/2929018941 wait complete. 2024-05-11T13:49:46.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:46 smithi071 ceph-mon[33293]: pgmap v499: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-05-11T13:49:46.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:46 smithi165 ceph-mon[33235]: pgmap v499: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-05-11T13:49:47.451 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '62bacf55', quiesce took 10.5 seconds. Will release after: 19.9 2024-05-11T13:49:48.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:48 smithi063 ceph-mon[29809]: pgmap v500: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-05-11T13:49:48.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:48 smithi071 ceph-mon[33293]: pgmap v500: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-05-11T13:49:48.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:48 smithi165 ceph-mon[33235]: pgmap v500: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-05-11T13:49:50.601 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:50 smithi071 ceph-mon[33293]: pgmap v501: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-05-11T13:49:50.601 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:50 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:50.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:50 smithi063 ceph-mon[29809]: pgmap v501: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-05-11T13:49:50.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:50 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:50.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:50 smithi165 ceph-mon[33235]: pgmap v501: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-05-11T13:49:50.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:50 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:49:51.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 505.0 s, 19.6 tps, lat 222.152 ms stddev 43.822 2024-05-11T13:49:52.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:52 smithi063 ceph-mon[29809]: pgmap v502: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-05-11T13:49:52.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:52 smithi071 ceph-mon[33293]: pgmap v502: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-05-11T13:49:52.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:52 smithi165 ceph-mon[33235]: pgmap v502: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-05-11T13:49:54.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:54 smithi063 ceph-mon[29809]: pgmap v503: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-05-11T13:49:54.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:54 smithi071 ceph-mon[33293]: pgmap v503: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-05-11T13:49:54.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:54 smithi165 ceph-mon[33235]: pgmap v503: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-05-11T13:49:56.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 510.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:49:56.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:56 smithi063 ceph-mon[29809]: pgmap v504: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 3.5 MiB/s wr, 326 op/s 2024-05-11T13:49:56.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:56 smithi071 ceph-mon[33293]: pgmap v504: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 3.5 MiB/s wr, 326 op/s 2024-05-11T13:49:56.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:56 smithi165 ceph-mon[33235]: pgmap v504: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 3.5 MiB/s wr, 326 op/s 2024-05-11T13:49:57.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:57 smithi063 ceph-mon[29809]: pgmap v505: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-05-11T13:49:57.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:57 smithi071 ceph-mon[33293]: pgmap v505: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-05-11T13:49:57.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:57 smithi165 ceph-mon[33235]: pgmap v505: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-05-11T13:49:59.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:49:59 smithi165 ceph-mon[33235]: pgmap v506: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-05-11T13:49:59.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:49:59 smithi063 ceph-mon[29809]: pgmap v506: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-05-11T13:49:59.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:49:59 smithi071 ceph-mon[33293]: pgmap v506: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-05-11T13:50:00.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:00 smithi165 ceph-mon[33235]: overall HEALTH_OK 2024-05-11T13:50:00.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:00 smithi063 ceph-mon[29809]: overall HEALTH_OK 2024-05-11T13:50:00.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:00 smithi071 ceph-mon[33293]: overall HEALTH_OK 2024-05-11T13:50:01.145 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 515.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:50:01.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:01 smithi165 ceph-mon[33235]: pgmap v507: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 191 KiB/s wr, 31 op/s 2024-05-11T13:50:01.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:01 smithi063 ceph-mon[29809]: pgmap v507: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 191 KiB/s wr, 31 op/s 2024-05-11T13:50:01.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:01 smithi071 ceph-mon[33293]: pgmap v507: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 191 KiB/s wr, 31 op/s 2024-05-11T13:50:03.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:03 smithi165 ceph-mon[33235]: pgmap v508: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 163 KiB/s wr, 27 op/s 2024-05-11T13:50:03.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:03 smithi063 ceph-mon[29809]: pgmap v508: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 163 KiB/s wr, 27 op/s 2024-05-11T13:50:03.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:03 smithi071 ceph-mon[33293]: pgmap v508: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 163 KiB/s wr, 27 op/s 2024-05-11T13:50:05.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:05 smithi165 ceph-mon[33235]: pgmap v509: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 315 KiB/s wr, 62 op/s 2024-05-11T13:50:05.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:05 smithi063 ceph-mon[29809]: pgmap v509: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 315 KiB/s wr, 62 op/s 2024-05-11T13:50:05.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:05 smithi071 ceph-mon[33293]: pgmap v509: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 315 KiB/s wr, 62 op/s 2024-05-11T13:50:06.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 520.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:50:07.353 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:50:07.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:07 smithi165 ceph-mon[33235]: pgmap v510: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 235 KiB/s wr, 48 op/s 2024-05-11T13:50:07.847 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:50:07.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:07 smithi063 ceph-mon[29809]: pgmap v510: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 235 KiB/s wr, 48 op/s 2024-05-11T13:50:07.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:07 smithi071 ceph-mon[33293]: pgmap v510: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 235 KiB/s wr, 48 op/s 2024-05-11T13:50:09.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:09 smithi063 ceph-mon[29809]: pgmap v511: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 354 KiB/s wr, 66 op/s 2024-05-11T13:50:09.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:09 smithi165 ceph-mon[33235]: pgmap v511: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 354 KiB/s wr, 66 op/s 2024-05-11T13:50:09.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:09 smithi071 ceph-mon[33293]: pgmap v511: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 354 KiB/s wr, 66 op/s 2024-05-11T13:50:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.227+0000 7f62d7e00700 1 -- 172.21.15.63:0/481610072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 msgr2=0x7f62d8103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.227+0000 7f62d7e00700 1 --2- 172.21.15.63:0/481610072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8103920 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f62c00099c0 tx=0x7f62c0009cd0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.228+0000 7f62d7e00700 1 -- 172.21.15.63:0/481610072 shutdown_connections 2024-05-11T13:50:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.228+0000 7f62d7e00700 1 --2- 172.21.15.63:0/481610072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62d8103e60 0x7f62d8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.228+0000 7f62d7e00700 1 --2- 172.21.15.63:0/481610072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62d8109550 0x7f62d8109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.228+0000 7f62d7e00700 1 --2- 172.21.15.63:0/481610072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8103920 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.228+0000 7f62d7e00700 1 -- 172.21.15.63:0/481610072 >> 172.21.15.63:0/481610072 conn(0x7f62d80fefd0 msgr2=0x7f62d81013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.228+0000 7f62d7e00700 1 -- 172.21.15.63:0/481610072 shutdown_connections 2024-05-11T13:50:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.228+0000 7f62d7e00700 1 -- 172.21.15.63:0/481610072 wait complete. 2024-05-11T13:50:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.229+0000 7f62d7e00700 1 Processor -- start 2024-05-11T13:50:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.229+0000 7f62d7e00700 1 -- start start 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62d8103e60 0x7f62d8075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62d8109550 0x7f62d80762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81135b0 con 0x7f62d81034a0 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d7e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8113430 con 0x7f62d8109550 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d7e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8113730 con 0x7f62d8103e60 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46912/0 (socket says 172.21.15.63:46912) 2024-05-11T13:50:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d6a00700 1 -- 172.21.15.63:0/246364720 learned_addr learned my addr 172.21.15.63:0/246364720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62cfe00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62d8103e60 0x7f62d8075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d6a00700 1 -- 172.21.15.63:0/246364720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62d8103e60 msgr2=0x7f62d8075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d7400700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62d8109550 0x7f62d80762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.230+0000 7f62d6a00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62d8103e60 0x7f62d8075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62d6a00700 1 -- 172.21.15.63:0/246364720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62d8109550 msgr2=0x7f62d80762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62d6a00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62d8109550 0x7f62d80762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62d6a00700 1 -- 172.21.15.63:0/246364720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d81a7f90 con 0x7f62d81034a0 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62cfe00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62d8103e60 0x7f62d8075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62d7400700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62d8109550 0x7f62d80762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:50:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62d6a00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8075850 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c920 tx=0x7f62c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:10.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62dca00700 1 -- 172.21.15.63:0/246364720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8007b00 con 0x7f62d81034a0 2024-05-11T13:50:10.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62dca00700 1 -- 172.21.15.63:0/246364720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62c8004cc0 con 0x7f62d81034a0 2024-05-11T13:50:10.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d81a8280 con 0x7f62d81034a0 2024-05-11T13:50:10.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62dca00700 1 -- 172.21.15.63:0/246364720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c80054f0 con 0x7f62d81034a0 2024-05-11T13:50:10.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.231+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d81a87c0 con 0x7f62d81034a0 2024-05-11T13:50:10.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.232+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d806a240 con 0x7f62d81034a0 2024-05-11T13:50:10.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.236+0000 7f62dca00700 1 -- 172.21.15.63:0/246364720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f62c8007ca0 con 0x7f62d81034a0 2024-05-11T13:50:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.238+0000 7f62dca00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f62bc0b29f0 0x7f62bc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.238+0000 7f62dca00700 1 -- 172.21.15.63:0/246364720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9613+0+0 (secure 0 0 0) 0x7f62c80e2b10 con 0x7f62d81034a0 2024-05-11T13:50:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.238+0000 7f62cfe00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f62bc0b29f0 0x7f62bc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:10.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.239+0000 7f62cfe00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f62bc0b29f0 0x7f62bc0b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c0009f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:10.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.241+0000 7f62dca00700 1 -- 172.21.15.63:0/246364720 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62c8014030 con 0x7f62d81034a0 2024-05-11T13:50:10.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.527+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62d802e570 con 0x7f62d81034a0 2024-05-11T13:50:10.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.529+0000 7f62dca00700 1 -- 172.21.15.63:0/246364720 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11949 (secure 0 0 0) 0x7f62c80aa050 con 0x7f62d81034a0 2024-05-11T13:50:10.519 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:50:10.519 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:49:48.852327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34451,34472,34427,34403,24472]},"id":1}]} 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.532+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f62bc0b29f0 msgr2=0x7f62bc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.532+0000 7f62d7e00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f62bc0b29f0 0x7f62bc0b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c0009f90 comp rx=0 tx=0).stop 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 msgr2=0x7f62d8075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8075850 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c920 tx=0x7f62c800cce0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 shutdown_connections 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f62bc0b29f0 0x7f62bc0b4eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62d8109550 0x7f62d80762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62d81034a0 0x7f62d8075850 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 --2- 172.21.15.63:0/246364720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62d8103e60 0x7f62d8075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:10.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 >> 172.21.15.63:0/246364720 conn(0x7f62d80fefd0 msgr2=0x7f62d8100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:10.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 shutdown_connections 2024-05-11T13:50:10.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:10.533+0000 7f62d7e00700 1 -- 172.21.15.63:0/246364720 wait complete. 2024-05-11T13:50:10.524 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 71 2024-05-11T13:50:10.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:10 smithi165 ceph-mon[33235]: osdmap e112: 12 total, 12 up, 12 in 2024-05-11T13:50:10.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:10 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/246364720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:50:10.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:10 smithi071 ceph-mon[33293]: osdmap e112: 12 total, 12 up, 12 in 2024-05-11T13:50:10.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:10 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/246364720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:50:10.901 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:10 smithi063 ceph-mon[29809]: osdmap e112: 12 total, 12 up, 12 in 2024-05-11T13:50:10.902 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:10 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/246364720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:50:11.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 525.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:50:11.331 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 62bacf55 --release --await' 2024-05-11T13:50:11.331 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 62bacf55 --release --await 2024-05-11T13:50:11.784 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:50:11.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:11 smithi165 ceph-mon[33235]: pgmap v513: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 324 KiB/s wr, 62 op/s 2024-05-11T13:50:11.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:11 smithi063 ceph-mon[29809]: pgmap v513: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 324 KiB/s wr, 62 op/s 2024-05-11T13:50:11.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:11 smithi071 ceph-mon[33293]: pgmap v513: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 324 KiB/s wr, 62 op/s 2024-05-11T13:50:13.819 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:13 smithi063 ceph-mon[29809]: pgmap v514: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 324 KiB/s wr, 62 op/s 2024-05-11T13:50:13.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:13 smithi071 ceph-mon[33293]: pgmap v514: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 324 KiB/s wr, 62 op/s 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 -- 172.21.15.63:0/23831130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f210410ff80 msgr2=0x7f2104112370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 --2- 172.21.15.63:0/23831130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f210410ff80 0x7f2104112370 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009990 tx=0x7f20fc009ca0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 -- 172.21.15.63:0/23831130 shutdown_connections 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 --2- 172.21.15.63:0/23831130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f210410ff80 0x7f2104112370 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 --2- 172.21.15.63:0/23831130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2104075780 0x7f2104075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 --2- 172.21.15.63:0/23831130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21040760a0 0x7f210410fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 -- 172.21.15.63:0/23831130 >> 172.21.15.63:0/23831130 conn(0x7f210407a700 msgr2=0x7f210407ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 -- 172.21.15.63:0/23831130 shutdown_connections 2024-05-11T13:50:14.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.016+0000 7f210a000700 1 -- 172.21.15.63:0/23831130 wait complete. 2024-05-11T13:50:14.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.018+0000 7f210a000700 1 Processor -- start 2024-05-11T13:50:14.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.018+0000 7f210a000700 1 -- start start 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f210a000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2104075780 0x7f210419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f210a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21040760a0 0x7f210419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f210a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f210410ff80 0x7f21041a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f210a000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2104115870 con 0x7f21040760a0 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f210a000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21041156f0 con 0x7f210410ff80 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f210a000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21041159f0 con 0x7f2104075780 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f20faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21040760a0 0x7f210419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f20faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21040760a0 0x7f210419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46934/0 (socket says 172.21.15.63:46934) 2024-05-11T13:50:14.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f20faa00700 1 -- 172.21.15.63:0/4093783302 learned_addr learned my addr 172.21.15.63:0/4093783302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:50:14.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f2103e00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f210410ff80 0x7f21041a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.019+0000 7f2103400700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2104075780 0x7f210419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f20faa00700 1 -- 172.21.15.63:0/4093783302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2104075780 msgr2=0x7f210419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f20faa00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2104075780 0x7f210419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f20faa00700 1 -- 172.21.15.63:0/4093783302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f210410ff80 msgr2=0x7f21041a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f20faa00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f210410ff80 0x7f21041a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f20faa00700 1 -- 172.21.15.63:0/4093783302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21041a0d20 con 0x7f21040760a0 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f2103e00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f210410ff80 0x7f21041a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f20faa00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21040760a0 0x7f210419c2a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d950 tx=0x7f20ec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f2100c00700 1 -- 172.21.15.63:0/4093783302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec016070 con 0x7f21040760a0 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f2100c00700 1 -- 172.21.15.63:0/4093783302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20ec010460 con 0x7f21040760a0 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f2100c00700 1 -- 172.21.15.63:0/4093783302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00f620 con 0x7f21040760a0 2024-05-11T13:50:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21041a1010 con 0x7f21040760a0 2024-05-11T13:50:14.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.020+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041a8890 con 0x7f21040760a0 2024-05-11T13:50:14.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.024+0000 7f2100c00700 1 -- 172.21.15.63:0/4093783302 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f20ec019070 con 0x7f21040760a0 2024-05-11T13:50:14.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.025+0000 7f2114929b80 1 Processor -- start 2024-05-11T13:50:14.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.025+0000 7f2114929b80 1 -- start start 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2100c00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f20e80b29f0 0x7f20e80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2114929b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9874580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2114929b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe986e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2114929b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe986ecc0 0x555fe986f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2114929b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555fe97c0b20 con 0x555fe986ecc0 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2114929b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fe975e860 con 0x555fe98741a0 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2114929b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555fe97bcae0 con 0x555fe9874ac0 2024-05-11T13:50:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2100c00700 1 -- 172.21.15.63:0/4093783302 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9613+0+0 (secure 0 0 0) 0x7f20ec0e2680 con 0x7f21040760a0 2024-05-11T13:50:14.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.026+0000 7f2103e00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f20e80b29f0 0x7f20e80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe986ecc0 0x555fe986f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9874580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9874580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:38488/0 (socket says 172.21.15.63:38488) 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 -- 172.21.15.63:0/3995664375 learned_addr learned my addr 172.21.15.63:0/3995664375 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 -- 172.21.15.63:0/3995664375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 msgr2=0x555fe986e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe986e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 -- 172.21.15.63:0/3995664375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe986ecc0 msgr2=0x555fe986f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe986ecc0 0x555fe986f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103400700 1 -- 172.21.15.63:0/3995664375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe970d9d0 con 0x555fe98741a0 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.027+0000 7f2103e00700 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe986ecc0 0x555fe986f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.028+0000 7f2103e00700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f20e80b29f0 0x7f20e80b4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005f50 tx=0x7f20fc0095d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.028+0000 7f2103400700 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9874580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20f400e210 tx=0x7f20f400e520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08bda57d92b8bd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:14.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.028+0000 7f20f8c00700 1 -- 172.21.15.63:0/3995664375 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f400edb0 con 0x555fe98741a0 2024-05-11T13:50:14.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.028+0000 7f20f8c00700 1 -- 172.21.15.63:0/3995664375 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20f4004bb0 con 0x555fe98741a0 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.028+0000 7f20f8c00700 1 -- 172.21.15.63:0/3995664375 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40058e0 con 0x555fe98741a0 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 -- 172.21.15.63:0/3995664375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 msgr2=0x555fe9874580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9874580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20f400e210 tx=0x7f20f400e520 comp rx=0 tx=0).stop 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 -- 172.21.15.63:0/3995664375 shutdown_connections 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe986ecc0 0x555fe986f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9874580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 --2- 172.21.15.63:0/3995664375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe986e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 -- 172.21.15.63:0/3995664375 >> 172.21.15.63:0/3995664375 conn(0x555fe979cc60 msgr2=0x555fe98692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.029+0000 7f2114929b80 1 -- 172.21.15.63:0/3995664375 shutdown_connections 2024-05-11T13:50:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.030+0000 7f2114929b80 1 -- 172.21.15.63:0/3995664375 wait complete. 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.030+0000 7f2114929b80 1 -- start start 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.030+0000 7f2114929b80 1 Processor -- start 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2114929b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe987a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2114929b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe98741a0 0x555fe9877ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2114929b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe98781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2114929b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555fe987b5e0 con 0x555fe98741a0 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2114929b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fe9726c20 con 0x555fe986ecc0 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2114929b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555fe987b760 con 0x555fe9874ac0 2024-05-11T13:50:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2114929b80 10 client.0 _ll_register_callbacks cb 0x555fe9754480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f20faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe98741a0 0x555fe9877ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f20faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe98741a0 0x555fe9877ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46950/0 (socket says 172.21.15.63:46950) 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f20faa00700 1 -- 172.21.15.63:0/784703223 learned_addr learned my addr 172.21.15.63:0/784703223 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2103400700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe987a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.031+0000 7f2103e00700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe98781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20faa00700 1 -- 172.21.15.63:0/784703223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 msgr2=0x555fe98781e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20faa00700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe98781e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20faa00700 1 -- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 msgr2=0x555fe987a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20faa00700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe987a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20faa00700 1 -- 172.21.15.63:0/784703223 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe9786520 con 0x555fe98741a0 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f2103e00700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe98781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20faa00700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe98741a0 0x555fe9877ca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0a9450 tx=0x7f20ec0dfe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88a1b8fc40a163b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20f1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:50:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20f1600700 1 -- 172.21.15.63:0/784703223 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec020070 con 0x555fe98741a0 2024-05-11T13:50:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f20f1600700 1 -- 172.21.15.63:0/784703223 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9613+0+0 (secure 0 0 0) 0x7f20ec0a3df0 con 0x555fe98741a0 2024-05-11T13:50:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.032+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f4004bb0 con 0x555fe98741a0 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f20f1600700 1 -- 172.21.15.63:0/784703223 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f20ec0a9bc0 con 0x555fe98741a0 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 10 client.16062 fetch_fsmap learned FSMap version 71 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x555fe9885fe0 con 0x555fe98741a0 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f20f1600700 1 -- 172.21.15.63:0/784703223 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20f4004bb0 con 0x555fe98741a0 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f20f1600700 1 -- 172.21.15.63:0/784703223 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 71) ==== 8394+0+0 (secure 0 0 0) 0x7f20ec0b1e90 con 0x555fe98741a0 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 10 client.16062 fetch_fsmap finished waiting for FSMap version 71 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 10 client.16062 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:50:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 20 client.16062 populate_metadata read hostname 'smithi063' 2024-05-11T13:50:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe9888620 0x555fe988aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 4 client.16062 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:50:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.033+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fe983b160 con 0x555fe9888620 2024-05-11T13:50:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.034+0000 7f2103400700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe9888620 0x555fe988aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.035+0000 7f2103400700 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe9888620 0x555fe988aa40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f05aabd3e046fe1 server_cookie=416da79206736fea in_seq=0 out_seq=0 2024-05-11T13:50:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.035+0000 7f20f1600700 10 client.16062 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:50:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f20f1600700 1 -- 172.21.15.63:0/784703223 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555fe983b160 con 0x555fe9888620 2024-05-11T13:50:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f20f1600700 10 client.16062 handle_command_reply: tid=0 2024-05-11T13:50:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 client.16062 shutdown 2024-05-11T13:50:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe98741a0 msgr2=0x555fe9877ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe98741a0 0x555fe9877ca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0a9450 tx=0x7f20ec0dfe60 comp rx=0 tx=0).stop 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 shutdown_connections 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe9874ac0 0x555fe98781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe9888620 0x555fe988aa40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe987a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 --2- 172.21.15.63:0/784703223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe98741a0 0x555fe9877ca0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 >> 172.21.15.63:0/784703223 conn(0x555fe979cc60 msgr2=0x555fe986d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f20f1600700 0 client.16062 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 shutdown_connections 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.037+0000 7f2114929b80 1 -- 172.21.15.63:0/784703223 wait complete. 2024-05-11T13:50:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.038+0000 7f2114929b80 20 client.16062 trim_cache size 0 max 16384 2024-05-11T13:50:14.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.059+0000 7f2114929b80 1 Processor -- start 2024-05-11T13:50:14.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2114929b80 1 -- start start 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2114929b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe986ecc0 0x555fe986e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2114929b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9884670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2114929b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe98734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2114929b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21041b1890 con 0x555fe9874ac0 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2114929b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21041b3490 con 0x555fe98741a0 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2114929b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21041b1f60 con 0x555fe986ecc0 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2103e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe98734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2103e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe98734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46960/0 (socket says 172.21.15.63:46960) 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.060+0000 7f2103e00700 1 -- 172.21.15.63:0/4051852151 learned_addr learned my addr 172.21.15.63:0/4051852151 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f2103400700 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe986ecc0 0x555fe986e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f20faa00700 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9884670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f2103e00700 1 -- 172.21.15.63:0/4051852151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe986ecc0 msgr2=0x555fe986e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f2103e00700 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe986ecc0 0x555fe986e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f2103e00700 1 -- 172.21.15.63:0/4051852151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 msgr2=0x555fe9884670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f2103e00700 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9884670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f2103e00700 1 -- 172.21.15.63:0/4051852151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe9884bb0 con 0x555fe9874ac0 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.061+0000 7f2103e00700 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe98734c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00ba60 tx=0x7f20fc02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862edfd8461a919e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f20f3e00700 1 -- 172.21.15.63:0/4051852151 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc036070 con 0x555fe9874ac0 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f20f3e00700 1 -- 172.21.15.63:0/4051852151 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20fc0054d0 con 0x555fe9874ac0 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f20f3e00700 1 -- 172.21.15.63:0/4051852151 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc021040 con 0x555fe9874ac0 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 -- 172.21.15.63:0/4051852151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 msgr2=0x555fe98734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe98734c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00ba60 tx=0x7f20fc02a000 comp rx=0 tx=0).stop 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 -- 172.21.15.63:0/4051852151 shutdown_connections 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe98734c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe986ecc0 0x555fe986e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 --2- 172.21.15.63:0/4051852151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe98741a0 0x555fe9884670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 -- 172.21.15.63:0/4051852151 >> 172.21.15.63:0/4051852151 conn(0x555fe979cc60 msgr2=0x555fe9876cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:14.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.062+0000 7f2114929b80 1 -- 172.21.15.63:0/4051852151 shutdown_connections 2024-05-11T13:50:14.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.063+0000 7f2114929b80 1 -- 172.21.15.63:0/4051852151 wait complete. 2024-05-11T13:50:14.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.063+0000 7f2114929b80 1 -- start start 2024-05-11T13:50:14.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.063+0000 7f2114929b80 1 Processor -- start 2024-05-11T13:50:14.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2114929b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe9880bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2114929b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe98741a0 0x555fe9881110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2114929b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe987dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2114929b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21041b2dc0 con 0x555fe9874ac0 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2114929b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21041b2630 con 0x555fe986ecc0 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2114929b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21041af850 con 0x555fe98741a0 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2114929b80 10 client.0 _ll_register_callbacks cb 0x555fe9754480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2103e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe987dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2103e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe987dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46976/0 (socket says 172.21.15.63:46976) 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2103e00700 1 -- 172.21.15.63:0/3497580061 learned_addr learned my addr 172.21.15.63:0/3497580061 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f2103400700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe9880bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.064+0000 7f20faa00700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe98741a0 0x555fe9881110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f2103e00700 1 -- 172.21.15.63:0/3497580061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe98741a0 msgr2=0x555fe9881110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f2103e00700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe98741a0 0x555fe9881110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f2103e00700 1 -- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 msgr2=0x555fe9880bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:50:14.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f2103e00700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe9880bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f2103e00700 1 -- 172.21.15.63:0/3497580061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe987e2c0 con 0x555fe9874ac0 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f20faa00700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe98741a0 0x555fe9881110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f2103400700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe9880bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f2103e00700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe987dab0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f20fc006010 tx=0x7f20fc009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc63efd0d8b5491 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f20f1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f20f1600700 1 -- 172.21.15.63:0/3497580061 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc036040 con 0x555fe9874ac0 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.065+0000 7f20f1600700 1 -- 172.21.15.63:0/3497580061 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9613+0+0 (secure 0 0 0) 0x7f20fc00f460 con 0x555fe9874ac0 2024-05-11T13:50:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.066+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20fc0054d0 con 0x555fe9874ac0 2024-05-11T13:50:14.058 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:13 smithi165 ceph-mon[33235]: pgmap v514: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 324 KiB/s wr, 62 op/s 2024-05-11T13:50:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.066+0000 7f20f1600700 1 -- 172.21.15.63:0/3497580061 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f20fc04a050 con 0x555fe9874ac0 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.066+0000 7f2114929b80 10 client.16068 fetch_fsmap learned FSMap version 71 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.066+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x555fe98f3ad0 con 0x555fe9874ac0 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.066+0000 7f20f1600700 1 -- 172.21.15.63:0/3497580061 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20fc0054d0 con 0x555fe9874ac0 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.066+0000 7f20f1600700 1 -- 172.21.15.63:0/3497580061 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 71) ==== 8394+0+0 (secure 0 0 0) 0x7f20fc039030 con 0x555fe9874ac0 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.067+0000 7f2114929b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 71 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.067+0000 7f2114929b80 10 client.16068 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.067+0000 7f2114929b80 20 client.16068 populate_metadata read hostname 'smithi063' 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.067+0000 7f2114929b80 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe98f40f0 0x555fe98f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.067+0000 7f2114929b80 4 client.16068 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "62bacf55", "release": true, "await": true}] 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.067+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "62bacf55", "release": true, "await": true}) -- 0x555fe97ce500 con 0x555fe98f40f0 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.067+0000 7f2103400700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe98f40f0 0x555fe98f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.068+0000 7f2103400700 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe98f40f0 0x555fe98f6510 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36340e784e74933b server_cookie=5ec69d4007e2ddf6 in_seq=0 out_seq=0 2024-05-11T13:50:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.068+0000 7f20f1600700 10 client.16068 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:50:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.142+0000 7f20f1600700 1 -- 172.21.15.63:0/3497580061 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555fe97ce500 con 0x555fe98f40f0 2024-05-11T13:50:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.142+0000 7f20f1600700 10 client.16068 handle_command_reply: tid=0 2024-05-11T13:50:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.143+0000 7f2114929b80 1 client.16068 shutdown 2024-05-11T13:50:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.143+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 msgr2=0x555fe987dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.143+0000 7f2114929b80 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe987dab0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f20fc006010 tx=0x7f20fc009380 comp rx=0 tx=0).stop 2024-05-11T13:50:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 shutdown_connections 2024-05-11T13:50:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f2114929b80 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555fe9874ac0 0x555fe987dab0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f2114929b80 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x555fe98f40f0 0x555fe98f6510 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f2114929b80 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fe986ecc0 0x555fe9880bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f2114929b80 1 --2- 172.21.15.63:0/3497580061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555fe98741a0 0x555fe9881110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 >> 172.21.15.63:0/3497580061 conn(0x555fe979cc60 msgr2=0x555fe9879b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f20f1600700 0 client.16068 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:50:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.144+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 shutdown_connections 2024-05-11T13:50:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.145+0000 7f2114929b80 1 -- 172.21.15.63:0/3497580061 wait complete. 2024-05-11T13:50:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.145+0000 7f2114929b80 20 client.16068 trim_cache size 0 max 16384 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 71, 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 48, 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "62bacf55": { 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 48, 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:50:14.137 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 79.9, 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:50:14.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:50:14.139 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:50:14.139 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:50:14.139 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:50:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.147+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f20e80b29f0 msgr2=0x7f20e80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.147+0000 7f210a000700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f20e80b29f0 0x7f20e80b4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005f50 tx=0x7f20fc0095d0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.147+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21040760a0 msgr2=0x7f210419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.147+0000 7f210a000700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21040760a0 0x7f210419c2a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d950 tx=0x7f20ec00dd10 comp rx=0 tx=0).stop 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 shutdown_connections 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f20e80b29f0 0x7f20e80b4eb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f210410ff80 0x7f21041a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2104075780 0x7f210419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 --2- 172.21.15.63:0/4093783302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21040760a0 0x7f210419c2a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 >> 172.21.15.63:0/4093783302 conn(0x7f210407a700 msgr2=0x7f2104100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 shutdown_connections 2024-05-11T13:50:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:50:14.148+0000 7f210a000700 1 -- 172.21.15.63:0/4093783302 wait complete. 2024-05-11T13:50:15.016 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '62bacf55', seconds elapsed: 38.1 2024-05-11T13:50:16.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 530.0 s, 213.4 tps, lat 2790.561 ms stddev 8085.679 2024-05-11T13:50:16.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:16 smithi165 ceph-mon[33235]: pgmap v515: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 251 KiB/s wr, 37 op/s 2024-05-11T13:50:16.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:16 smithi165 ceph-mon[33235]: osdmap e113: 12 total, 12 up, 12 in 2024-05-11T13:50:16.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:16 smithi063 ceph-mon[29809]: pgmap v515: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 251 KiB/s wr, 37 op/s 2024-05-11T13:50:16.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:16 smithi063 ceph-mon[29809]: osdmap e113: 12 total, 12 up, 12 in 2024-05-11T13:50:16.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:16 smithi071 ceph-mon[33293]: pgmap v515: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 251 KiB/s wr, 37 op/s 2024-05-11T13:50:16.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:16 smithi071 ceph-mon[33293]: osdmap e113: 12 total, 12 up, 12 in 2024-05-11T13:50:18.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:18 smithi165 ceph-mon[33235]: pgmap v517: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 136 KiB/s wr, 19 op/s 2024-05-11T13:50:18.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:18 smithi063 ceph-mon[29809]: pgmap v517: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 136 KiB/s wr, 19 op/s 2024-05-11T13:50:18.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:18 smithi071 ceph-mon[33293]: pgmap v517: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 136 KiB/s wr, 19 op/s 2024-05-11T13:50:20.558 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:20 smithi165 ceph-mon[33235]: pgmap v518: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-11T13:50:20.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:20 smithi063 ceph-mon[29809]: pgmap v518: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-11T13:50:20.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:20 smithi071 ceph-mon[33293]: pgmap v518: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-11T13:50:21.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 535.0 s, 413.0 tps, lat 243.565 ms stddev 45.454 2024-05-11T13:50:22.559 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:22 smithi165 ceph-mon[33235]: pgmap v519: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-05-11T13:50:22.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:22 smithi063 ceph-mon[29809]: pgmap v519: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-05-11T13:50:22.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:22 smithi071 ceph-mon[33293]: pgmap v519: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-05-11T13:50:24.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:24 smithi165 ceph-mon[33235]: pgmap v520: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-05-11T13:50:24.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:24 smithi063 ceph-mon[29809]: pgmap v520: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-05-11T13:50:24.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:24 smithi071 ceph-mon[33293]: pgmap v520: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-05-11T13:50:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 540.0 s, 430.8 tps, lat 230.103 ms stddev 35.724 2024-05-11T13:50:26.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:26 smithi165 ceph-mon[33235]: pgmap v521: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 419 op/s 2024-05-11T13:50:26.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:26 smithi063 ceph-mon[29809]: pgmap v521: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 419 op/s 2024-05-11T13:50:26.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:26 smithi071 ceph-mon[33293]: pgmap v521: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 419 op/s 2024-05-11T13:50:28.569 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:28 smithi165 ceph-mon[33235]: pgmap v522: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-05-11T13:50:28.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:28 smithi063 ceph-mon[29809]: pgmap v522: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-05-11T13:50:28.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:28 smithi071 ceph-mon[33293]: pgmap v522: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-05-11T13:50:30.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:30 smithi165 ceph-mon[33235]: pgmap v523: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.1 MiB/s wr, 568 op/s 2024-05-11T13:50:30.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:30 smithi063 ceph-mon[29809]: pgmap v523: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.1 MiB/s wr, 568 op/s 2024-05-11T13:50:30.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:30 smithi071 ceph-mon[33293]: pgmap v523: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.1 MiB/s wr, 568 op/s 2024-05-11T13:50:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 545.0 s, 339.3 tps, lat 229.855 ms stddev 49.538 2024-05-11T13:50:32.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:32 smithi165 ceph-mon[33235]: pgmap v524: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-05-11T13:50:32.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:32 smithi063 ceph-mon[29809]: pgmap v524: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-05-11T13:50:32.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:32 smithi071 ceph-mon[33293]: pgmap v524: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-05-11T13:50:34.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:34 smithi165 ceph-mon[33235]: pgmap v525: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.7 MiB/s wr, 446 op/s 2024-05-11T13:50:34.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:34 smithi063 ceph-mon[29809]: pgmap v525: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.7 MiB/s wr, 446 op/s 2024-05-11T13:50:34.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:34 smithi071 ceph-mon[33293]: pgmap v525: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.7 MiB/s wr, 446 op/s 2024-05-11T13:50:36.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 550.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:50:36.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:36 smithi165 ceph-mon[33235]: pgmap v526: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.8 MiB/s wr, 568 op/s 2024-05-11T13:50:36.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:36 smithi063 ceph-mon[29809]: pgmap v526: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.8 MiB/s wr, 568 op/s 2024-05-11T13:50:36.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:36 smithi071 ceph-mon[33293]: pgmap v526: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.8 MiB/s wr, 568 op/s 2024-05-11T13:50:38.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:38 smithi063 ceph-mon[29809]: pgmap v527: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.6 MiB/s wr, 348 op/s 2024-05-11T13:50:38.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:38 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:50:38.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:38 smithi071 ceph-mon[33293]: pgmap v527: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.6 MiB/s wr, 348 op/s 2024-05-11T13:50:38.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:38 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:50:38.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:38 smithi165 ceph-mon[33235]: pgmap v527: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.6 MiB/s wr, 348 op/s 2024-05-11T13:50:38.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:38 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:50:40.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:39 smithi165 ceph-mon[33235]: pgmap v528: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-05-11T13:50:40.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:39 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:39 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:39 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:39 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:39 smithi063 ceph-mon[29809]: pgmap v528: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-05-11T13:50:40.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:39 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:39 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:39 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:39 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:39 smithi071 ceph-mon[33293]: pgmap v528: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-05-11T13:50:40.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:39 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:39 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:39 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:40.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:39 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:41.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 555.0 s, 137.3 tps, lat 1614.510 ms stddev 3318.704 2024-05-11T13:50:41.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:41 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:50:41.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:41 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:50:41.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:41 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:41.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:41 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:50:41.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:41 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:50:41.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:41 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:41.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:41 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:50:41.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:41 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:50:41.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:41 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:50:42.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:42 smithi165 ceph-mon[33235]: pgmap v529: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-11T13:50:42.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:42 smithi063 ceph-mon[29809]: pgmap v529: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-11T13:50:42.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:42 smithi071 ceph-mon[33293]: pgmap v529: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-11T13:50:44.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:44 smithi165 ceph-mon[33235]: pgmap v530: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-05-11T13:50:44.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:44 smithi063 ceph-mon[29809]: pgmap v530: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-05-11T13:50:44.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:44 smithi071 ceph-mon[33293]: pgmap v530: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-05-11T13:50:46.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 560.0 s, 421.6 tps, lat 237.898 ms stddev 59.309 2024-05-11T13:50:46.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:46 smithi165 ceph-mon[33235]: pgmap v531: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-11T13:50:46.362 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:46 smithi063 ceph-mon[29809]: pgmap v531: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-11T13:50:46.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:46 smithi071 ceph-mon[33293]: pgmap v531: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-11T13:50:48.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:48 smithi165 ceph-mon[33235]: pgmap v532: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-05-11T13:50:48.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:48 smithi063 ceph-mon[29809]: pgmap v532: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-05-11T13:50:48.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:48 smithi071 ceph-mon[33293]: pgmap v532: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-05-11T13:50:50.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:50 smithi165 ceph-mon[33235]: pgmap v533: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 6.0 MiB/s wr, 418 op/s 2024-05-11T13:50:50.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:50 smithi063 ceph-mon[29809]: pgmap v533: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 6.0 MiB/s wr, 418 op/s 2024-05-11T13:50:50.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:50 smithi071 ceph-mon[33293]: pgmap v533: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 6.0 MiB/s wr, 418 op/s 2024-05-11T13:50:51.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 565.0 s, 405.4 tps, lat 248.534 ms stddev 78.274 2024-05-11T13:50:52.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:52 smithi165 ceph-mon[33235]: pgmap v534: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.0 MiB/s wr, 384 op/s 2024-05-11T13:50:52.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:52 smithi063 ceph-mon[29809]: pgmap v534: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.0 MiB/s wr, 384 op/s 2024-05-11T13:50:52.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:52 smithi071 ceph-mon[33293]: pgmap v534: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.0 MiB/s wr, 384 op/s 2024-05-11T13:50:54.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:54 smithi165 ceph-mon[33235]: pgmap v535: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.0 MiB/s wr, 381 op/s 2024-05-11T13:50:54.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:54 smithi063 ceph-mon[29809]: pgmap v535: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.0 MiB/s wr, 381 op/s 2024-05-11T13:50:54.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:54 smithi071 ceph-mon[33293]: pgmap v535: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.0 MiB/s wr, 381 op/s 2024-05-11T13:50:56.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 570.0 s, 445.8 tps, lat 224.445 ms stddev 56.275 2024-05-11T13:50:56.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:56 smithi165 ceph-mon[33235]: pgmap v536: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 9.6 MiB/s wr, 629 op/s 2024-05-11T13:50:56.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:56 smithi063 ceph-mon[29809]: pgmap v536: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 9.6 MiB/s wr, 629 op/s 2024-05-11T13:50:56.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:56 smithi071 ceph-mon[33293]: pgmap v536: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 9.6 MiB/s wr, 629 op/s 2024-05-11T13:50:58.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:50:58 smithi063 ceph-mon[29809]: pgmap v537: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 6.6 MiB/s wr, 458 op/s 2024-05-11T13:50:58.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:50:58 smithi071 ceph-mon[33293]: pgmap v537: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 6.6 MiB/s wr, 458 op/s 2024-05-11T13:50:58.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:50:58 smithi165 ceph-mon[33235]: pgmap v537: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 6.6 MiB/s wr, 458 op/s 2024-05-11T13:51:00.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:00 smithi063 ceph-mon[29809]: pgmap v538: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 9.4 MiB/s wr, 631 op/s 2024-05-11T13:51:00.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:00 smithi071 ceph-mon[33293]: pgmap v538: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 9.4 MiB/s wr, 631 op/s 2024-05-11T13:51:00.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:00 smithi165 ceph-mon[33235]: pgmap v538: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 9.4 MiB/s wr, 631 op/s 2024-05-11T13:51:01.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 575.0 s, 391.0 tps, lat 254.789 ms stddev 96.597 2024-05-11T13:51:02.423 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:02 smithi063 ceph-mon[29809]: pgmap v539: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 6.9 MiB/s wr, 439 op/s 2024-05-11T13:51:02.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:02 smithi071 ceph-mon[33293]: pgmap v539: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 6.9 MiB/s wr, 439 op/s 2024-05-11T13:51:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:02 smithi165 ceph-mon[33235]: pgmap v539: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 6.9 MiB/s wr, 439 op/s 2024-05-11T13:51:04.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:04 smithi063 ceph-mon[29809]: pgmap v540: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 6.4 MiB/s wr, 424 op/s 2024-05-11T13:51:04.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:04 smithi071 ceph-mon[33293]: pgmap v540: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 6.4 MiB/s wr, 424 op/s 2024-05-11T13:51:04.556 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:04 smithi165 ceph-mon[33235]: pgmap v540: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 6.4 MiB/s wr, 424 op/s 2024-05-11T13:51:06.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 580.0 s, 401.2 tps, lat 247.145 ms stddev 96.425 2024-05-11T13:51:06.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:06 smithi063 ceph-mon[29809]: pgmap v541: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 9.2 MiB/s wr, 617 op/s 2024-05-11T13:51:06.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:06 smithi071 ceph-mon[33293]: pgmap v541: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 9.2 MiB/s wr, 617 op/s 2024-05-11T13:51:06.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:06 smithi165 ceph-mon[33235]: pgmap v541: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 9.2 MiB/s wr, 617 op/s 2024-05-11T13:51:08.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:08 smithi063 ceph-mon[29809]: pgmap v542: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 5.6 MiB/s wr, 369 op/s 2024-05-11T13:51:08.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:08 smithi071 ceph-mon[33293]: pgmap v542: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 5.6 MiB/s wr, 369 op/s 2024-05-11T13:51:08.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:08 smithi165 ceph-mon[33235]: pgmap v542: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 5.6 MiB/s wr, 369 op/s 2024-05-11T13:51:10.357 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:10 smithi165 ceph-mon[33235]: pgmap v543: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 8.2 MiB/s wr, 532 op/s 2024-05-11T13:51:10.424 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:10 smithi063 ceph-mon[29809]: pgmap v543: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 8.2 MiB/s wr, 532 op/s 2024-05-11T13:51:10.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:10 smithi071 ceph-mon[33293]: pgmap v543: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 8.2 MiB/s wr, 532 op/s 2024-05-11T13:51:11.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 585.0 s, 340.8 tps, lat 296.456 ms stddev 214.605 2024-05-11T13:51:11.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:11 smithi165 ceph-mon[33235]: osdmap e114: 12 total, 12 up, 12 in 2024-05-11T13:51:11.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:11 smithi063 ceph-mon[29809]: osdmap e114: 12 total, 12 up, 12 in 2024-05-11T13:51:11.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:11 smithi071 ceph-mon[33293]: osdmap e114: 12 total, 12 up, 12 in 2024-05-11T13:51:12.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:12 smithi165 ceph-mon[33235]: pgmap v545: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.5 MiB/s wr, 431 op/s 2024-05-11T13:51:12.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:12 smithi063 ceph-mon[29809]: pgmap v545: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.5 MiB/s wr, 431 op/s 2024-05-11T13:51:12.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:12 smithi071 ceph-mon[33293]: pgmap v545: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.5 MiB/s wr, 431 op/s 2024-05-11T13:51:14.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:14 smithi165 ceph-mon[33235]: pgmap v546: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.5 MiB/s wr, 431 op/s 2024-05-11T13:51:14.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:14 smithi063 ceph-mon[29809]: pgmap v546: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.5 MiB/s wr, 431 op/s 2024-05-11T13:51:14.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:14 smithi071 ceph-mon[33293]: pgmap v546: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.5 MiB/s wr, 431 op/s 2024-05-11T13:51:16.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 590.0 s, 443.4 tps, lat 223.394 ms stddev 56.833 2024-05-11T13:51:16.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:16 smithi165 ceph-mon[33235]: pgmap v547: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.0 MiB/s wr, 408 op/s 2024-05-11T13:51:16.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:16 smithi063 ceph-mon[29809]: pgmap v547: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.0 MiB/s wr, 408 op/s 2024-05-11T13:51:16.696 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:16 smithi071 ceph-mon[33293]: pgmap v547: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.0 MiB/s wr, 408 op/s 2024-05-11T13:51:18.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:18 smithi063 ceph-mon[29809]: pgmap v548: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.0 MiB/s wr, 408 op/s 2024-05-11T13:51:18.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:18 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:51:18.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:18 smithi071 ceph-mon[33293]: pgmap v548: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.0 MiB/s wr, 408 op/s 2024-05-11T13:51:18.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:18 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:51:18.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:18 smithi165 ceph-mon[33235]: pgmap v548: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.0 MiB/s wr, 408 op/s 2024-05-11T13:51:18.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:18 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:51:20.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:20 smithi063 ceph-mon[29809]: pgmap v549: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 461 op/s 2024-05-11T13:51:20.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:20 smithi063 ceph-mon[29809]: osdmap e115: 12 total, 12 up, 12 in 2024-05-11T13:51:20.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:20 smithi071 ceph-mon[33293]: pgmap v549: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 461 op/s 2024-05-11T13:51:20.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:20 smithi071 ceph-mon[33293]: osdmap e115: 12 total, 12 up, 12 in 2024-05-11T13:51:20.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:20 smithi165 ceph-mon[33235]: pgmap v549: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 461 op/s 2024-05-11T13:51:20.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:20 smithi165 ceph-mon[33235]: osdmap e115: 12 total, 12 up, 12 in 2024-05-11T13:51:21.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 595.0 s, 442.0 tps, lat 229.707 ms stddev 62.454 2024-05-11T13:51:22.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:22 smithi063 ceph-mon[29809]: pgmap v551: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2024-05-11T13:51:22.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:22 smithi071 ceph-mon[33293]: pgmap v551: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2024-05-11T13:51:22.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:22 smithi165 ceph-mon[33235]: pgmap v551: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2024-05-11T13:51:24.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:24 smithi063 ceph-mon[29809]: pgmap v552: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2024-05-11T13:51:24.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:24 smithi071 ceph-mon[33293]: pgmap v552: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2024-05-11T13:51:24.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:24 smithi165 ceph-mon[33235]: pgmap v552: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2024-05-11T13:51:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 600.0 s, 422.2 tps, lat 236.225 ms stddev 43.086 2024-05-11T13:51:26.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:26 smithi063 ceph-mon[29809]: pgmap v553: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.1 MiB/s wr, 516 op/s 2024-05-11T13:51:26.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:26 smithi071 ceph-mon[33293]: pgmap v553: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.1 MiB/s wr, 516 op/s 2024-05-11T13:51:26.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:26 smithi165 ceph-mon[33235]: pgmap v553: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.1 MiB/s wr, 516 op/s 2024-05-11T13:51:28.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:28 smithi063 ceph-mon[29809]: pgmap v554: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.1 MiB/s wr, 516 op/s 2024-05-11T13:51:28.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:28 smithi071 ceph-mon[33293]: pgmap v554: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.1 MiB/s wr, 516 op/s 2024-05-11T13:51:28.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:28 smithi165 ceph-mon[33235]: pgmap v554: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.1 MiB/s wr, 516 op/s 2024-05-11T13:51:28.932 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-11T13:51:28.932 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:51:30.582 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:30 smithi063 ceph-mon[29809]: pgmap v555: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 6.3 MiB/s wr, 490 op/s 2024-05-11T13:51:30.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:30 smithi071 ceph-mon[33293]: pgmap v555: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 6.3 MiB/s wr, 490 op/s 2024-05-11T13:51:30.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:30 smithi165 ceph-mon[33235]: pgmap v555: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 6.3 MiB/s wr, 490 op/s 2024-05-11T13:51:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 605.0 s, 367.2 tps, lat 271.849 ms stddev 112.752 2024-05-11T13:51:31.443 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:51:32.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:32 smithi063 ceph-mon[29809]: pgmap v556: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 6.8 MiB/s wr, 507 op/s 2024-05-11T13:51:32.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:32 smithi071 ceph-mon[33293]: pgmap v556: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 6.8 MiB/s wr, 507 op/s 2024-05-11T13:51:32.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:32 smithi165 ceph-mon[33235]: pgmap v556: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 6.8 MiB/s wr, 507 op/s 2024-05-11T13:51:33.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:33 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:51:33.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:33 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:51:33.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:33 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:51:34.574 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:34 smithi063 ceph-mon[29809]: pgmap v557: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.8 MiB/s wr, 430 op/s 2024-05-11T13:51:34.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:34 smithi071 ceph-mon[33293]: pgmap v557: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.8 MiB/s wr, 430 op/s 2024-05-11T13:51:34.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:34 smithi165 ceph-mon[33235]: pgmap v557: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.8 MiB/s wr, 430 op/s 2024-05-11T13:51:35.548 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:35 smithi063 ceph-mon[29809]: pgmap v558: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 593 op/s 2024-05-11T13:51:35.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:35 smithi071 ceph-mon[33293]: pgmap v558: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 593 op/s 2024-05-11T13:51:35.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:35 smithi165 ceph-mon[33235]: pgmap v558: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 593 op/s 2024-05-11T13:51:36.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 610.0 s, 417.6 tps, lat 239.294 ms stddev 51.441 2024-05-11T13:51:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.562+0000 7fac68400700 1 -- 172.21.15.63:0/1837488010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600757c0 msgr2=0x7fac60075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.562+0000 7fac68400700 1 --2- 172.21.15.63:0/1837488010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600757c0 0x7fac60075ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac580098f0 tx=0x7fac58009c00 comp rx=0 tx=0).stop 2024-05-11T13:51:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.564+0000 7fac68400700 1 -- 172.21.15.63:0/1837488010 shutdown_connections 2024-05-11T13:51:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.564+0000 7fac68400700 1 --2- 172.21.15.63:0/1837488010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac60114720 0x7fac60116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.564+0000 7fac68400700 1 --2- 172.21.15.63:0/1837488010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600757c0 0x7fac60075ba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.564+0000 7fac68400700 1 --2- 172.21.15.63:0/1837488010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac600760e0 0x7fac601141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.564+0000 7fac68400700 1 -- 172.21.15.63:0/1837488010 >> 172.21.15.63:0/1837488010 conn(0x7fac6007a740 msgr2=0x7fac6007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:37.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.565+0000 7fac68400700 1 -- 172.21.15.63:0/1837488010 shutdown_connections 2024-05-11T13:51:37.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.565+0000 7fac68400700 1 -- 172.21.15.63:0/1837488010 wait complete. 2024-05-11T13:51:37.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.566+0000 7fac68400700 1 Processor -- start 2024-05-11T13:51:37.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.569+0000 7fac68400700 1 -- start start 2024-05-11T13:51:37.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.569+0000 7fac68400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac600757c0 0x7fac601a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:37.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.569+0000 7fac68400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600760e0 0x7fac601a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:37.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.569+0000 7fac68400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac60114720 0x7fac601a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:37.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.569+0000 7fac68400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac60119ca0 con 0x7fac60114720 2024-05-11T13:51:37.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.569+0000 7fac68400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fac60119b20 con 0x7fac600757c0 2024-05-11T13:51:37.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.569+0000 7fac68400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fac60119e20 con 0x7fac600760e0 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.570+0000 7fac65a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac600757c0 0x7fac601a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.570+0000 7fac65a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac600757c0 0x7fac601a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:53454/0 (socket says 172.21.15.63:53454) 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.570+0000 7fac65a00700 1 -- 172.21.15.63:0/2726245993 learned_addr learned my addr 172.21.15.63:0/2726245993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.570+0000 7fac65000700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600760e0 0x7fac601a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.570+0000 7fac66400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac60114720 0x7fac601a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.571+0000 7fac65a00700 1 -- 172.21.15.63:0/2726245993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600760e0 msgr2=0x7fac601a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.571+0000 7fac65a00700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600760e0 0x7fac601a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:37.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.571+0000 7fac65a00700 1 -- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac60114720 msgr2=0x7fac601a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.571+0000 7fac65a00700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac60114720 0x7fac601a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.571+0000 7fac65a00700 1 -- 172.21.15.63:0/2726245993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac601a5200 con 0x7fac600757c0 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.571+0000 7fac66400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac60114720 0x7fac601a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.571+0000 7fac65a00700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac600757c0 0x7fac601a01f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac58005950 tx=0x7fac5800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.572+0000 7fac5e000700 1 -- 172.21.15.63:0/2726245993 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5800b850 con 0x7fac600757c0 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.572+0000 7fac5e000700 1 -- 172.21.15.63:0/2726245993 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac5800b9f0 con 0x7fac600757c0 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.572+0000 7fac5e000700 1 -- 172.21.15.63:0/2726245993 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58003b80 con 0x7fac600757c0 2024-05-11T13:51:37.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.572+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac601a5490 con 0x7fac600757c0 2024-05-11T13:51:37.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.572+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac601acd70 con 0x7fac600757c0 2024-05-11T13:51:37.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.574+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6006a240 con 0x7fac600757c0 2024-05-11T13:51:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.577+0000 7fac5e000700 1 -- 172.21.15.63:0/2726245993 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fac5800bba0 con 0x7fac600757c0 2024-05-11T13:51:37.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.580+0000 7fac5e000700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fac480b2b40 0x7fac480b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.580+0000 7fac5e000700 1 -- 172.21.15.63:0/2726245993 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fac580e3ba0 con 0x7fac600757c0 2024-05-11T13:51:37.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.583+0000 7fac65000700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fac480b2b40 0x7fac480b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.584+0000 7fac5e000700 1 -- 172.21.15.63:0/2726245993 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac5805fdf0 con 0x7fac600757c0 2024-05-11T13:51:37.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:37.585+0000 7fac65000700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fac480b2b40 0x7fac480b5000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac60102670 tx=0x7fac4c00a380 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:37.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:37 smithi165 ceph-mon[33235]: pgmap v559: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.2 MiB/s wr, 375 op/s 2024-05-11T13:51:37.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:37 smithi063 ceph-mon[29809]: pgmap v559: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.2 MiB/s wr, 375 op/s 2024-05-11T13:51:37.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:37 smithi071 ceph-mon[33293]: pgmap v559: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.2 MiB/s wr, 375 op/s 2024-05-11T13:51:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.068+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac601acf40 con 0x7fac600757c0 2024-05-11T13:51:38.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.069+0000 7fac5e000700 1 -- 172.21.15.63:0/2726245993 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+11942 (secure 0 0 0) 0x7fac58013030 con 0x7fac600757c0 2024-05-11T13:51:38.058 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:51:38.058 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:51:31.551746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:51:38.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fac480b2b40 msgr2=0x7fac480b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:38.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fac480b2b40 0x7fac480b5000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac60102670 tx=0x7fac4c00a380 comp rx=0 tx=0).stop 2024-05-11T13:51:38.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac600757c0 msgr2=0x7fac601a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:38.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac600757c0 0x7fac601a01f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac58005950 tx=0x7fac5800fa00 comp rx=0 tx=0).stop 2024-05-11T13:51:38.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 shutdown_connections 2024-05-11T13:51:38.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fac480b2b40 0x7fac480b5000 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:38.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac60114720 0x7fac601a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:38.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac600757c0 0x7fac601a01f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:38.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 --2- 172.21.15.63:0/2726245993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac600760e0 0x7fac601a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:38.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.073+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 >> 172.21.15.63:0/2726245993 conn(0x7fac6007a740 msgr2=0x7fac60101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:38.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.074+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 shutdown_connections 2024-05-11T13:51:38.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:38.074+0000 7fac68400700 1 -- 172.21.15.63:0/2726245993 wait complete. 2024-05-11T13:51:38.064 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 73 2024-05-11T13:51:38.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:38 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2726245993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:51:38.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:38 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2726245993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:51:38.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:38 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2726245993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:51:39.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:39 smithi165 ceph-mon[33235]: pgmap v560: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 7.6 MiB/s wr, 561 op/s 2024-05-11T13:51:39.899 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:39 smithi063 ceph-mon[29809]: pgmap v560: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 7.6 MiB/s wr, 561 op/s 2024-05-11T13:51:39.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:39 smithi071 ceph-mon[33293]: pgmap v560: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 7.6 MiB/s wr, 561 op/s 2024-05-11T13:51:39.980 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-05-11T13:51:39.981 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-05-11T13:51:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:51:41.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 615.0 s, 335.0 tps, lat 295.788 ms stddev 67.406 2024-05-11T13:51:41.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:41 smithi063 ceph-mon[29809]: pgmap v561: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-05-11T13:51:41.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:41 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:51:41.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:41 smithi165 ceph-mon[33235]: pgmap v561: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-05-11T13:51:41.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:41 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:51:41.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:41 smithi071 ceph-mon[33293]: pgmap v561: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 380 op/s 2024-05-11T13:51:41.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:41 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:51:43.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:43 smithi165 ceph-mon[33235]: pgmap v562: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 4.8 MiB/s wr, 352 op/s 2024-05-11T13:51:43.886 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:43 smithi063 ceph-mon[29809]: pgmap v562: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 4.8 MiB/s wr, 352 op/s 2024-05-11T13:51:43.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:43 smithi071 ceph-mon[33293]: pgmap v562: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 4.8 MiB/s wr, 352 op/s 2024-05-11T13:51:44.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:44 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:51:44.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:44 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:51:44.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:44 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:51:45.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:44 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:51:45.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:44 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:51:45.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:44 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:51:45.376 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:44 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:51:45.376 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:44 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:51:45.376 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:44 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:51:45.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:45 smithi063 ceph-mon[29809]: pgmap v563: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-05-11T13:51:45.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:45 smithi071 ceph-mon[33293]: pgmap v563: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-05-11T13:51:46.056 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:45 smithi165 ceph-mon[33235]: pgmap v563: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-05-11T13:51:46.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 620.0 s, 354.6 tps, lat 280.401 ms stddev 109.988 2024-05-11T13:51:46.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.607+0000 7f8703400700 1 -- 172.21.15.63:0/3434490691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 msgr2=0x7f86fc105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.607+0000 7f8703400700 1 --2- 172.21.15.63:0/3434490691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc105c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009990 tx=0x7f86e8009ca0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.608+0000 7f8703400700 1 -- 172.21.15.63:0/3434490691 shutdown_connections 2024-05-11T13:51:46.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.608+0000 7f8703400700 1 --2- 172.21.15.63:0/3434490691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86fc106150 0x7f86fc112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.608+0000 7f8703400700 1 --2- 172.21.15.63:0/3434490691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86fc10b790 0x7f86fc10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.608+0000 7f8703400700 1 --2- 172.21.15.63:0/3434490691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc105c10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.608+0000 7f8703400700 1 -- 172.21.15.63:0/3434490691 >> 172.21.15.63:0/3434490691 conn(0x7f86fc101280 msgr2=0x7f86fc1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:46.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.609+0000 7f8703400700 1 -- 172.21.15.63:0/3434490691 shutdown_connections 2024-05-11T13:51:46.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.609+0000 7f8703400700 1 -- 172.21.15.63:0/3434490691 wait complete. 2024-05-11T13:51:46.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.611+0000 7f8703400700 1 Processor -- start 2024-05-11T13:51:46.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.611+0000 7f8703400700 1 -- start start 2024-05-11T13:51:46.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.611+0000 7f8703400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.611+0000 7f8703400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86fc106150 0x7f86fc1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8703400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86fc10b790 0x7f86fc19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8703400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1158d0 con 0x7f86fc10b790 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8703400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc115750 con 0x7f86fc105790 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8703400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc115a50 con 0x7f86fc106150 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc1a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:45840/0 (socket says 172.21.15.63:45840) 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 -- 172.21.15.63:0/127157134 learned_addr learned my addr 172.21.15.63:0/127157134 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8701400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86fc10b790 0x7f86fc19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f86fbe00700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86fc106150 0x7f86fc1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 -- 172.21.15.63:0/127157134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86fc106150 msgr2=0x7f86fc1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86fc106150 0x7f86fc1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 -- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86fc10b790 msgr2=0x7f86fc19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86fc10b790 0x7f86fc19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.612+0000 7f8700a00700 1 -- 172.21.15.63:0/127157134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc19c1d0 con 0x7f86fc105790 2024-05-11T13:51:46.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.613+0000 7f8700a00700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc1a18b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c9b0 tx=0x7f86f000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.614+0000 7f86f9600700 1 -- 172.21.15.63:0/127157134 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0007b00 con 0x7f86fc105790 2024-05-11T13:51:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.614+0000 7f86f9600700 1 -- 172.21.15.63:0/127157134 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f0007ca0 con 0x7f86fc105790 2024-05-11T13:51:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.614+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc19c4c0 con 0x7f86fc105790 2024-05-11T13:51:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.614+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a8780 con 0x7f86fc105790 2024-05-11T13:51:46.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.614+0000 7f86f9600700 1 -- 172.21.15.63:0/127157134 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0018660 con 0x7f86fc105790 2024-05-11T13:51:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.622+0000 7f86f9600700 1 -- 172.21.15.63:0/127157134 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f86f000f450 con 0x7f86fc105790 2024-05-11T13:51:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.622+0000 7f870ddf5b80 1 Processor -- start 2024-05-11T13:51:46.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.624+0000 7f870ddf5b80 1 -- start start 2024-05-11T13:51:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.625+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf816e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf817360 0x564daf810f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf8119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f86f9600700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f86e40b2b60 0x7f86e40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f8701400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf816e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f8701400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf816e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:45842/0 (socket says 172.21.15.63:45842) 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f8701400700 1 -- 172.21.15.63:0/3556175875 learned_addr learned my addr 172.21.15.63:0/3556175875 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564daf75abb0 con 0x564daf811560 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564daf6f88f0 con 0x564daf816a40 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564daf756b70 con 0x564daf817360 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f86f9600700 1 -- 172.21.15.63:0/127157134 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f86f00e29a0 con 0x7f86fc105790 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.626+0000 7f8700a00700 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf817360 0x564daf810f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.627+0000 7f8701400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f86e40b2b60 0x7f86e40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.627+0000 7f8701400700 1 -- 172.21.15.63:0/3556175875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf817360 msgr2=0x564daf810f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.627+0000 7f8701400700 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf817360 0x564daf810f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.627+0000 7f8701400700 1 -- 172.21.15.63:0/3556175875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 msgr2=0x564daf8119f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.627+0000 7f8701400700 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf8119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.627+0000 7f8701400700 1 -- 172.21.15.63:0/3556175875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564daf6a7a60 con 0x564daf816a40 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.627+0000 7f8701400700 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf816e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007650 tx=0x7f86f4007960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b920bef945455d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.628+0000 7f86ee000700 1 -- 172.21.15.63:0/3556175875 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f401c040 con 0x564daf816a40 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.628+0000 7f86ee000700 1 -- 172.21.15.63:0/3556175875 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f4018cc0 con 0x564daf816a40 2024-05-11T13:51:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.628+0000 7f86ee000700 1 -- 172.21.15.63:0/3556175875 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4022e10 con 0x564daf816a40 2024-05-11T13:51:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.629+0000 7f8701400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f86e40b2b60 0x7f86e40b5020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f86f4021000 tx=0x7f86f4016810 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.629+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 msgr2=0x564daf816e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.629+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf816e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007650 tx=0x7f86f4007960 comp rx=0 tx=0).stop 2024-05-11T13:51:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.630+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 shutdown_connections 2024-05-11T13:51:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.630+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf8119f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.630+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf816e20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.630+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3556175875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf817360 0x564daf810f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.630+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 >> 172.21.15.63:0/3556175875 conn(0x564daf736cf0 msgr2=0x564daf803350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.631+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 shutdown_connections 2024-05-11T13:51:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.631+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3556175875 wait complete. 2024-05-11T13:51:46.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.631+0000 7f870ddf5b80 1 -- start start 2024-05-11T13:51:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.633+0000 7f870ddf5b80 1 Processor -- start 2024-05-11T13:51:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.635+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf81d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.635+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf816a40 0x564daf81a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.635+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf81aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.635+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564daf81de80 con 0x564daf816a40 2024-05-11T13:51:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.635+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564daf6c0cb0 con 0x564daf817360 2024-05-11T13:51:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.635+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564daf81e000 con 0x564daf811560 2024-05-11T13:51:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.635+0000 7f870ddf5b80 10 client.0 _ll_register_callbacks cb 0x564daf6ee510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:51:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.636+0000 7f8701400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf81aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.636+0000 7f8701400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf81aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:45852/0 (socket says 172.21.15.63:45852) 2024-05-11T13:51:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.636+0000 7f8701400700 1 -- 172.21.15.63:0/3269204462 learned_addr learned my addr 172.21.15.63:0/3269204462 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:51:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.636+0000 7f8700a00700 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf81d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.636+0000 7f86fbe00700 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf816a40 0x564daf81a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.637+0000 7f8700a00700 1 -- 172.21.15.63:0/3269204462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 msgr2=0x564daf81aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.637+0000 7f8700a00700 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf81aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.637+0000 7f8700a00700 1 -- 172.21.15.63:0/3269204462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf816a40 msgr2=0x564daf81a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.637+0000 7f8700a00700 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf816a40 0x564daf81a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.637+0000 7f8700a00700 1 -- 172.21.15.63:0/3269204462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564daf7205b0 con 0x564daf811560 2024-05-11T13:51:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.638+0000 7f8700a00700 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf81d120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f86f00dfd00 tx=0x7f86f000fba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a4800c4f15ff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.638+0000 7f86e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-05-11T13:51:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.638+0000 7f86e2a00700 1 -- 172.21.15.63:0/3269204462 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00a93a0 con 0x564daf811560 2024-05-11T13:51:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.638+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f4018cc0 con 0x564daf811560 2024-05-11T13:51:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.638+0000 7f86e2a00700 1 -- 172.21.15.63:0/3269204462 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f86f00a9af0 con 0x564daf811560 2024-05-11T13:51:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.639+0000 7f86e2a00700 1 -- 172.21.15.63:0/3269204462 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f86f00a9f90 con 0x564daf811560 2024-05-11T13:51:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.639+0000 7f870ddf5b80 10 client.25642 fetch_fsmap learned FSMap version 73 2024-05-11T13:51:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.639+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x564daf828830 con 0x564daf811560 2024-05-11T13:51:46.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.639+0000 7f86e2a00700 1 -- 172.21.15.63:0/3269204462 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f4018cc0 con 0x564daf811560 2024-05-11T13:51:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.640+0000 7f86e2a00700 1 -- 172.21.15.63:0/3269204462 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f86f00bece0 con 0x564daf811560 2024-05-11T13:51:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.640+0000 7f870ddf5b80 10 client.25642 fetch_fsmap finished waiting for FSMap version 73 2024-05-11T13:51:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.640+0000 7f870ddf5b80 10 client.25642 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:51:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.641+0000 7f870ddf5b80 20 client.25642 populate_metadata read hostname 'smithi063' 2024-05-11T13:51:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.641+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf82ad00 0x564daf82d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.641+0000 7f870ddf5b80 4 client.25642 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:51:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.641+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564daf7d51f0 con 0x564daf82ad00 2024-05-11T13:51:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.641+0000 7f86fbe00700 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf82ad00 0x564daf82d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.642+0000 7f86fbe00700 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf82ad00 0x564daf82d0f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca2f16950a2e25f3 server_cookie=aa2a713be8392b15 in_seq=0 out_seq=0 2024-05-11T13:51:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.642+0000 7f86e2a00700 10 client.25642 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:51:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.643+0000 7f86e2a00700 1 -- 172.21.15.63:0/3269204462 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564daf7d51f0 con 0x564daf82ad00 2024-05-11T13:51:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.643+0000 7f86e2a00700 10 client.25642 handle_command_reply: tid=0 2024-05-11T13:51:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.643+0000 7f870ddf5b80 1 client.25642 shutdown 2024-05-11T13:51:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.643+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 msgr2=0x564daf81d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.643+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf81d120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f86f00dfd00 tx=0x7f86f000fba0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 shutdown_connections 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf82ad00 0x564daf82d0f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf81aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf81d120 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3269204462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf816a40 0x564daf81a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 >> 172.21.15.63:0/3269204462 conn(0x564daf736cf0 msgr2=0x564daf80ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f86e2a00700 0 client.25642 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:51:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.644+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 shutdown_connections 2024-05-11T13:51:46.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.645+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3269204462 wait complete. 2024-05-11T13:51:46.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.645+0000 7f870ddf5b80 20 client.25642 trim_cache size 0 max 16384 2024-05-11T13:51:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.679+0000 7f870ddf5b80 1 Processor -- start 2024-05-11T13:51:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.679+0000 7f870ddf5b80 1 -- start start 2024-05-11T13:51:46.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf810f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf816a40 0x564daf819420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf818070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ad420 con 0x564daf811560 2024-05-11T13:51:46.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1aef60 con 0x564daf817360 2024-05-11T13:51:46.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1adaf0 con 0x564daf816a40 2024-05-11T13:51:46.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f8700a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf810f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f8700a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf810f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34036/0 (socket says 172.21.15.63:34036) 2024-05-11T13:51:46.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.680+0000 7f8700a00700 1 -- 172.21.15.63:0/1067021119 learned_addr learned my addr 172.21.15.63:0/1067021119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:51:46.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.681+0000 7f86fbe00700 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf816a40 0x564daf819420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.681+0000 7f8701400700 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf818070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.682+0000 7f8700a00700 1 -- 172.21.15.63:0/1067021119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf816a40 msgr2=0x564daf819420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.682+0000 7f8700a00700 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf816a40 0x564daf819420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.682+0000 7f8700a00700 1 -- 172.21.15.63:0/1067021119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 msgr2=0x564daf818070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.682+0000 7f8700a00700 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf818070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.682+0000 7f8700a00700 1 -- 172.21.15.63:0/1067021119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564daf81dd60 con 0x564daf811560 2024-05-11T13:51:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.683+0000 7f8700a00700 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf810f40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f86f00a9820 tx=0x7f86f00cc2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969383dc03201797 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.683+0000 7f86ed600700 1 -- 172.21.15.63:0/1067021119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00dc070 con 0x564daf811560 2024-05-11T13:51:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.683+0000 7f86ed600700 1 -- 172.21.15.63:0/1067021119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f00c5020 con 0x564daf811560 2024-05-11T13:51:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.683+0000 7f86ed600700 1 -- 172.21.15.63:0/1067021119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00a9f90 con 0x564daf811560 2024-05-11T13:51:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.687+0000 7f870ddf5b80 1 -- 172.21.15.63:0/1067021119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 msgr2=0x564daf810f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.687+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf810f40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f86f00a9820 tx=0x7f86f00cc2d0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.687+0000 7f870ddf5b80 1 -- 172.21.15.63:0/1067021119 shutdown_connections 2024-05-11T13:51:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.688+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf817360 0x564daf818070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.688+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf811560 0x564daf810f40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.688+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/1067021119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf816a40 0x564daf819420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.688+0000 7f870ddf5b80 1 -- 172.21.15.63:0/1067021119 >> 172.21.15.63:0/1067021119 conn(0x564daf736cf0 msgr2=0x564daf815660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:46.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.689+0000 7f870ddf5b80 1 -- 172.21.15.63:0/1067021119 shutdown_connections 2024-05-11T13:51:46.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.689+0000 7f870ddf5b80 1 -- 172.21.15.63:0/1067021119 wait complete. 2024-05-11T13:51:46.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.690+0000 7f870ddf5b80 1 -- start start 2024-05-11T13:51:46.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.691+0000 7f870ddf5b80 1 Processor -- start 2024-05-11T13:51:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.692+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf820510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.692+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf820a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.692+0000 7f870ddf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf817360 0x564daf895440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.692+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ae890 con 0x564daf817360 2024-05-11T13:51:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.692+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ae1c0 con 0x564daf816a40 2024-05-11T13:51:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.692+0000 7f870ddf5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ab360 con 0x564daf811560 2024-05-11T13:51:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.692+0000 7f870ddf5b80 10 client.0 _ll_register_callbacks cb 0x564daf6ee510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:51:46.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.693+0000 7f86fbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf820a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.693+0000 7f86fbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf820a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:45870/0 (socket says 172.21.15.63:45870) 2024-05-11T13:51:46.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.693+0000 7f8701400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf817360 0x564daf895440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.693+0000 7f86fbe00700 1 -- 172.21.15.63:0/3554718028 learned_addr learned my addr 172.21.15.63:0/3554718028 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:51:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.696+0000 7f8701400700 1 -- 172.21.15.63:0/3554718028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 msgr2=0x564daf820510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.696+0000 7f8701400700 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf820510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.696+0000 7f8701400700 1 -- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 msgr2=0x564daf820a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:51:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.696+0000 7f8701400700 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf820a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.696+0000 7f8701400700 1 -- 172.21.15.63:0/3554718028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564daf895980 con 0x564daf817360 2024-05-11T13:51:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.697+0000 7f86fbe00700 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf820a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:51:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.697+0000 7f8701400700 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf817360 0x564daf895440 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f86f4016c10 tx=0x7f86f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c585386c4464ee50 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:51:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.699+0000 7f86e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:51:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.700+0000 7f86e2a00700 1 -- 172.21.15.63:0/3554718028 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f401c040 con 0x564daf817360 2024-05-11T13:51:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.700+0000 7f86e2a00700 1 -- 172.21.15.63:0/3554718028 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f86f4022c60 con 0x564daf817360 2024-05-11T13:51:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.700+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f00c5020 con 0x564daf817360 2024-05-11T13:51:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.701+0000 7f86e2a00700 1 -- 172.21.15.63:0/3554718028 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f86f4037050 con 0x564daf817360 2024-05-11T13:51:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.701+0000 7f870ddf5b80 10 client.16083 fetch_fsmap learned FSMap version 73 2024-05-11T13:51:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.701+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x564daf805c30 con 0x564daf817360 2024-05-11T13:51:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.702+0000 7f86e2a00700 1 -- 172.21.15.63:0/3554718028 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f00c5020 con 0x564daf817360 2024-05-11T13:51:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.703+0000 7f86e2a00700 1 -- 172.21.15.63:0/3554718028 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f86f4020020 con 0x564daf817360 2024-05-11T13:51:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.703+0000 7f870ddf5b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 73 2024-05-11T13:51:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.703+0000 7f870ddf5b80 10 client.16083 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:51:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.703+0000 7f870ddf5b80 20 client.16083 populate_metadata read hostname 'smithi063' 2024-05-11T13:51:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.703+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf806120 0x564daf808540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:51:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.703+0000 7f870ddf5b80 4 client.16083 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-05-11T13:51:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.703+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x564daf768590 con 0x564daf806120 2024-05-11T13:51:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.705+0000 7f8700a00700 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf806120 0x564daf808540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:51:46.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.710+0000 7f8700a00700 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf806120 0x564daf808540 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162e1665fec903e3 server_cookie=776502a4b38ef569 in_seq=0 out_seq=0 2024-05-11T13:51:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.712+0000 7f86e2a00700 10 client.16083 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:51:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.996+0000 7f86e2a00700 1 -- 172.21.15.63:0/3554718028 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564daf768590 con 0x564daf806120 2024-05-11T13:51:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.996+0000 7f86e2a00700 10 client.16083 handle_command_reply: tid=0 2024-05-11T13:51:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.996+0000 7f870ddf5b80 1 client.16083 shutdown 2024-05-11T13:51:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf817360 msgr2=0x564daf895440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf817360 0x564daf895440 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f86f4016c10 tx=0x7f86f4009920 comp rx=0 tx=0).stop 2024-05-11T13:51:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 shutdown_connections 2024-05-11T13:51:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564daf817360 0x564daf895440 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564daf811560 0x564daf820510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x564daf806120 0x564daf808540 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 --2- 172.21.15.63:0/3554718028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564daf816a40 0x564daf820a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 >> 172.21.15.63:0/3554718028 conn(0x564daf736cf0 msgr2=0x564daf817ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f86e2a00700 0 client.16083 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:51:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 shutdown_connections 2024-05-11T13:51:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 1 -- 172.21.15.63:0/3554718028 wait complete. 2024-05-11T13:51:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.997+0000 7f870ddf5b80 20 client.16083 trim_cache size 0 max 16384 2024-05-11T13:51:46.990 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:51:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 73, 2024-05-11T13:51:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:51:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 50, 2024-05-11T13:51:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:51:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "2d212162": { 2024-05-11T13:51:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 50, 2024-05-11T13:51:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:51:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:51:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:51:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:51:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:51:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:51:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 70.0, 2024-05-11T13:51:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:51:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:51:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:51:46.994 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:51:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.998+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f86e40b2b60 msgr2=0x7f86e40b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.998+0000 7f8703400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f86e40b2b60 0x7f86e40b5020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f86f4021000 tx=0x7f86f4016810 comp rx=0 tx=0).stop 2024-05-11T13:51:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.998+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 msgr2=0x7f86fc1a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.998+0000 7f8703400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc1a18b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c9b0 tx=0x7f86f000cd70 comp rx=0 tx=0).stop 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 shutdown_connections 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f86e40b2b60 0x7f86e40b5020 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86fc10b790 0x7f86fc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86fc105790 0x7f86fc1a18b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 --2- 172.21.15.63:0/127157134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86fc106150 0x7f86fc1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 >> 172.21.15.63:0/127157134 conn(0x7f86fc101280 msgr2=0x7f86fc1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 shutdown_connections 2024-05-11T13:51:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:51:46.999+0000 7f8703400700 1 -- 172.21.15.63:0/127157134 wait complete. 2024-05-11T13:51:47.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:47 smithi063 ceph-mon[29809]: pgmap v564: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 5.4 MiB/s wr, 427 op/s 2024-05-11T13:51:47.842 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2d212162', quiesce took 18.9 seconds. Will release after: 10 2024-05-11T13:51:48.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:47 smithi165 ceph-mon[33235]: pgmap v564: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 5.4 MiB/s wr, 427 op/s 2024-05-11T13:51:48.314 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:47 smithi071 ceph-mon[33293]: pgmap v564: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 5.4 MiB/s wr, 427 op/s 2024-05-11T13:51:49.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:49 smithi063 ceph-mon[29809]: pgmap v565: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 6.9 MiB/s wr, 550 op/s 2024-05-11T13:51:49.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:49 smithi071 ceph-mon[33293]: pgmap v565: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 6.9 MiB/s wr, 550 op/s 2024-05-11T13:51:50.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:49 smithi165 ceph-mon[33235]: pgmap v565: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 6.9 MiB/s wr, 550 op/s 2024-05-11T13:51:51.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 625.0 s, 61.9 tps, lat 262.889 ms stddev 110.315 2024-05-11T13:51:51.929 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:51 smithi063 ceph-mon[29809]: pgmap v566: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 4.6 MiB/s wr, 371 op/s 2024-05-11T13:51:51.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:51 smithi071 ceph-mon[33293]: pgmap v566: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 4.6 MiB/s wr, 371 op/s 2024-05-11T13:51:52.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:51 smithi165 ceph-mon[33235]: pgmap v566: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 4.6 MiB/s wr, 371 op/s 2024-05-11T13:51:53.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:53 smithi063 ceph-mon[29809]: pgmap v567: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-05-11T13:51:53.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:53 smithi071 ceph-mon[33293]: pgmap v567: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-05-11T13:51:54.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:53 smithi165 ceph-mon[33235]: pgmap v567: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-05-11T13:51:55.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:55 smithi063 ceph-mon[29809]: pgmap v568: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 5.3 MiB/s wr, 455 op/s 2024-05-11T13:51:55.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:55 smithi071 ceph-mon[33293]: pgmap v568: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 5.3 MiB/s wr, 455 op/s 2024-05-11T13:51:56.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:55 smithi165 ceph-mon[33235]: pgmap v568: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 5.3 MiB/s wr, 455 op/s 2024-05-11T13:51:56.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 630.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:51:57.843 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:51:57.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:57 smithi063 ceph-mon[29809]: pgmap v569: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-05-11T13:51:57.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:57 smithi071 ceph-mon[33293]: pgmap v569: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-05-11T13:51:58.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:57 smithi165 ceph-mon[33235]: pgmap v569: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-05-11T13:51:58.480 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:51:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:51:59 smithi063 ceph-mon[29809]: pgmap v570: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.3 MiB/s wr, 223 op/s 2024-05-11T13:51:59.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:51:59 smithi071 ceph-mon[33293]: pgmap v570: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.3 MiB/s wr, 223 op/s 2024-05-11T13:52:00.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:51:59 smithi165 ceph-mon[33235]: pgmap v570: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.3 MiB/s wr, 223 op/s 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 -- 172.21.15.63:0/933315734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 msgr2=0x7fd0601088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 --2- 172.21.15.63:0/933315734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0601088f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd048009990 tx=0x7fd048009ca0 comp rx=0 tx=0).stop 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 -- 172.21.15.63:0/933315734 shutdown_connections 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 --2- 172.21.15.63:0/933315734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd060103fa0 0x7fd060110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 --2- 172.21.15.63:0/933315734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0601088f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 --2- 172.21.15.63:0/933315734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060103600 0x7fd060103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 -- 172.21.15.63:0/933315734 >> 172.21.15.63:0/933315734 conn(0x7fd06007a6b0 msgr2=0x7fd06007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 -- 172.21.15.63:0/933315734 shutdown_connections 2024-05-11T13:52:00.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.709+0000 7fd067600700 1 -- 172.21.15.63:0/933315734 wait complete. 2024-05-11T13:52:00.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 Processor -- start 2024-05-11T13:52:00.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 -- start start 2024-05-11T13:52:00.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060103600 0x7fd060075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:00.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd060103fa0 0x7fd060075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:00.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0600762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:00.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd060113660 con 0x7fd060108510 2024-05-11T13:52:00.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601134e0 con 0x7fd060103600 2024-05-11T13:52:00.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.711+0000 7fd067600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601137e0 con 0x7fd060103fa0 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0600762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0600762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57034/0 (socket says 172.21.15.63:57034) 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 -- 172.21.15.63:0/162982716 learned_addr learned my addr 172.21.15.63:0/162982716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd064c00700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060103600 0x7fd060075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd057e00700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd060103fa0 0x7fd060075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 -- 172.21.15.63:0/162982716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd060103fa0 msgr2=0x7fd060075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd060103fa0 0x7fd060075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:00.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 -- 172.21.15.63:0/162982716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060103600 msgr2=0x7fd060075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:00.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060103600 0x7fd060075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:00.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd065600700 1 -- 172.21.15.63:0/162982716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0601a80e0 con 0x7fd060108510 2024-05-11T13:52:00.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.712+0000 7fd057e00700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd060103fa0 0x7fd060075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:52:00.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.713+0000 7fd065600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0600762d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd05800efc0 tx=0x7fd05800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:00.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.713+0000 7fd05e000700 1 -- 172.21.15.63:0/162982716 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0580092e0 con 0x7fd060108510 2024-05-11T13:52:00.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.713+0000 7fd05e000700 1 -- 172.21.15.63:0/162982716 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd058004bb0 con 0x7fd060108510 2024-05-11T13:52:00.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.713+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0601a8370 con 0x7fd060108510 2024-05-11T13:52:00.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.713+0000 7fd05e000700 1 -- 172.21.15.63:0/162982716 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058004020 con 0x7fd060108510 2024-05-11T13:52:00.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.713+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0601a8900 con 0x7fd060108510 2024-05-11T13:52:00.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.714+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06006a240 con 0x7fd060108510 2024-05-11T13:52:00.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.718+0000 7fd05e000700 1 -- 172.21.15.63:0/162982716 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd05801b030 con 0x7fd060108510 2024-05-11T13:52:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.720+0000 7fd05e000700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd0400b2a70 0x7fd0400b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:00.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.720+0000 7fd05e000700 1 -- 172.21.15.63:0/162982716 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fd0580e2d50 con 0x7fd060108510 2024-05-11T13:52:00.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.720+0000 7fd064c00700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd0400b2a70 0x7fd0400b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:00.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.721+0000 7fd064c00700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd0400b2a70 0x7fd0400b4f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd04800b2e0 tx=0x7fd048005fb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:00.723+0000 7fd05e000700 1 -- 172.21.15.63:0/162982716 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd058022020 con 0x7fd060108510 2024-05-11T13:52:01.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.034+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd06002e570 con 0x7fd060108510 2024-05-11T13:52:01.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.036+0000 7fd05e000700 1 -- 172.21.15.63:0/162982716 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+11942 (secure 0 0 0) 0x7fd058010040 con 0x7fd060108510 2024-05-11T13:52:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:52:01.026 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:51:31.551746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:52:01.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd0400b2a70 msgr2=0x7fd0400b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:01.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd0400b2a70 0x7fd0400b4f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd04800b2e0 tx=0x7fd048005fb0 comp rx=0 tx=0).stop 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 msgr2=0x7fd0600762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0600762d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd05800efc0 tx=0x7fd05800c3a0 comp rx=0 tx=0).stop 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 shutdown_connections 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd0400b2a70 0x7fd0400b4f30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd060108510 0x7fd0600762d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060103600 0x7fd060075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 --2- 172.21.15.63:0/162982716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd060103fa0 0x7fd060075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.040+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 >> 172.21.15.63:0/162982716 conn(0x7fd06007a6b0 msgr2=0x7fd060101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:01.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.041+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 shutdown_connections 2024-05-11T13:52:01.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:01.041+0000 7fd067600700 1 -- 172.21.15.63:0/162982716 wait complete. 2024-05-11T13:52:01.032 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 73 2024-05-11T13:52:01.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 635.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:52:01.614 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:01 smithi063 ceph-mon[29809]: pgmap v571: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 814 KiB/s wr, 99 op/s 2024-05-11T13:52:01.803 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 2d212162 --release --await' 2024-05-11T13:52:01.803 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 2d212162 --release --await 2024-05-11T13:52:01.884 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:01 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/162982716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:52:01.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:01 smithi071 ceph-mon[33293]: pgmap v571: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 814 KiB/s wr, 99 op/s 2024-05-11T13:52:01.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:01 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/162982716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:52:02.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:01 smithi165 ceph-mon[33235]: pgmap v571: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 814 KiB/s wr, 99 op/s 2024-05-11T13:52:02.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:01 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/162982716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:52:02.264 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:52:03.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:03 smithi063 ceph-mon[29809]: pgmap v572: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 696 KiB/s wr, 93 op/s 2024-05-11T13:52:03.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:03 smithi071 ceph-mon[33293]: pgmap v572: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 696 KiB/s wr, 93 op/s 2024-05-11T13:52:04.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:03 smithi165 ceph-mon[33235]: pgmap v572: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 696 KiB/s wr, 93 op/s 2024-05-11T13:52:04.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.520+0000 7f315f200700 1 -- 172.21.15.63:0/1750407298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158106130 msgr2=0x7f3158112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.520+0000 7f315f200700 1 --2- 172.21.15.63:0/1750407298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158106130 0x7f3158112640 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f315000b0a0 tx=0x7f315000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.520+0000 7f315f200700 1 -- 172.21.15.63:0/1750407298 shutdown_connections 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.520+0000 7f315f200700 1 --2- 172.21.15.63:0/1750407298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158106130 0x7f3158112640 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.520+0000 7f315f200700 1 --2- 172.21.15.63:0/1750407298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f315810b770 0x7f315810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.520+0000 7f315f200700 1 --2- 172.21.15.63:0/1750407298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3158105770 0x7f3158105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.520+0000 7f315f200700 1 -- 172.21.15.63:0/1750407298 >> 172.21.15.63:0/1750407298 conn(0x7f3158101260 msgr2=0x7f3158103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.521+0000 7f315f200700 1 -- 172.21.15.63:0/1750407298 shutdown_connections 2024-05-11T13:52:04.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.521+0000 7f315f200700 1 -- 172.21.15.63:0/1750407298 wait complete. 2024-05-11T13:52:04.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 Processor -- start 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 -- start start 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158105770 0x7f31581a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3158106130 0x7f31581a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f315810b770 0x7f315819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3158115800 con 0x7f3158105770 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3158115680 con 0x7f3158106130 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.522+0000 7f315f200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3158115980 con 0x7f315810b770 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158105770 0x7f31581a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158105770 0x7f31581a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57052/0 (socket says 172.21.15.63:57052) 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 -- 172.21.15.63:0/885432861 learned_addr learned my addr 172.21.15.63:0/885432861 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157400700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3158106130 0x7f31581a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f315d200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f315810b770 0x7f315819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 -- 172.21.15.63:0/885432861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f315810b770 msgr2=0x7f315819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f315810b770 0x7f315819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 -- 172.21.15.63:0/885432861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3158106130 msgr2=0x7f31581a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3158106130 0x7f31581a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f3157e00700 1 -- 172.21.15.63:0/885432861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315819c240 con 0x7f3158105770 2024-05-11T13:52:04.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.523+0000 7f315d200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f315810b770 0x7f315819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:52:04.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.524+0000 7f3157e00700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158105770 0x7f31581a1920 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f314800d950 tx=0x7f314800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:04.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.524+0000 7f3154c00700 1 -- 172.21.15.63:0/885432861 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148016070 con 0x7f3158105770 2024-05-11T13:52:04.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.524+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315819c4d0 con 0x7f3158105770 2024-05-11T13:52:04.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.524+0000 7f3154c00700 1 -- 172.21.15.63:0/885432861 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3148010460 con 0x7f3158105770 2024-05-11T13:52:04.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.524+0000 7f3154c00700 1 -- 172.21.15.63:0/885432861 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314800f620 con 0x7f3158105770 2024-05-11T13:52:04.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.524+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31581a87f0 con 0x7f3158105770 2024-05-11T13:52:04.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.528+0000 7f3154c00700 1 -- 172.21.15.63:0/885432861 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3148019070 con 0x7f3158105770 2024-05-11T13:52:04.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 Processor -- start 2024-05-11T13:52:04.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 -- start start 2024-05-11T13:52:04.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601df41eb20 con 0x5601df4d2ac0 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5601df3bc860 con 0x5601df4cccc0 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.530+0000 7f3169b70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601df41aae0 con 0x5601df4d21a0 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f3154c00700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f313c0b29f0 0x7f313c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f3157e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f3157e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57058/0 (socket says 172.21.15.63:57058) 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f3157e00700 1 -- 172.21.15.63:0/597194302 learned_addr learned my addr 172.21.15.63:0/597194302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f315d200700 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f3154c00700 1 -- 172.21.15.63:0/885432861 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f31480e2630 con 0x7f3158105770 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f3157400700 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.531+0000 7f315d200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f313c0b29f0 0x7f313c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f3157e00700 1 -- 172.21.15.63:0/597194302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 msgr2=0x5601df4d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f3157e00700 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f3157e00700 1 -- 172.21.15.63:0/597194302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 msgr2=0x5601df4cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f3157e00700 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f3157e00700 1 -- 172.21.15.63:0/597194302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601df36b9d0 con 0x5601df4d2ac0 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f315d200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f313c0b29f0 0x7f313c0b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3150006130 tx=0x7f31500092e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f3157e00700 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4cc6a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f31480df990 tx=0x7f3148077090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4243449d969045 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f314d600700 1 -- 172.21.15.63:0/597194302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148020020 con 0x5601df4d2ac0 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f314d600700 1 -- 172.21.15.63:0/597194302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31480a8fd0 con 0x5601df4d2ac0 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.532+0000 7f314d600700 1 -- 172.21.15.63:0/597194302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148075e60 con 0x5601df4d2ac0 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.533+0000 7f3169b70b80 1 -- 172.21.15.63:0/597194302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 msgr2=0x5601df4cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.533+0000 7f3169b70b80 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4cc6a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f31480df990 tx=0x7f3148077090 comp rx=0 tx=0).stop 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.533+0000 7f3169b70b80 1 -- 172.21.15.63:0/597194302 shutdown_connections 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.533+0000 7f3169b70b80 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.534+0000 7f3169b70b80 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4d2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.534+0000 7f3169b70b80 1 --2- 172.21.15.63:0/597194302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4cc6a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.534+0000 7f3169b70b80 1 -- 172.21.15.63:0/597194302 >> 172.21.15.63:0/597194302 conn(0x5601df3fac60 msgr2=0x5601df4c72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.534+0000 7f3169b70b80 1 -- 172.21.15.63:0/597194302 shutdown_connections 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.534+0000 7f3169b70b80 1 -- 172.21.15.63:0/597194302 wait complete. 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.534+0000 7f3169b70b80 1 -- start start 2024-05-11T13:52:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 1 Processor -- start 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d21a0 0x5601df4d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601df4d9580 con 0x5601df4d2ac0 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5601df384c20 con 0x5601df4d21a0 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601df4d9700 con 0x5601df4cccc0 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.535+0000 7f3169b70b80 10 client.0 _ll_register_callbacks cb 0x5601df3b2480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4d61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57062/0 (socket says 172.21.15.63:57062) 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 -- 172.21.15.63:0/3145069986 learned_addr learned my addr 172.21.15.63:0/3145069986 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f3157e00700 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f3157400700 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d21a0 0x5601df4d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 -- 172.21.15.63:0/3145069986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 msgr2=0x5601df4d8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 -- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d21a0 msgr2=0x5601df4d5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d21a0 0x5601df4d5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 -- 172.21.15.63:0/3145069986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601df3e4520 con 0x5601df4d2ac0 2024-05-11T13:52:04.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f315d200700 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4d61e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f315000b0a0 tx=0x7f315000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef252390a1f300b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:04.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.536+0000 7f3142000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:52:04.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.537+0000 7f3142000700 1 -- 172.21.15.63:0/3145069986 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150030060 con 0x5601df4d2ac0 2024-05-11T13:52:04.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.537+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31480a8fd0 con 0x5601df4d2ac0 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.537+0000 7f3142000700 1 -- 172.21.15.63:0/3145069986 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f3150007c90 con 0x5601df4d2ac0 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.537+0000 7f3142000700 1 -- 172.21.15.63:0/3145069986 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3150037050 con 0x5601df4d2ac0 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.537+0000 7f3169b70b80 10 client.16101 fetch_fsmap learned FSMap version 73 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.537+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5601df4e3f20 con 0x5601df4d2ac0 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.537+0000 7f3142000700 1 -- 172.21.15.63:0/3145069986 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31480a8fd0 con 0x5601df4d2ac0 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.538+0000 7f3142000700 1 -- 172.21.15.63:0/3145069986 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f3150011070 con 0x5601df4d2ac0 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.538+0000 7f3169b70b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 73 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.538+0000 7f3169b70b80 10 client.16101 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.538+0000 7f3169b70b80 20 client.16101 populate_metadata read hostname 'smithi063' 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.538+0000 7f3169b70b80 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df4e6560 0x5601df4e8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.538+0000 7f3169b70b80 4 client.16101 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.538+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601df499160 con 0x5601df4e6560 2024-05-11T13:52:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.539+0000 7f3157e00700 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df4e6560 0x5601df4e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.540+0000 7f3157e00700 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df4e6560 0x5601df4e8970 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a905ba34ca0edba server_cookie=225b5f9cb0575f47 in_seq=0 out_seq=0 2024-05-11T13:52:04.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.540+0000 7f3142000700 10 client.16101 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:04.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.541+0000 7f3142000700 1 -- 172.21.15.63:0/3145069986 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601df499160 con 0x5601df4e6560 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3142000700 10 client.16101 handle_command_reply: tid=0 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 client.16101 shutdown 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 msgr2=0x5601df4d61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4d61e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f315000b0a0 tx=0x7f315000b700 comp rx=0 tx=0).stop 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 shutdown_connections 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4d61e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df4e6560 0x5601df4e8970 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 --2- 172.21.15.63:0/3145069986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d21a0 0x5601df4d5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3142000700 0 client.16101 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 >> 172.21.15.63:0/3145069986 conn(0x5601df3fac60 msgr2=0x5601df4cb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 shutdown_connections 2024-05-11T13:52:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 1 -- 172.21.15.63:0/3145069986 wait complete. 2024-05-11T13:52:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.542+0000 7f3169b70b80 20 client.16101 trim_cache size 0 max 16384 2024-05-11T13:52:04.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.564+0000 7f3169b70b80 1 Processor -- start 2024-05-11T13:52:04.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3169b70b80 1 -- start start 2024-05-11T13:52:04.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d21a0 0x5601df4d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d2ac0 0x5601df4d6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3169b70b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31581b17a0 con 0x5601df4d21a0 2024-05-11T13:52:04.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3169b70b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31581b32e0 con 0x5601df4d2ac0 2024-05-11T13:52:04.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3169b70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31581b1e70 con 0x5601df4cccc0 2024-05-11T13:52:04.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3157400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d21a0 0x5601df4d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3157400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d21a0 0x5601df4d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57074/0 (socket says 172.21.15.63:57074) 2024-05-11T13:52:04.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.565+0000 7f3157400700 1 -- 172.21.15.63:0/16693435 learned_addr learned my addr 172.21.15.63:0/16693435 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:04.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f3157e00700 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f315d200700 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d2ac0 0x5601df4d6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f3157400700 1 -- 172.21.15.63:0/16693435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 msgr2=0x5601df4d4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f3157400700 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f3157400700 1 -- 172.21.15.63:0/16693435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d2ac0 msgr2=0x5601df4d6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f3157400700 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d2ac0 0x5601df4d6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f3157400700 1 -- 172.21.15.63:0/16693435 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601df4d4210 con 0x5601df4d21a0 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f315d200700 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d2ac0 0x5601df4d6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.566+0000 7f3157400700 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d21a0 0x5601df4d16e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3144009cf0 tx=0x7f3144012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee824d549f405eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f314cc00700 1 -- 172.21.15.63:0/16693435 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144021070 con 0x5601df4d21a0 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f314cc00700 1 -- 172.21.15.63:0/16693435 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3144012c80 con 0x5601df4d21a0 2024-05-11T13:52:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f314cc00700 1 -- 172.21.15.63:0/16693435 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314401c650 con 0x5601df4d21a0 2024-05-11T13:52:04.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f3169b70b80 1 -- 172.21.15.63:0/16693435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d21a0 msgr2=0x5601df4d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f3169b70b80 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d21a0 0x5601df4d16e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3144009cf0 tx=0x7f3144012450 comp rx=0 tx=0).stop 2024-05-11T13:52:04.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f3169b70b80 1 -- 172.21.15.63:0/16693435 shutdown_connections 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f3169b70b80 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4d2ac0 0x5601df4d6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f3169b70b80 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4cccc0 0x5601df4d4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f3169b70b80 1 --2- 172.21.15.63:0/16693435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d21a0 0x5601df4d16e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.567+0000 7f3169b70b80 1 -- 172.21.15.63:0/16693435 >> 172.21.15.63:0/16693435 conn(0x5601df3fac60 msgr2=0x5601df4d2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.568+0000 7f3169b70b80 1 -- 172.21.15.63:0/16693435 shutdown_connections 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.568+0000 7f3169b70b80 1 -- 172.21.15.63:0/16693435 wait complete. 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.568+0000 7f3169b70b80 1 -- start start 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.568+0000 7f3169b70b80 1 Processor -- start 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4db960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f3169b70b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f3169b70b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31581b2c10 con 0x5601df4d2ac0 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f3169b70b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31581b2540 con 0x5601df4cccc0 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f3169b70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31581af760 con 0x5601df4d21a0 2024-05-11T13:52:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f3169b70b80 10 client.0 _ll_register_callbacks cb 0x5601df3b2480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f315d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f315d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4dc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57076/0 (socket says 172.21.15.63:57076) 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.569+0000 7f315d200700 1 -- 172.21.15.63:0/1237190087 learned_addr learned my addr 172.21.15.63:0/1237190087 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f3157400700 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f3157e00700 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f315d200700 1 -- 172.21.15.63:0/1237190087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 msgr2=0x5601df4dbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f315d200700 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4dbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f315d200700 1 -- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 msgr2=0x5601df4db960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f315d200700 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4db960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f315d200700 1 -- 172.21.15.63:0/1237190087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601df550e90 con 0x5601df4d2ac0 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f315d200700 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4dc3e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f315000b0a0 tx=0x7f3150020620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7327772a7b8d74a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f3142000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f3142000700 1 -- 172.21.15.63:0/1237190087 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150040060 con 0x5601df4d2ac0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f3142000700 1 -- 172.21.15.63:0/1237190087 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f3150007b10 con 0x5601df4d2ac0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.570+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3144012c80 con 0x5601df4d2ac0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3142000700 1 -- 172.21.15.63:0/1237190087 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3150047050 con 0x5601df4d2ac0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 10 client.16107 fetch_fsmap learned FSMap version 73 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5601df552090 con 0x5601df4d2ac0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3142000700 1 -- 172.21.15.63:0/1237190087 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3144012c80 con 0x5601df4d2ac0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3142000700 1 -- 172.21.15.63:0/1237190087 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f3150011040 con 0x5601df4d2ac0 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 73 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 10 client.16107 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 20 client.16107 populate_metadata read hostname 'smithi063' 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df552710 0x5601df554b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 4 client.16107 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2d212162", "release": true, "await": true}] 2024-05-11T13:52:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.571+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2d212162", "release": true, "await": true}) -- 0x5601df42c500 con 0x5601df552710 2024-05-11T13:52:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.572+0000 7f3157e00700 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df552710 0x5601df554b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.573+0000 7f3157e00700 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df552710 0x5601df554b30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82375c537841bbc0 server_cookie=473e333cec34564f in_seq=0 out_seq=0 2024-05-11T13:52:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.573+0000 7f3142000700 10 client.16107 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:04.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.661+0000 7f3142000700 1 -- 172.21.15.63:0/1237190087 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5601df42c500 con 0x5601df552710 2024-05-11T13:52:04.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.661+0000 7f3142000700 10 client.16107 handle_command_reply: tid=0 2024-05-11T13:52:04.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.661+0000 7f3169b70b80 1 client.16107 shutdown 2024-05-11T13:52:04.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 msgr2=0x5601df4dc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4dc3e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f315000b0a0 tx=0x7f3150020620 comp rx=0 tx=0).stop 2024-05-11T13:52:04.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 shutdown_connections 2024-05-11T13:52:04.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601df4d2ac0 0x5601df4dc3e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5601df552710 0x5601df554b30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601df4cccc0 0x5601df4db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 --2- 172.21.15.63:0/1237190087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601df4d21a0 0x5601df4dbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 >> 172.21.15.63:0/1237190087 conn(0x5601df3fac60 msgr2=0x5601df4d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.662+0000 7f3142000700 0 client.16107 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.663+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 shutdown_connections 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.663+0000 7f3169b70b80 1 -- 172.21.15.63:0/1237190087 wait complete. 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.663+0000 7f3169b70b80 20 client.16107 trim_cache size 0 max 16384 2024-05-11T13:52:04.653 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 73, 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 52, 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "2d212162": { 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 52, 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:52:04.654 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 70.0, 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:04.655 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:52:04.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f313c0b29f0 msgr2=0x7f313c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f313c0b29f0 0x7f313c0b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3150006130 tx=0x7f31500092e0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158105770 msgr2=0x7f31581a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158105770 0x7f31581a1920 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f314800d950 tx=0x7f314800dd10 comp rx=0 tx=0).stop 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 shutdown_connections 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f313c0b29f0 0x7f313c0b4eb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f315810b770 0x7f315819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3158105770 0x7f31581a1920 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 --2- 172.21.15.63:0/885432861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3158106130 0x7f31581a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 >> 172.21.15.63:0/885432861 conn(0x7f3158101260 msgr2=0x7f3158101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.665+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 shutdown_connections 2024-05-11T13:52:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:04.666+0000 7f315f200700 1 -- 172.21.15.63:0/885432861 wait complete. 2024-05-11T13:52:05.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:05 smithi063 ceph-mon[29809]: pgmap v573: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 746 KiB/s wr, 114 op/s 2024-05-11T13:52:05.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:05 smithi071 ceph-mon[33293]: pgmap v573: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 746 KiB/s wr, 114 op/s 2024-05-11T13:52:06.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:05 smithi165 ceph-mon[33235]: pgmap v573: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 746 KiB/s wr, 114 op/s 2024-05-11T13:52:06.121 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2d212162', seconds elapsed: 37.2 2024-05-11T13:52:06.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 640.0 s, 151.3 tps, lat 2546.649 ms stddev 6037.901 2024-05-11T13:52:07.907 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:07 smithi063 ceph-mon[29809]: pgmap v574: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-11T13:52:07.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:07 smithi071 ceph-mon[33293]: pgmap v574: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-11T13:52:08.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:07 smithi165 ceph-mon[33235]: pgmap v574: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-11T13:52:09.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:09 smithi063 ceph-mon[29809]: pgmap v575: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-05-11T13:52:09.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:09 smithi071 ceph-mon[33293]: pgmap v575: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-05-11T13:52:10.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:09 smithi165 ceph-mon[33235]: pgmap v575: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-05-11T13:52:10.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:10 smithi063 ceph-mon[29809]: osdmap e116: 12 total, 12 up, 12 in 2024-05-11T13:52:10.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:10 smithi071 ceph-mon[33293]: osdmap e116: 12 total, 12 up, 12 in 2024-05-11T13:52:11.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:10 smithi165 ceph-mon[33235]: osdmap e116: 12 total, 12 up, 12 in 2024-05-11T13:52:11.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 645.0 s, 402.2 tps, lat 249.282 ms stddev 95.922 2024-05-11T13:52:11.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:11 smithi063 ceph-mon[29809]: pgmap v577: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-11T13:52:11.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:11 smithi071 ceph-mon[33293]: pgmap v577: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-11T13:52:11.955 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:11 smithi165 ceph-mon[33235]: pgmap v577: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-11T13:52:13.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:13 smithi063 ceph-mon[29809]: pgmap v578: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-11T13:52:13.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:13 smithi071 ceph-mon[33293]: pgmap v578: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-11T13:52:14.055 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:13 smithi165 ceph-mon[33235]: pgmap v578: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-11T13:52:16.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 650.0 s, 436.0 tps, lat 228.633 ms stddev 42.565 2024-05-11T13:52:16.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:16 smithi165 ceph-mon[33235]: pgmap v579: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.2 MiB/s wr, 289 op/s 2024-05-11T13:52:16.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:16 smithi165 ceph-mon[33235]: osdmap e117: 12 total, 12 up, 12 in 2024-05-11T13:52:16.625 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:16 smithi063 ceph-mon[29809]: pgmap v579: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.2 MiB/s wr, 289 op/s 2024-05-11T13:52:16.625 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:16 smithi063 ceph-mon[29809]: osdmap e117: 12 total, 12 up, 12 in 2024-05-11T13:52:16.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:16 smithi071 ceph-mon[33293]: pgmap v579: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.2 MiB/s wr, 289 op/s 2024-05-11T13:52:16.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:16 smithi071 ceph-mon[33293]: osdmap e117: 12 total, 12 up, 12 in 2024-05-11T13:52:18.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:18 smithi063 ceph-mon[29809]: pgmap v581: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.6 MiB/s wr, 286 op/s 2024-05-11T13:52:18.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:18 smithi071 ceph-mon[33293]: pgmap v581: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.6 MiB/s wr, 286 op/s 2024-05-11T13:52:18.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:18 smithi165 ceph-mon[33235]: pgmap v581: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.6 MiB/s wr, 286 op/s 2024-05-11T13:52:20.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:20 smithi063 ceph-mon[29809]: pgmap v582: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.0 MiB/s wr, 479 op/s 2024-05-11T13:52:20.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:20 smithi071 ceph-mon[33293]: pgmap v582: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.0 MiB/s wr, 479 op/s 2024-05-11T13:52:20.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:20 smithi165 ceph-mon[33235]: pgmap v582: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.0 MiB/s wr, 479 op/s 2024-05-11T13:52:21.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 655.0 s, 395.8 tps, lat 253.575 ms stddev 69.669 2024-05-11T13:52:22.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:22 smithi063 ceph-mon[29809]: pgmap v583: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-05-11T13:52:22.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:22 smithi071 ceph-mon[33293]: pgmap v583: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-05-11T13:52:22.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:22 smithi165 ceph-mon[33235]: pgmap v583: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-05-11T13:52:24.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:24 smithi063 ceph-mon[29809]: pgmap v584: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-05-11T13:52:24.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:24 smithi071 ceph-mon[33293]: pgmap v584: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-05-11T13:52:24.693 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-11T13:52:24.693 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.15 2024-05-11T13:52:24.693 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:52:24.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:24 smithi165 ceph-mon[33235]: pgmap v584: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.1 MiB/s wr, 478 op/s 2024-05-11T13:52:25.469 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:52:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 660.0 s, 395.0 tps, lat 253.764 ms stddev 110.174 2024-05-11T13:52:26.492 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:26 smithi063 ceph-mon[29809]: pgmap v585: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-05-11T13:52:26.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:26 smithi071 ceph-mon[33293]: pgmap v585: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-05-11T13:52:26.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:26 smithi165 ceph-mon[33235]: pgmap v585: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-05-11T13:52:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.330+0000 7fb090000700 1 -- 172.21.15.63:0/2881720814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 msgr2=0x7fb0881123f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.330+0000 7fb090000700 1 --2- 172.21.15.63:0/2881720814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 0x7fb0881123f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b0a0 tx=0x7fb08000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.330+0000 7fb090000700 1 -- 172.21.15.63:0/2881720814 shutdown_connections 2024-05-11T13:52:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.330+0000 7fb090000700 1 --2- 172.21.15.63:0/2881720814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 0x7fb0881123f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.330+0000 7fb090000700 1 --2- 172.21.15.63:0/2881720814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb08806b260 0x7fb08806b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.330+0000 7fb090000700 1 --2- 172.21.15.63:0/2881720814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08806bb80 0x7fb08810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.330+0000 7fb090000700 1 -- 172.21.15.63:0/2881720814 >> 172.21.15.63:0/2881720814 conn(0x7fb088079400 msgr2=0x7fb088079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.332+0000 7fb090000700 1 -- 172.21.15.63:0/2881720814 shutdown_connections 2024-05-11T13:52:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.332+0000 7fb090000700 1 -- 172.21.15.63:0/2881720814 wait complete. 2024-05-11T13:52:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.334+0000 7fb090000700 1 Processor -- start 2024-05-11T13:52:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.334+0000 7fb090000700 1 -- start start 2024-05-11T13:52:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.334+0000 7fb090000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb08806b260 0x7fb088104730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.334+0000 7fb090000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08806bb80 0x7fb088104c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.334+0000 7fb090000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 0x7fb0881051b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.334+0000 7fb090000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb088115880 con 0x7fb08806bb80 2024-05-11T13:52:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.335+0000 7fb090000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb088115700 con 0x7fb088110000 2024-05-11T13:52:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.335+0000 7fb090000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb088115a00 con 0x7fb08806b260 2024-05-11T13:52:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.335+0000 7fb08cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08806bb80 0x7fb088104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.336+0000 7fb08d600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb08806b260 0x7fb088104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.336+0000 7fb08d600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb08806b260 0x7fb088104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:46020/0 (socket says 172.21.15.63:46020) 2024-05-11T13:52:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.336+0000 7fb08d600700 1 -- 172.21.15.63:0/3971483604 learned_addr learned my addr 172.21.15.63:0/3971483604 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.336+0000 7fb08e000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 0x7fb0881051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:28.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.337+0000 7fb08e000700 1 -- 172.21.15.63:0/3971483604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb08806b260 msgr2=0x7fb088104730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:28.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.337+0000 7fb08e000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb08806b260 0x7fb088104730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.337+0000 7fb08e000700 1 -- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08806bb80 msgr2=0x7fb088104c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.337+0000 7fb08e000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08806bb80 0x7fb088104c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.337+0000 7fb08e000700 1 -- 172.21.15.63:0/3971483604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08819bb70 con 0x7fb088110000 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.337+0000 7fb08e000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 0x7fb0881051b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b0a0 tx=0x7fb080004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.338+0000 7fb086000700 1 -- 172.21.15.63:0/3971483604 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080021070 con 0x7fb088110000 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.338+0000 7fb086000700 1 -- 172.21.15.63:0/3971483604 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb080007b80 con 0x7fb088110000 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.338+0000 7fb086000700 1 -- 172.21.15.63:0/3971483604 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800052a0 con 0x7fb088110000 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.339+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08819be00 con 0x7fb088110000 2024-05-11T13:52:28.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.339+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08819c290 con 0x7fb088110000 2024-05-11T13:52:28.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.341+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088102a70 con 0x7fb088110000 2024-05-11T13:52:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.353+0000 7fb086000700 1 -- 172.21.15.63:0/3971483604 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb080024070 con 0x7fb088110000 2024-05-11T13:52:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.355+0000 7fb086000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb0700b2c00 0x7fb0700b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.355+0000 7fb086000700 1 -- 172.21.15.63:0/3971483604 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fb08000f020 con 0x7fb088110000 2024-05-11T13:52:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.356+0000 7fb08d600700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb0700b2c00 0x7fb0700b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.356+0000 7fb086000700 1 -- 172.21.15.63:0/3971483604 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb08002b020 con 0x7fb088110000 2024-05-11T13:52:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.359+0000 7fb08d600700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb0700b2c00 0x7fb0700b50c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0099d0 tx=0x7fb07c008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:28.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:28 smithi063 ceph-mon[29809]: pgmap v586: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 5.5 MiB/s wr, 437 op/s 2024-05-11T13:52:28.694 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:28 smithi071 ceph-mon[33293]: pgmap v586: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 5.5 MiB/s wr, 437 op/s 2024-05-11T13:52:28.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:28 smithi165 ceph-mon[33235]: pgmap v586: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 5.5 MiB/s wr, 437 op/s 2024-05-11T13:52:28.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.872+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb08802e570 con 0x7fb088110000 2024-05-11T13:52:28.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.874+0000 7fb086000700 1 -- 172.21.15.63:0/3971483604 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+11942 (secure 0 0 0) 0x7fb080009cb0 con 0x7fb088110000 2024-05-11T13:52:28.864 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:52:28.864 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:51:31.551746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:52:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb0700b2c00 msgr2=0x7fb0700b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb0700b2c00 0x7fb0700b50c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0099d0 tx=0x7fb07c008040 comp rx=0 tx=0).stop 2024-05-11T13:52:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 msgr2=0x7fb0881051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 0x7fb0881051b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b0a0 tx=0x7fb080004dc0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 shutdown_connections 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fb0700b2c00 0x7fb0700b50c0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0099d0 tx=0x7fb07c008040 comp rx=0 tx=0).stop 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb088110000 0x7fb0881051b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb08806b260 0x7fb088104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.879+0000 7fb090000700 1 --2- 172.21.15.63:0/3971483604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08806bb80 0x7fb088104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.880+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 >> 172.21.15.63:0/3971483604 conn(0x7fb088079400 msgr2=0x7fb0881114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.880+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 shutdown_connections 2024-05-11T13:52:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:28.880+0000 7fb090000700 1 -- 172.21.15.63:0/3971483604 wait complete. 2024-05-11T13:52:28.873 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 73 2024-05-11T13:52:29.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:29 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3971483604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:52:29.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:29 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3971483604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:52:29.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:29 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3971483604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:52:30.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:30 smithi063 ceph-mon[29809]: pgmap v587: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 6.7 MiB/s wr, 546 op/s 2024-05-11T13:52:30.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:30 smithi071 ceph-mon[33293]: pgmap v587: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 6.7 MiB/s wr, 546 op/s 2024-05-11T13:52:30.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:30 smithi165 ceph-mon[33235]: pgmap v587: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 6.7 MiB/s wr, 546 op/s 2024-05-11T13:52:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 665.0 s, 396.2 tps, lat 242.244 ms stddev 63.418 2024-05-11T13:52:32.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:32 smithi063 ceph-mon[29809]: pgmap v588: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.2 MiB/s wr, 420 op/s 2024-05-11T13:52:32.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:32 smithi071 ceph-mon[33293]: pgmap v588: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.2 MiB/s wr, 420 op/s 2024-05-11T13:52:32.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:32 smithi165 ceph-mon[33235]: pgmap v588: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.2 MiB/s wr, 420 op/s 2024-05-11T13:52:32.915 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0' 2024-05-11T13:52:32.916 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0 2024-05-11T13:52:33.649 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:52:34.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:34 smithi063 ceph-mon[29809]: pgmap v589: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.6 MiB/s wr, 374 op/s 2024-05-11T13:52:34.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:34 smithi071 ceph-mon[33293]: pgmap v589: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.6 MiB/s wr, 374 op/s 2024-05-11T13:52:34.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:34 smithi165 ceph-mon[33235]: pgmap v589: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.6 MiB/s wr, 374 op/s 2024-05-11T13:52:36.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 670.0 s, 347.4 tps, lat 298.698 ms stddev 146.061 2024-05-11T13:52:36.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:36 smithi063 ceph-mon[29809]: pgmap v590: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 6.9 MiB/s wr, 561 op/s 2024-05-11T13:52:36.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:36 smithi071 ceph-mon[33293]: pgmap v590: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 6.9 MiB/s wr, 561 op/s 2024-05-11T13:52:36.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:36 smithi165 ceph-mon[33235]: pgmap v590: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 6.9 MiB/s wr, 561 op/s 2024-05-11T13:52:37.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:37 smithi165 ceph-mon[33235]: pgmap v591: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-05-11T13:52:37.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:37 smithi063 ceph-mon[29809]: pgmap v591: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-05-11T13:52:37.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:37 smithi071 ceph-mon[33293]: pgmap v591: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-05-11T13:52:38.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.551+0000 7fd567a00700 1 -- 172.21.15.63:0/3398304315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd560105790 msgr2=0x7fd560105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:38.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.551+0000 7fd567a00700 1 --2- 172.21.15.63:0/3398304315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd560105790 0x7fd560105c10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009990 tx=0x7fd54c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.551+0000 7fd567a00700 1 -- 172.21.15.63:0/3398304315 shutdown_connections 2024-05-11T13:52:38.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.551+0000 7fd567a00700 1 --2- 172.21.15.63:0/3398304315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd560106150 0x7fd560112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.551+0000 7fd567a00700 1 --2- 172.21.15.63:0/3398304315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd56010b790 0x7fd56010bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.551+0000 7fd567a00700 1 --2- 172.21.15.63:0/3398304315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd560105790 0x7fd560105c10 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009990 tx=0x7fd54c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.551+0000 7fd567a00700 1 -- 172.21.15.63:0/3398304315 >> 172.21.15.63:0/3398304315 conn(0x7fd560101280 msgr2=0x7fd5601036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:38.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.554+0000 7fd567a00700 1 -- 172.21.15.63:0/3398304315 shutdown_connections 2024-05-11T13:52:38.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.554+0000 7fd567a00700 1 -- 172.21.15.63:0/3398304315 wait complete. 2024-05-11T13:52:38.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.556+0000 7fd567a00700 1 Processor -- start 2024-05-11T13:52:38.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.556+0000 7fd567a00700 1 -- start start 2024-05-11T13:52:38.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.557+0000 7fd567a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd560106150 0x7fd56010f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.557+0000 7fd567a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd56010b790 0x7fd56010fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.559+0000 7fd567a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd560110300 0x7fd5601a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.559+0000 7fd567a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd560115930 con 0x7fd560110300 2024-05-11T13:52:38.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.559+0000 7fd567a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601157b0 con 0x7fd560106150 2024-05-11T13:52:38.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.559+0000 7fd567a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd560115ab0 con 0x7fd56010b790 2024-05-11T13:52:38.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.560+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd56010b790 0x7fd56010fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.560+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd56010b790 0x7fd56010fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:33264/0 (socket says 172.21.15.63:33264) 2024-05-11T13:52:38.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.560+0000 7fd55fe00700 1 -- 172.21.15.63:0/1112969609 learned_addr learned my addr 172.21.15.63:0/1112969609 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:38.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.561+0000 7fd55fe00700 1 -- 172.21.15.63:0/1112969609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd560106150 msgr2=0x7fd56010f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:38.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.561+0000 7fd55fe00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd560106150 0x7fd56010f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.561+0000 7fd55fe00700 1 -- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd560110300 msgr2=0x7fd5601a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:38.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.561+0000 7fd55fe00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd560110300 0x7fd5601a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.561+0000 7fd55fe00700 1 -- 172.21.15.63:0/1112969609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5601a2390 con 0x7fd56010b790 2024-05-11T13:52:38.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.561+0000 7fd55fe00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd56010b790 0x7fd56010fdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098e0 tx=0x7fd54c003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:38.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.563+0000 7fd55d600700 1 -- 172.21.15.63:0/1112969609 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c026070 con 0x7fd56010b790 2024-05-11T13:52:38.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.563+0000 7fd55d600700 1 -- 172.21.15.63:0/1112969609 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd54c00fc00 con 0x7fd56010b790 2024-05-11T13:52:38.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.563+0000 7fd55d600700 1 -- 172.21.15.63:0/1112969609 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c021b90 con 0x7fd56010b790 2024-05-11T13:52:38.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.563+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5601a2620 con 0x7fd56010b790 2024-05-11T13:52:38.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.564+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5601a2ab0 con 0x7fd56010b790 2024-05-11T13:52:38.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.571+0000 7fd55d600700 1 -- 172.21.15.63:0/1112969609 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd54c029070 con 0x7fd56010b790 2024-05-11T13:52:38.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.577+0000 7fd55d600700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd5440b2c60 0x7fd5440b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.577+0000 7fd55d600700 1 -- 172.21.15.63:0/1112969609 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fd54c0e4140 con 0x7fd56010b790 2024-05-11T13:52:38.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.578+0000 7fd565000700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd5440b2c60 0x7fd5440b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.590+0000 7fd5723bbb80 1 Processor -- start 2024-05-11T13:52:38.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.591+0000 7fd5723bbb80 1 -- start start 2024-05-11T13:52:38.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.592+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 0x562a795b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.592+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.592+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.592+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562a79502bb0 con 0x562a795b6230 2024-05-11T13:52:38.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.592+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562a794a08f0 con 0x562a795b6b50 2024-05-11T13:52:38.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.592+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562a794feb70 con 0x562a795b0d50 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.593+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.594+0000 7fd565000700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd5440b2c60 0x7fd5440b5120 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd55800ab90 tx=0x7fd558009250 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.594+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:52672/0 (socket says 172.21.15.63:52672) 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.594+0000 7fd55fe00700 1 -- 172.21.15.63:0/71274554 learned_addr learned my addr 172.21.15.63:0/71274554 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.594+0000 7fd565a00700 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.594+0000 7fd565000700 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 0x562a795b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.595+0000 7fd565a00700 1 -- 172.21.15.63:0/71274554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 msgr2=0x562a795b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.595+0000 7fd565a00700 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.595+0000 7fd565a00700 1 -- 172.21.15.63:0/71274554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 msgr2=0x562a795b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:38.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.595+0000 7fd565a00700 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 0x562a795b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.595+0000 7fd565a00700 1 -- 172.21.15.63:0/71274554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a7944fa60 con 0x562a795b0d50 2024-05-11T13:52:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.595+0000 7fd565a00700 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795b11e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd550009770 tx=0x7fd550009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c74c452ba37fa4e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.596+0000 7fd556000700 1 -- 172.21.15.63:0/71274554 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550015020 con 0x562a795b0d50 2024-05-11T13:52:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.596+0000 7fd556000700 1 -- 172.21.15.63:0/71274554 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd550011cc0 con 0x562a795b0d50 2024-05-11T13:52:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.596+0000 7fd556000700 1 -- 172.21.15.63:0/71274554 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550010c80 con 0x562a795b0d50 2024-05-11T13:52:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.598+0000 7fd5723bbb80 1 -- 172.21.15.63:0/71274554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 msgr2=0x562a795b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.598+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795b11e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd550009770 tx=0x7fd550009a80 comp rx=0 tx=0).stop 2024-05-11T13:52:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.599+0000 7fd5723bbb80 1 -- 172.21.15.63:0/71274554 shutdown_connections 2024-05-11T13:52:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.599+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795b11e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.599+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 0x562a795b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.599+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/71274554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.599+0000 7fd5723bbb80 1 -- 172.21.15.63:0/71274554 >> 172.21.15.63:0/71274554 conn(0x562a794decf0 msgr2=0x562a795ab350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.599+0000 7fd5723bbb80 1 -- 172.21.15.63:0/71274554 shutdown_connections 2024-05-11T13:52:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.599+0000 7fd5723bbb80 1 -- 172.21.15.63:0/71274554 wait complete. 2024-05-11T13:52:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.600+0000 7fd5723bbb80 1 -- start start 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.601+0000 7fd5723bbb80 1 Processor -- start 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 0x562a795b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562a795bd670 con 0x562a795b6230 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562a79468cb0 con 0x562a795b6b50 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562a795bd7f0 con 0x562a795b0d50 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd565000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd565000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:33288/0 (socket says 172.21.15.63:33288) 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd565000700 1 -- 172.21.15.63:0/3781724696 learned_addr learned my addr 172.21.15.63:0/3781724696 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.602+0000 7fd565a00700 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.603+0000 7fd5723bbb80 10 client.0 _ll_register_callbacks cb 0x562a79496510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.604+0000 7fd565a00700 1 -- 172.21.15.63:0/3781724696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 msgr2=0x562a795bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.604+0000 7fd565a00700 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.604+0000 7fd565a00700 1 -- 172.21.15.63:0/3781724696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 msgr2=0x562a795b9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.604+0000 7fd565a00700 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 0x562a795b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.604+0000 7fd565a00700 1 -- 172.21.15.63:0/3781724696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a794c85b0 con 0x562a795b6b50 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.604+0000 7fd565a00700 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd550009fd0 tx=0x7fd5500105b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19db15aa959eac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.605+0000 7fd54aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:52:38.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.605+0000 7fd54aa00700 1 -- 172.21.15.63:0/3781724696 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550015050 con 0x562a795b6b50 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.605+0000 7fd54aa00700 1 -- 172.21.15.63:0/3781724696 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fd550010900 con 0x562a795b6b50 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.605+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550011cc0 con 0x562a795b6b50 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.606+0000 7fd54aa00700 1 -- 172.21.15.63:0/3781724696 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fd550010b00 con 0x562a795b6b50 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.606+0000 7fd5723bbb80 10 client.35714 fetch_fsmap learned FSMap version 73 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.606+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x562a795be510 con 0x562a795b6b50 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd54aa00700 1 -- 172.21.15.63:0/3781724696 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd550011cc0 con 0x562a795b6b50 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd54aa00700 1 -- 172.21.15.63:0/3781724696 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fd55002cc10 con 0x562a795b6b50 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd5723bbb80 10 client.35714 fetch_fsmap finished waiting for FSMap version 73 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd5723bbb80 10 client.35714 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd5723bbb80 20 client.35714 populate_metadata read hostname 'smithi063' 2024-05-11T13:52:38.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a795c0a40 0x562a795c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd5723bbb80 4 client.35714 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:52:38.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.607+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a7957d1f0 con 0x562a795c0a40 2024-05-11T13:52:38.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.608+0000 7fd565000700 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a795c0a40 0x562a795c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.608+0000 7fd54aa00700 10 client.35714 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:38.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.610+0000 7fd565000700 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a795c0a40 0x562a795c2f00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e953f43b6857d46 server_cookie=1a7d317d3883e39a in_seq=0 out_seq=0 2024-05-11T13:52:38.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.612+0000 7fd54aa00700 1 -- 172.21.15.63:0/3781724696 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a7957d1f0 con 0x562a795c0a40 2024-05-11T13:52:38.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.612+0000 7fd54aa00700 10 client.35714 handle_command_reply: tid=0 2024-05-11T13:52:38.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.612+0000 7fd5723bbb80 1 client.35714 shutdown 2024-05-11T13:52:38.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.612+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 msgr2=0x562a795ba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:38.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.612+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd550009fd0 tx=0x7fd5500105b0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.613+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 shutdown_connections 2024-05-11T13:52:38.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.613+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a795c0a40 0x562a795c2f00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.613+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.613+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.613+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3781724696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b6230 0x562a795b9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.613+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 >> 172.21.15.63:0/3781724696 conn(0x562a794decf0 msgr2=0x562a795af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:38.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.613+0000 7fd54aa00700 0 client.35714 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:38.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.614+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 shutdown_connections 2024-05-11T13:52:38.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.614+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3781724696 wait complete. 2024-05-11T13:52:38.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.615+0000 7fd5723bbb80 20 client.35714 trim_cache size 0 max 16384 2024-05-11T13:52:38.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.663+0000 7fd5723bbb80 1 Processor -- start 2024-05-11T13:52:38.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.663+0000 7fd5723bbb80 1 -- start start 2024-05-11T13:52:38.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.664+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.664+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6230 0x562a795b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.664+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.664+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd560075160 con 0x562a795b0d50 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.664+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601b7b50 con 0x562a795b6b50 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.664+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601b6700 con 0x562a795b6230 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56560/0 (socket says 172.21.15.63:56560) 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565000700 1 -- 172.21.15.63:0/2758032428 learned_addr learned my addr 172.21.15.63:0/2758032428 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:38.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565a00700 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565000700 1 -- 172.21.15.63:0/2758032428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6230 msgr2=0x562a795b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565000700 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6230 0x562a795b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565000700 1 -- 172.21.15.63:0/2758032428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 msgr2=0x562a795ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.665+0000 7fd565000700 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.667+0000 7fd565000700 1 -- 172.21.15.63:0/2758032428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a795acaf0 con 0x562a795b0d50 2024-05-11T13:52:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.667+0000 7fd565000700 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd55800cc10 tx=0x7fd55800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d4c12b245b9e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:38.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.668+0000 7fd555600700 1 -- 172.21.15.63:0/2758032428 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55803d310 con 0x562a795b0d50 2024-05-11T13:52:38.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.669+0000 7fd555600700 1 -- 172.21.15.63:0/2758032428 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd55803d4b0 con 0x562a795b0d50 2024-05-11T13:52:38.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.669+0000 7fd555600700 1 -- 172.21.15.63:0/2758032428 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55803d310 con 0x562a795b0d50 2024-05-11T13:52:38.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.671+0000 7fd5723bbb80 1 -- 172.21.15.63:0/2758032428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 msgr2=0x562a795bc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:38.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.671+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd55800cc10 tx=0x7fd55800f000 comp rx=0 tx=0).stop 2024-05-11T13:52:38.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.672+0000 7fd5723bbb80 1 -- 172.21.15.63:0/2758032428 shutdown_connections 2024-05-11T13:52:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.672+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6b50 0x562a795ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.672+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795bc8a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.672+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/2758032428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6230 0x562a795b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.672+0000 7fd5723bbb80 1 -- 172.21.15.63:0/2758032428 >> 172.21.15.63:0/2758032428 conn(0x562a794decf0 msgr2=0x562a795c2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.672+0000 7fd5723bbb80 1 -- 172.21.15.63:0/2758032428 shutdown_connections 2024-05-11T13:52:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.673+0000 7fd5723bbb80 1 -- 172.21.15.63:0/2758032428 wait complete. 2024-05-11T13:52:38.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.675+0000 7fd5723bbb80 1 -- start start 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.677+0000 7fd5723bbb80 1 Processor -- start 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795c6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6230 0x562a795c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd5723bbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6b50 0x562a796333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601b7480 con 0x562a795b0d50 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601b6db0 con 0x562a795b6230 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd5723bbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd560073120 con 0x562a795b6b50 2024-05-11T13:52:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd5723bbb80 10 client.0 _ll_register_callbacks cb 0x562a795a6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:52:38.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd565000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd565a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6b50 0x562a796333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd565000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56570/0 (socket says 172.21.15.63:56570) 2024-05-11T13:52:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd565a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6b50 0x562a796333a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:33308/0 (socket says 172.21.15.63:33308) 2024-05-11T13:52:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.678+0000 7fd565000700 1 -- 172.21.15.63:0/3972109985 learned_addr learned my addr 172.21.15.63:0/3972109985 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:52:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.679+0000 7fd565a00700 1 -- 172.21.15.63:0/3972109985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6230 msgr2=0x562a795c6b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T13:52:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.679+0000 7fd565a00700 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6230 0x562a795c6b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.679+0000 7fd565a00700 1 -- 172.21.15.63:0/3972109985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 msgr2=0x562a795c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:52:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.679+0000 7fd565a00700 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.679+0000 7fd565a00700 1 -- 172.21.15.63:0/3972109985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a796338e0 con 0x562a795b6b50 2024-05-11T13:52:38.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.680+0000 7fd565a00700 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6b50 0x562a796333a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd55001a5e0 tx=0x7fd550011790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10dfc2bbe154d14e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:52:38.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.680+0000 7fd54aa00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-05-11T13:52:38.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.681+0000 7fd54aa00700 1 -- 172.21.15.63:0/3972109985 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550015030 con 0x562a795b6b50 2024-05-11T13:52:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.681+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd55803d4b0 con 0x562a795b6b50 2024-05-11T13:52:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.681+0000 7fd54aa00700 1 -- 172.21.15.63:0/3972109985 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fd55001a660 con 0x562a795b6b50 2024-05-11T13:52:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.682+0000 7fd54aa00700 1 -- 172.21.15.63:0/3972109985 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fd55001a860 con 0x562a795b6b50 2024-05-11T13:52:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.682+0000 7fd5723bbb80 10 client.25684 fetch_fsmap learned FSMap version 73 2024-05-11T13:52:38.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.682+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x562a79634a80 con 0x562a795b6b50 2024-05-11T13:52:38.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.683+0000 7fd54aa00700 1 -- 172.21.15.63:0/3972109985 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd55803d4b0 con 0x562a795b6b50 2024-05-11T13:52:38.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.683+0000 7fd54aa00700 1 -- 172.21.15.63:0/3972109985 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fd55002c9b0 con 0x562a795b6b50 2024-05-11T13:52:38.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.683+0000 7fd5723bbb80 10 client.25684 fetch_fsmap finished waiting for FSMap version 73 2024-05-11T13:52:38.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.684+0000 7fd5723bbb80 10 client.25684 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:52:38.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.684+0000 7fd5723bbb80 20 client.25684 populate_metadata read hostname 'smithi063' 2024-05-11T13:52:38.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.684+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a79635100 0x562a79637520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:52:38.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.684+0000 7fd5723bbb80 4 client.25684 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}] 2024-05-11T13:52:38.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.684+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}) -- 0x562a79510590 con 0x562a79635100 2024-05-11T13:52:38.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.686+0000 7fd565000700 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a79635100 0x562a79637520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:52:38.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.687+0000 7fd54aa00700 10 client.25684 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:38.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:38.689+0000 7fd565000700 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a79635100 0x562a79637520 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c078f62b08f27f59 server_cookie=467583ac005a6297 in_seq=0 out_seq=0 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.162+0000 7fd54aa00700 1 -- 172.21.15.63:0/3972109985 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562a79510590 con 0x562a79635100 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.162+0000 7fd54aa00700 10 client.25684 handle_command_reply: tid=0 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.162+0000 7fd5723bbb80 1 client.25684 shutdown 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6b50 msgr2=0x562a796333a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6b50 0x562a796333a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd55001a5e0 tx=0x7fd550011790 comp rx=0 tx=0).stop 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 shutdown_connections 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562a795b6b50 0x562a796333a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a795b0d50 0x562a795c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x562a79635100 0x562a79637520 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 --2- 172.21.15.63:0/3972109985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a795b6230 0x562a795c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd54aa00700 0 client.25684 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 >> 172.21.15.63:0/3972109985 conn(0x562a794decf0 msgr2=0x562a795b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 shutdown_connections 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 1 -- 172.21.15.63:0/3972109985 wait complete. 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.163+0000 7fd5723bbb80 20 client.25684 trim_cache size 0 max 16384 2024-05-11T13:52:39.153 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 73, 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 54, 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "8b76808c": { 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 54, 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:52:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.2, 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:52:39.156 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:52:39.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.164+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd5440b2c60 msgr2=0x7fd5440b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:39.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.164+0000 7fd567a00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd5440b2c60 0x7fd5440b5120 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd55800ab90 tx=0x7fd558009250 comp rx=0 tx=0).stop 2024-05-11T13:52:39.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.164+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd56010b790 msgr2=0x7fd56010fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:52:39.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.164+0000 7fd567a00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd56010b790 0x7fd56010fdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098e0 tx=0x7fd54c003680 comp rx=0 tx=0).stop 2024-05-11T13:52:39.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 shutdown_connections 2024-05-11T13:52:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd5440b2c60 0x7fd5440b5120 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd560110300 0x7fd5601a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd560106150 0x7fd56010f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 --2- 172.21.15.63:0/1112969609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd56010b790 0x7fd56010fdc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:52:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 >> 172.21.15.63:0/1112969609 conn(0x7fd560101280 msgr2=0x7fd560101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:52:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 shutdown_connections 2024-05-11T13:52:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:52:39.165+0000 7fd567a00700 1 -- 172.21.15.63:0/1112969609 wait complete. 2024-05-11T13:52:39.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:39 smithi063 ceph-mon[29809]: pgmap v592: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 6.1 MiB/s wr, 511 op/s 2024-05-11T13:52:39.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:39 smithi165 ceph-mon[33235]: pgmap v592: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 6.1 MiB/s wr, 511 op/s 2024-05-11T13:52:39.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:39 smithi071 ceph-mon[33293]: pgmap v592: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 6.1 MiB/s wr, 511 op/s 2024-05-11T13:52:40.750 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8b76808c', quiesce took 16.1 seconds. Will release after: 25.15 2024-05-11T13:52:41.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 675.0 s, 221.9 tps, lat 245.367 ms stddev 59.359 2024-05-11T13:52:41.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:41 smithi063 ceph-mon[29809]: pgmap v593: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 5.0 MiB/s wr, 409 op/s 2024-05-11T13:52:41.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:41 smithi165 ceph-mon[33235]: pgmap v593: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 5.0 MiB/s wr, 409 op/s 2024-05-11T13:52:41.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:41 smithi071 ceph-mon[33293]: pgmap v593: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 5.0 MiB/s wr, 409 op/s 2024-05-11T13:52:43.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:43 smithi063 ceph-mon[29809]: pgmap v594: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 4.4 MiB/s wr, 368 op/s 2024-05-11T13:52:43.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:43 smithi071 ceph-mon[33293]: pgmap v594: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 4.4 MiB/s wr, 368 op/s 2024-05-11T13:52:44.055 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:43 smithi165 ceph-mon[33235]: pgmap v594: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 4.4 MiB/s wr, 368 op/s 2024-05-11T13:52:46.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 680.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:52:46.626 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:46 smithi063 ceph-mon[29809]: pgmap v595: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.8 MiB/s wr, 406 op/s 2024-05-11T13:52:46.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:46 smithi071 ceph-mon[33293]: pgmap v595: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.8 MiB/s wr, 406 op/s 2024-05-11T13:52:46.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:46 smithi165 ceph-mon[33235]: pgmap v595: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.8 MiB/s wr, 406 op/s 2024-05-11T13:52:48.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:48 smithi063 ceph-mon[29809]: pgmap v596: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-05-11T13:52:48.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:48 smithi071 ceph-mon[33293]: pgmap v596: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-05-11T13:52:48.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:48 smithi165 ceph-mon[33235]: pgmap v596: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-05-11T13:52:50.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:50 smithi063 ceph-mon[29809]: pgmap v597: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-05-11T13:52:50.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:50 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:52:50.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:50 smithi071 ceph-mon[33293]: pgmap v597: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-05-11T13:52:50.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:50 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:52:50.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:50 smithi165 ceph-mon[33235]: pgmap v597: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-05-11T13:52:50.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:50 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:52:51.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 685.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:52:51.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:51 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:52:51.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:51 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:52:51.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:51 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:52:51.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:51 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:52:51.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:51 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:52:51.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:51 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:52:51.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:51 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:52:51.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:51 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:52:51.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:51 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:52:52.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:52 smithi063 ceph-mon[29809]: pgmap v598: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-11T13:52:52.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:52 smithi071 ceph-mon[33293]: pgmap v598: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-11T13:52:52.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:52 smithi165 ceph-mon[33235]: pgmap v598: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-11T13:52:54.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:54 smithi063 ceph-mon[29809]: pgmap v599: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 598 KiB/s wr, 61 op/s 2024-05-11T13:52:54.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:54 smithi071 ceph-mon[33293]: pgmap v599: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 598 KiB/s wr, 61 op/s 2024-05-11T13:52:54.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:54 smithi165 ceph-mon[33235]: pgmap v599: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 598 KiB/s wr, 61 op/s 2024-05-11T13:52:56.146 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 690.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:52:56.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:56 smithi063 ceph-mon[29809]: pgmap v600: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-11T13:52:56.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:56 smithi071 ceph-mon[33293]: pgmap v600: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-11T13:52:56.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:56 smithi165 ceph-mon[33235]: pgmap v600: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-11T13:52:58.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:52:58 smithi063 ceph-mon[29809]: pgmap v601: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 634 KiB/s wr, 84 op/s 2024-05-11T13:52:58.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:52:58 smithi071 ceph-mon[33293]: pgmap v601: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 634 KiB/s wr, 84 op/s 2024-05-11T13:52:58.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:52:58 smithi165 ceph-mon[33235]: pgmap v601: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 634 KiB/s wr, 84 op/s 2024-05-11T13:53:00.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:00 smithi063 ceph-mon[29809]: pgmap v602: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 654 KiB/s wr, 89 op/s 2024-05-11T13:53:00.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:00 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:53:00.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:00 smithi071 ceph-mon[33293]: pgmap v602: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 654 KiB/s wr, 89 op/s 2024-05-11T13:53:00.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:00 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:53:00.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:00 smithi165 ceph-mon[33235]: pgmap v602: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 654 KiB/s wr, 89 op/s 2024-05-11T13:53:00.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:00 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:53:01.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 695.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:02 smithi063 ceph-mon[29809]: pgmap v603: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 532 KiB/s wr, 72 op/s 2024-05-11T13:53:02.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:02 smithi071 ceph-mon[33293]: pgmap v603: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 532 KiB/s wr, 72 op/s 2024-05-11T13:53:02.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:02 smithi165 ceph-mon[33235]: pgmap v603: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 532 KiB/s wr, 72 op/s 2024-05-11T13:53:04.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:04 smithi063 ceph-mon[29809]: pgmap v604: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 490 KiB/s wr, 66 op/s 2024-05-11T13:53:04.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:04 smithi071 ceph-mon[33293]: pgmap v604: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 490 KiB/s wr, 66 op/s 2024-05-11T13:53:04.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:04 smithi165 ceph-mon[33235]: pgmap v604: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 490 KiB/s wr, 66 op/s 2024-05-11T13:53:05.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:05 smithi063 ceph-mon[29809]: pgmap v605: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 542 KiB/s wr, 88 op/s 2024-05-11T13:53:05.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:05 smithi071 ceph-mon[33293]: pgmap v605: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 542 KiB/s wr, 88 op/s 2024-05-11T13:53:05.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:05 smithi165 ceph-mon[33235]: pgmap v605: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 542 KiB/s wr, 88 op/s 2024-05-11T13:53:05.901 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:53:06.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 700.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:06.373 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:53:07.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:07 smithi063 ceph-mon[29809]: pgmap v606: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 72 KiB/s wr, 27 op/s 2024-05-11T13:53:07.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:07 smithi071 ceph-mon[33293]: pgmap v606: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 72 KiB/s wr, 27 op/s 2024-05-11T13:53:08.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:07 smithi165 ceph-mon[33235]: pgmap v606: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 72 KiB/s wr, 27 op/s 2024-05-11T13:53:08.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 -- 172.21.15.63:0/3395248870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194105680 msgr2=0x7ff194105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:08.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 --2- 172.21.15.63:0/3395248870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194105680 0x7ff194105b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff184009990 tx=0x7ff184009ca0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 -- 172.21.15.63:0/3395248870 shutdown_connections 2024-05-11T13:53:08.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 --2- 172.21.15.63:0/3395248870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff194106040 0x7ff194112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 --2- 172.21.15.63:0/3395248870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff19410b680 0x7ff19410ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 --2- 172.21.15.63:0/3395248870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194105680 0x7ff194105b00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 -- 172.21.15.63:0/3395248870 >> 172.21.15.63:0/3395248870 conn(0x7ff194101170 msgr2=0x7ff194103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:08.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 -- 172.21.15.63:0/3395248870 shutdown_connections 2024-05-11T13:53:08.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.610+0000 7ff19b200700 1 -- 172.21.15.63:0/3395248870 wait complete. 2024-05-11T13:53:08.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.611+0000 7ff19b200700 1 Processor -- start 2024-05-11T13:53:08.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff19b200700 1 -- start start 2024-05-11T13:53:08.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff19b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff194105680 0x7ff1941a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:08.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff19b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194106040 0x7ff1941a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:08.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff19b200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19410b680 0x7ff19419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:08.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff19b200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff194115780 con 0x7ff194106040 2024-05-11T13:53:08.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff19b200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff194115600 con 0x7ff194105680 2024-05-11T13:53:08.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff19b200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff194115900 con 0x7ff19410b680 2024-05-11T13:53:08.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff199400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194106040 0x7ff1941a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:08.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff199400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194106040 0x7ff1941a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60572/0 (socket says 172.21.15.63:60572) 2024-05-11T13:53:08.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.612+0000 7ff199400700 1 -- 172.21.15.63:0/1429408756 learned_addr learned my addr 172.21.15.63:0/1429408756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:08.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff199e00700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff194105680 0x7ff1941a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:08.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff19a800700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19410b680 0x7ff19419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:08.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff199400700 1 -- 172.21.15.63:0/1429408756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19410b680 msgr2=0x7ff19419baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:08.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff199400700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19410b680 0x7ff19419baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff199400700 1 -- 172.21.15.63:0/1429408756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff194105680 msgr2=0x7ff1941a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:08.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff199400700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff194105680 0x7ff1941a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff199400700 1 -- 172.21.15.63:0/1429408756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19419c200 con 0x7ff194106040 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.613+0000 7ff199400700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194106040 0x7ff1941a1e20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff184005910 tx=0x7ff1840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.614+0000 7ff18ea00700 1 -- 172.21.15.63:0/1429408756 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184027070 con 0x7ff194106040 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.614+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19419c490 con 0x7ff194106040 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.614+0000 7ff18ea00700 1 -- 172.21.15.63:0/1429408756 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff18400fc00 con 0x7ff194106040 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.614+0000 7ff18ea00700 1 -- 172.21.15.63:0/1429408756 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184022d40 con 0x7ff194106040 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.614+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1941a87b0 con 0x7ff194106040 2024-05-11T13:53:08.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.615+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19406a240 con 0x7ff194106040 2024-05-11T13:53:08.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.617+0000 7ff18ea00700 1 -- 172.21.15.63:0/1429408756 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff18402a070 con 0x7ff194106040 2024-05-11T13:53:08.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.619+0000 7ff18ea00700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff17c0b2a70 0x7ff17c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:08.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.619+0000 7ff18ea00700 1 -- 172.21.15.63:0/1429408756 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7ff1840e3ec0 con 0x7ff194106040 2024-05-11T13:53:08.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.619+0000 7ff199e00700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff17c0b2a70 0x7ff17c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:08.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.620+0000 7ff199e00700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff17c0b2a70 0x7ff17c0b4f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff19419cbe0 tx=0x7ff18800b4d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:08.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.622+0000 7ff18ea00700 1 -- 172.21.15.63:0/1429408756 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff184031070 con 0x7ff194106040 2024-05-11T13:53:08.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.913+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff19419cfd0 con 0x7ff194106040 2024-05-11T13:53:08.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.915+0000 7ff18ea00700 1 -- 172.21.15.63:0/1429408756 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+11936 (secure 0 0 0) 0x7ff184013070 con 0x7ff194106040 2024-05-11T13:53:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:53:08.904 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:52:58.859384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:53:08.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff17c0b2a70 msgr2=0x7ff17c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff17c0b2a70 0x7ff17c0b4f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff19419cbe0 tx=0x7ff18800b4d0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194106040 msgr2=0x7ff1941a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194106040 0x7ff1941a1e20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff184005910 tx=0x7ff1840047b0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 shutdown_connections 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff17c0b2a70 0x7ff17c0b4f30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19410b680 0x7ff19419baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff194105680 0x7ff1941a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 --2- 172.21.15.63:0/1429408756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff194106040 0x7ff1941a1e20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.918+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 >> 172.21.15.63:0/1429408756 conn(0x7ff194101170 msgr2=0x7ff194101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.919+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 shutdown_connections 2024-05-11T13:53:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:08.919+0000 7ff19b200700 1 -- 172.21.15.63:0/1429408756 wait complete. 2024-05-11T13:53:08.908 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 74 2024-05-11T13:53:09.593 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:09 smithi063 ceph-mon[29809]: pgmap v607: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 KiB/s wr, 31 op/s 2024-05-11T13:53:09.593 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:09 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1429408756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:53:09.744 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 8b76808c --release --await' 2024-05-11T13:53:09.744 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 8b76808c --release --await 2024-05-11T13:53:09.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:09 smithi071 ceph-mon[33293]: pgmap v607: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 KiB/s wr, 31 op/s 2024-05-11T13:53:09.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:09 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1429408756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:53:10.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:09 smithi165 ceph-mon[33235]: pgmap v607: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 KiB/s wr, 31 op/s 2024-05-11T13:53:10.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:09 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1429408756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:53:10.416 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:53:10.899 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:10 smithi063 ceph-mon[29809]: osdmap e118: 12 total, 12 up, 12 in 2024-05-11T13:53:10.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:10 smithi071 ceph-mon[33293]: osdmap e118: 12 total, 12 up, 12 in 2024-05-11T13:53:11.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:10 smithi165 ceph-mon[33235]: osdmap e118: 12 total, 12 up, 12 in 2024-05-11T13:53:11.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 705.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:11.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:11 smithi063 ceph-mon[29809]: pgmap v609: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-05-11T13:53:11.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:11 smithi071 ceph-mon[33293]: pgmap v609: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-05-11T13:53:11.956 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:11 smithi165 ceph-mon[33235]: pgmap v609: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-05-11T13:53:13.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 -- 172.21.15.63:0/3819624333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce8106150 msgr2=0x7f1ce8112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 --2- 172.21.15.63:0/3819624333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce8106150 0x7f1ce8112660 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000b0a0 tx=0x7f1ce000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 -- 172.21.15.63:0/3819624333 shutdown_connections 2024-05-11T13:53:13.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 --2- 172.21.15.63:0/3819624333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce8106150 0x7f1ce8112660 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 --2- 172.21.15.63:0/3819624333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce810b790 0x7f1ce810bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 --2- 172.21.15.63:0/3819624333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce8105790 0x7f1ce8105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 -- 172.21.15.63:0/3819624333 >> 172.21.15.63:0/3819624333 conn(0x7f1ce8101280 msgr2=0x7f1ce81036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:13.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 -- 172.21.15.63:0/3819624333 shutdown_connections 2024-05-11T13:53:13.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.642+0000 7f1ceee00700 1 -- 172.21.15.63:0/3819624333 wait complete. 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.643+0000 7f1ceee00700 1 Processor -- start 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.643+0000 7f1ceee00700 1 -- start start 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1ceee00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce8105790 0x7f1ce819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1ceee00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce8106150 0x7f1ce819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1ceee00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce810b790 0x7f1ce81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1ceee00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81158d0 con 0x7f1ce810b790 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1ceee00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8115750 con 0x7f1ce8105790 2024-05-11T13:53:13.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1ceee00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8115a50 con 0x7f1ce8106150 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1cece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce810b790 0x7f1ce81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1cece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce810b790 0x7f1ce81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54008/0 (socket says 172.21.15.63:54008) 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.644+0000 7f1cece00700 1 -- 172.21.15.63:0/1526364241 learned_addr learned my addr 172.21.15.63:0/1526364241 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce8105790 0x7f1ce819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1ce7400700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce8106150 0x7f1ce819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1cece00700 1 -- 172.21.15.63:0/1526364241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce8106150 msgr2=0x7f1ce819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1cece00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce8106150 0x7f1ce819c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1cece00700 1 -- 172.21.15.63:0/1526364241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce8105790 msgr2=0x7f1ce819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1cece00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce8105790 0x7f1ce819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1cece00700 1 -- 172.21.15.63:0/1526364241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce81a0d20 con 0x7f1ce810b790 2024-05-11T13:53:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce8105790 0x7f1ce819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:53:13.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.645+0000 7f1cece00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce810b790 0x7f1ce81a05e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000b070 tx=0x7f1ce0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:13.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.646+0000 7f1ce4c00700 1 -- 172.21.15.63:0/1526364241 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0021070 con 0x7f1ce810b790 2024-05-11T13:53:13.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.646+0000 7f1ce4c00700 1 -- 172.21.15.63:0/1526364241 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ce0007b80 con 0x7f1ce810b790 2024-05-11T13:53:13.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.646+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce81a0fb0 con 0x7f1ce810b790 2024-05-11T13:53:13.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.646+0000 7f1ce4c00700 1 -- 172.21.15.63:0/1526364241 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00046e0 con 0x7f1ce810b790 2024-05-11T13:53:13.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.646+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce81a8890 con 0x7f1ce810b790 2024-05-11T13:53:13.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.650+0000 7f1ce4c00700 1 -- 172.21.15.63:0/1526364241 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1ce0003a00 con 0x7f1ce810b790 2024-05-11T13:53:13.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.651+0000 7f1ce4c00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ccc0b2a70 0x7f1ccc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.652+0000 7f1ce4c00700 1 -- 172.21.15.63:0/1526364241 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7f1ce000f020 con 0x7f1ce810b790 2024-05-11T13:53:13.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.652+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ccc0b2a70 0x7f1ccc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.653+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ccc0b2a70 0x7f1ccc0b4f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8005e10 tx=0x7f1cd8005d60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:13.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 Processor -- start 2024-05-11T13:53:13.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 -- start start 2024-05-11T13:53:13.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3aac0 0x558eecb346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb35150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558eeca86b20 con 0x558eecb34cc0 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558eeca24860 con 0x558eecb3aac0 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.660+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558eeca82ae0 con 0x558eecb3a1a0 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb35150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb35150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54016/0 (socket says 172.21.15.63:54016) 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 -- 172.21.15.63:0/1193836209 learned_addr learned my addr 172.21.15.63:0/1193836209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1ce7400700 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3aac0 0x558eecb346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 -- 172.21.15.63:0/1193836209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 msgr2=0x558eecb3a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb3a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 -- 172.21.15.63:0/1193836209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3aac0 msgr2=0x558eecb346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3aac0 0x558eecb346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.661+0000 7f1cece00700 1 -- 172.21.15.63:0/1193836209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eec9d39d0 con 0x558eecb34cc0 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.662+0000 7f1cece00700 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb35150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00a6eb0 tx=0x7f1ce00aa670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d629eab15bb802 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.662+0000 7f1cdd600700 1 -- 172.21.15.63:0/1193836209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00aae30 con 0x558eecb34cc0 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.662+0000 7f1cdd600700 1 -- 172.21.15.63:0/1193836209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ce00af1e0 con 0x558eecb34cc0 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.662+0000 7f1cdd600700 1 -- 172.21.15.63:0/1193836209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00ab090 con 0x558eecb34cc0 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 -- 172.21.15.63:0/1193836209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 msgr2=0x558eecb35150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb35150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00a6eb0 tx=0x7f1ce00aa670 comp rx=0 tx=0).stop 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 -- 172.21.15.63:0/1193836209 shutdown_connections 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb35150 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb3a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/1193836209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3aac0 0x558eecb346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 -- 172.21.15.63:0/1193836209 >> 172.21.15.63:0/1193836209 conn(0x558eeca62c60 msgr2=0x558eecb2f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:13.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 -- 172.21.15.63:0/1193836209 shutdown_connections 2024-05-11T13:53:13.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.663+0000 7f1cf9841b80 1 -- 172.21.15.63:0/1193836209 wait complete. 2024-05-11T13:53:13.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.664+0000 7f1cf9841b80 1 -- start start 2024-05-11T13:53:13.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.664+0000 7f1cf9841b80 1 Processor -- start 2024-05-11T13:53:13.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb40840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb3dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb3e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558eecb413a0 con 0x558eecb34cc0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558eec9ecc20 con 0x558eecb3a1a0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558eecb41520 con 0x558eecb3aac0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cf9841b80 10 client.0 _ll_register_callbacks cb 0x558eeca1a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1ce7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb40840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1ce7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb40840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54024/0 (socket says 172.21.15.63:54024) 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3854612147 learned_addr learned my addr 172.21.15.63:0/3854612147 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1ce7400700 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb3dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.665+0000 7f1cece00700 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb3e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.666+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3854612147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 msgr2=0x558eecb3e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.666+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb3e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.666+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 msgr2=0x558eecb3dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.666+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb3dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.666+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3854612147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eeca4c520 con 0x558eecb34cc0 2024-05-11T13:53:13.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.666+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb40840 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8004680 tx=0x7f1cd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9983f1dce6c308 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:13.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.666+0000 7f1cd2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:53:13.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.667+0000 7f1cd2000700 1 -- 172.21.15.63:0/3854612147 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8024020 con 0x558eecb34cc0 2024-05-11T13:53:13.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.667+0000 7f1cd2000700 1 -- 172.21.15.63:0/3854612147 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7f1cd8003a90 con 0x558eecb34cc0 2024-05-11T13:53:13.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.667+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce00af1e0 con 0x558eecb34cc0 2024-05-11T13:53:13.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.667+0000 7f1cd2000700 1 -- 172.21.15.63:0/3854612147 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f1cd802f050 con 0x558eecb34cc0 2024-05-11T13:53:13.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.667+0000 7f1cf9841b80 10 client.16140 fetch_fsmap learned FSMap version 74 2024-05-11T13:53:13.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.667+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x558eecb4bc10 con 0x558eecb34cc0 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.667+0000 7f1cd2000700 1 -- 172.21.15.63:0/3854612147 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ce00af1e0 con 0x558eecb34cc0 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.668+0000 7f1cd2000700 1 -- 172.21.15.63:0/3854612147 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 74) ==== 8366+0+0 (secure 0 0 0) 0x7f1cd8011440 con 0x558eecb34cc0 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.668+0000 7f1cf9841b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 74 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.668+0000 7f1cf9841b80 10 client.16140 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.668+0000 7f1cf9841b80 20 client.16140 populate_metadata read hostname 'smithi063' 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.668+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecb4e2f0 0x558eecb50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.668+0000 7f1cf9841b80 4 client.16140 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.668+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eecb01160 con 0x558eecb4e2f0 2024-05-11T13:53:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.669+0000 7f1ce7400700 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecb4e2f0 0x558eecb50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.669+0000 7f1ce7400700 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecb4e2f0 0x558eecb50720 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d13595dae54032f server_cookie=c4b4c4625cc24496 in_seq=0 out_seq=0 2024-05-11T13:53:13.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.669+0000 7f1cd2000700 10 client.16140 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:13.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.671+0000 7f1cd2000700 1 -- 172.21.15.63:0/3854612147 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558eecb01160 con 0x558eecb4e2f0 2024-05-11T13:53:13.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.671+0000 7f1cd2000700 10 client.16140 handle_command_reply: tid=0 2024-05-11T13:53:13.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.671+0000 7f1cf9841b80 1 client.16140 shutdown 2024-05-11T13:53:13.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.671+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 msgr2=0x558eecb40840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.671+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb40840 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8004680 tx=0x7f1cd8004750 comp rx=0 tx=0).stop 2024-05-11T13:53:13.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 shutdown_connections 2024-05-11T13:53:13.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb3e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb40840 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecb4e2f0 0x558eecb50720 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3854612147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb3dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 >> 172.21.15.63:0/3854612147 conn(0x558eeca62c60 msgr2=0x558eecb33690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cd2000700 0 client.16140 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 shutdown_connections 2024-05-11T13:53:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3854612147 wait complete. 2024-05-11T13:53:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.672+0000 7f1cf9841b80 20 client.16140 trim_cache size 0 max 16384 2024-05-11T13:53:13.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.694+0000 7f1cf9841b80 1 Processor -- start 2024-05-11T13:53:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.694+0000 7f1cf9841b80 1 -- start start 2024-05-11T13:53:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb38400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb4a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b17b0 con 0x558eecb34cc0 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b32f0 con 0x558eecb3a1a0 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b1e80 con 0x558eecb3aac0 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1ce7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb38400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1ce7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb38400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54028/0 (socket says 172.21.15.63:54028) 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3996621189 learned_addr learned my addr 172.21.15.63:0/3996621189 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.695+0000 7f1ce7400700 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1cece00700 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb4a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3996621189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 msgr2=0x558eecb4a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb4a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3996621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 msgr2=0x558eecb291d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb291d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1ce7e00700 1 -- 172.21.15.63:0/3996621189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eecb50da0 con 0x558eecb34cc0 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1cece00700 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb4a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb38400 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000c00 tx=0x7f1cd8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c42c214f4a3b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1cdcc00700 1 -- 172.21.15.63:0/3996621189 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8024040 con 0x558eecb34cc0 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1cdcc00700 1 -- 172.21.15.63:0/3996621189 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cd8037020 con 0x558eecb34cc0 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.696+0000 7f1cdcc00700 1 -- 172.21.15.63:0/3996621189 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8020440 con 0x558eecb34cc0 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3996621189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 msgr2=0x558eecb38400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb38400 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000c00 tx=0x7f1cd8005ee0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3996621189 shutdown_connections 2024-05-11T13:53:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3aac0 0x558eecb4a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb34cc0 0x558eecb38400 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/3996621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb3a1a0 0x558eecb291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3996621189 >> 172.21.15.63:0/3996621189 conn(0x558eeca62c60 msgr2=0x558eecb37cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3996621189 shutdown_connections 2024-05-11T13:53:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.697+0000 7f1cf9841b80 1 -- 172.21.15.63:0/3996621189 wait complete. 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.698+0000 7f1cf9841b80 1 -- start start 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.698+0000 7f1cf9841b80 1 Processor -- start 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb34cc0 0x558eecb46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb43740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cf9841b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb3aac0 0x558eecb43e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b2c20 con 0x558eecb3aac0 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b2550 con 0x558eecb34cc0 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cf9841b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81af770 con 0x558eecb3a1a0 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cf9841b80 10 client.0 _ll_register_callbacks cb 0x558eeca1a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:53:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb3aac0 0x558eecb43e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb3aac0 0x558eecb43e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54038/0 (socket says 172.21.15.63:54038) 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1cece00700 1 -- 172.21.15.63:0/832638923 learned_addr learned my addr 172.21.15.63:0/832638923 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb34cc0 0x558eecb46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.699+0000 7f1ce7400700 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb43740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cece00700 1 -- 172.21.15.63:0/832638923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 msgr2=0x558eecb43740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cece00700 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb43740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cece00700 1 -- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb34cc0 msgr2=0x558eecb46e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cece00700 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb34cc0 0x558eecb46e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cece00700 1 -- 172.21.15.63:0/832638923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eecbc1400 con 0x558eecb3aac0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cece00700 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb3aac0 0x558eecb43e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00aadc0 tx=0x7f1ce0009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ccfc4ab45d5ebf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cd2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cd2000700 1 -- 172.21.15.63:0/832638923 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce006e070 con 0x558eecb3aac0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cd2000700 1 -- 172.21.15.63:0/832638923 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7f1ce00ab2a0 con 0x558eecb3aac0 2024-05-11T13:53:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd8037020 con 0x558eecb3aac0 2024-05-11T13:53:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cd2000700 1 -- 172.21.15.63:0/832638923 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f1ce00b3c00 con 0x558eecb3aac0 2024-05-11T13:53:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cf9841b80 10 client.16146 fetch_fsmap learned FSMap version 74 2024-05-11T13:53:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.700+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x558eecbc2590 con 0x558eecb3aac0 2024-05-11T13:53:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cd2000700 1 -- 172.21.15.63:0/832638923 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cd8037020 con 0x558eecb3aac0 2024-05-11T13:53:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cd2000700 1 -- 172.21.15.63:0/832638923 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 74) ==== 8366+0+0 (secure 0 0 0) 0x7f1ce0011070 con 0x558eecb3aac0 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cf9841b80 10 client.16146 fetch_fsmap finished waiting for FSMap version 74 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cf9841b80 10 client.16146 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cf9841b80 20 client.16146 populate_metadata read hostname 'smithi063' 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecbc2ac0 0x558eecbc4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cf9841b80 4 client.16146 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8b76808c", "release": true, "await": true}] 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.701+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8b76808c", "release": true, "await": true}) -- 0x558eeca94500 con 0x558eecbc2ac0 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.702+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecbc2ac0 0x558eecbc4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.703+0000 7f1ce7e00700 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecbc2ac0 0x558eecbc4ee0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906c4fb5ac1b4aa6 server_cookie=8b7286a9ea0645b9 in_seq=0 out_seq=0 2024-05-11T13:53:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.703+0000 7f1cd2000700 10 client.16146 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:13.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.801+0000 7f1cd2000700 1 -- 172.21.15.63:0/832638923 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558eeca94500 con 0x558eecbc2ac0 2024-05-11T13:53:13.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.801+0000 7f1cd2000700 10 client.16146 handle_command_reply: tid=0 2024-05-11T13:53:13.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.801+0000 7f1cf9841b80 1 client.16146 shutdown 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.802+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb3aac0 msgr2=0x558eecb43e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.802+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb3aac0 0x558eecb43e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00aadc0 tx=0x7f1ce0009db0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 shutdown_connections 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558eecb3aac0 0x558eecb43e80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x558eecbc2ac0 0x558eecbc4ee0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558eecb34cc0 0x558eecb46e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 --2- 172.21.15.63:0/832638923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eecb3a1a0 0x558eecb43740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 >> 172.21.15.63:0/832638923 conn(0x558eeca62c60 msgr2=0x558eecb3aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:13.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cd2000700 0 client.16146 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:13.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 shutdown_connections 2024-05-11T13:53:13.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.803+0000 7f1cf9841b80 1 -- 172.21.15.63:0/832638923 wait complete. 2024-05-11T13:53:13.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.804+0000 7f1cf9841b80 20 client.16146 trim_cache size 0 max 16384 2024-05-11T13:53:13.793 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 74, 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 56, 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "8b76808c": { 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 56, 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:53:13.794 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.2, 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:13.795 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.805+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ccc0b2a70 msgr2=0x7f1ccc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.805+0000 7f1ceee00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ccc0b2a70 0x7f1ccc0b4f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8005e10 tx=0x7f1cd8005d60 comp rx=0 tx=0).stop 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce810b790 msgr2=0x7f1ce81a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce810b790 0x7f1ce81a05e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000b070 tx=0x7f1ce0004d50 comp rx=0 tx=0).stop 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 shutdown_connections 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1ccc0b2a70 0x7f1ccc0b4f30 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8005e10 tx=0x7f1cd8005d60 comp rx=0 tx=0).stop 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce810b790 0x7f1ce81a05e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce8105790 0x7f1ce819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 --2- 172.21.15.63:0/1526364241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce8106150 0x7f1ce819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 >> 172.21.15.63:0/1526364241 conn(0x7f1ce8101280 msgr2=0x7f1ce8101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:13.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.806+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 shutdown_connections 2024-05-11T13:53:13.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:13.807+0000 7f1ceee00700 1 -- 172.21.15.63:0/1526364241 wait complete. 2024-05-11T13:53:13.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:13 smithi063 ceph-mon[29809]: pgmap v610: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-05-11T13:53:13.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:13 smithi071 ceph-mon[33293]: pgmap v610: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-05-11T13:53:14.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:13 smithi165 ceph-mon[33235]: pgmap v610: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-05-11T13:53:15.106 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8b76808c', seconds elapsed: 50.4 2024-05-11T13:53:15.106 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.15 2024-05-11T13:53:15.106 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:53:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:53:16.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 710.0 s, 150.3 tps, lat 4962.008 ms stddev 11851.367 2024-05-11T13:53:16.577 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:16 smithi063 ceph-mon[29809]: pgmap v611: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-11T13:53:16.577 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:16 smithi063 ceph-mon[29809]: osdmap e119: 12 total, 12 up, 12 in 2024-05-11T13:53:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:16 smithi071 ceph-mon[33293]: pgmap v611: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-11T13:53:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:16 smithi071 ceph-mon[33293]: osdmap e119: 12 total, 12 up, 12 in 2024-05-11T13:53:16.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:16 smithi165 ceph-mon[33235]: pgmap v611: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-11T13:53:16.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:16 smithi165 ceph-mon[33235]: osdmap e119: 12 total, 12 up, 12 in 2024-05-11T13:53:18.657 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:18 smithi063 ceph-mon[29809]: pgmap v613: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-11T13:53:18.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:18 smithi071 ceph-mon[33293]: pgmap v613: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-11T13:53:18.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:18 smithi165 ceph-mon[33235]: pgmap v613: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-11T13:53:20.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.237+0000 7f65b2600700 1 -- 172.21.15.63:0/3211434152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac10ffb0 msgr2=0x7f65ac1123a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.237+0000 7f65b2600700 1 --2- 172.21.15.63:0/3211434152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac10ffb0 0x7f65ac1123a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f65a400b0a0 tx=0x7f65a400b3b0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.238+0000 7f65b2600700 1 -- 172.21.15.63:0/3211434152 shutdown_connections 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.238+0000 7f65b2600700 1 --2- 172.21.15.63:0/3211434152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac10ffb0 0x7f65ac1123a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.238+0000 7f65b2600700 1 --2- 172.21.15.63:0/3211434152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ac075780 0x7f65ac075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.238+0000 7f65b2600700 1 --2- 172.21.15.63:0/3211434152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65ac0760a0 0x7f65ac10fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.238+0000 7f65b2600700 1 -- 172.21.15.63:0/3211434152 >> 172.21.15.63:0/3211434152 conn(0x7f65ac07a6b0 msgr2=0x7f65ac07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.238+0000 7f65b2600700 1 -- 172.21.15.63:0/3211434152 shutdown_connections 2024-05-11T13:53:20.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.238+0000 7f65b2600700 1 -- 172.21.15.63:0/3211434152 wait complete. 2024-05-11T13:53:20.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.240+0000 7f65b2600700 1 Processor -- start 2024-05-11T13:53:20.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.240+0000 7f65b2600700 1 -- start start 2024-05-11T13:53:20.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.240+0000 7f65b2600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ac075780 0x7f65ac074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:20.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.240+0000 7f65b2600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac0760a0 0x7f65ac075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.240+0000 7f65b2600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65ac10ffb0 0x7f65ac06dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65b2600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac115820 con 0x7f65ac0760a0 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65b2600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1156a0 con 0x7f65ac075780 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65b2600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1159a0 con 0x7f65ac10ffb0 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65a3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac0760a0 0x7f65ac075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65a3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac0760a0 0x7f65ac075200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39252/0 (socket says 172.21.15.63:39252) 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65a3e00700 1 -- 172.21.15.63:0/2234259507 learned_addr learned my addr 172.21.15.63:0/2234259507 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65ab400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ac075780 0x7f65ac074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:20.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.241+0000 7f65abe00700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65ac10ffb0 0x7f65ac06dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:20.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.242+0000 7f65a3e00700 1 -- 172.21.15.63:0/2234259507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65ac10ffb0 msgr2=0x7f65ac06dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:20.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.242+0000 7f65a3e00700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65ac10ffb0 0x7f65ac06dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.242+0000 7f65a3e00700 1 -- 172.21.15.63:0/2234259507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ac075780 msgr2=0x7f65ac074cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:20.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.242+0000 7f65a3e00700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ac075780 0x7f65ac074cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.242+0000 7f65a3e00700 1 -- 172.21.15.63:0/2234259507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac06e860 con 0x7f65ac0760a0 2024-05-11T13:53:20.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.243+0000 7f65a3e00700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac0760a0 0x7f65ac075200 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f659400e9a0 tx=0x7f659400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:20.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.243+0000 7f65a8c00700 1 -- 172.21.15.63:0/2234259507 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659400cbd0 con 0x7f65ac0760a0 2024-05-11T13:53:20.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.244+0000 7f65a8c00700 1 -- 172.21.15.63:0/2234259507 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f659400cd70 con 0x7f65ac0760a0 2024-05-11T13:53:20.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.244+0000 7f65a8c00700 1 -- 172.21.15.63:0/2234259507 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594018680 con 0x7f65ac0760a0 2024-05-11T13:53:20.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.244+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac06eb50 con 0x7f65ac0760a0 2024-05-11T13:53:20.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.244+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac072a80 con 0x7f65ac0760a0 2024-05-11T13:53:20.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.246+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ac076e40 con 0x7f65ac0760a0 2024-05-11T13:53:20.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.252+0000 7f65a8c00700 1 -- 172.21.15.63:0/2234259507 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6594010430 con 0x7f65ac0760a0 2024-05-11T13:53:20.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.254+0000 7f65a8c00700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65900b2a70 0x7f65900b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:20.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.254+0000 7f65a8c00700 1 -- 172.21.15.63:0/2234259507 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f65940e2e50 con 0x7f65ac0760a0 2024-05-11T13:53:20.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.255+0000 7f65ab400700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65900b2a70 0x7f65900b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:20.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.257+0000 7f65ab400700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65900b2a70 0x7f65900b4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f659c009c60 tx=0x7f659c009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:20.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.259+0000 7f65a8c00700 1 -- 172.21.15.63:0/2234259507 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65940aa4f0 con 0x7f65ac0760a0 2024-05-11T13:53:20.674 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:20 smithi063 ceph-mon[29809]: pgmap v614: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-05-11T13:53:20.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:20 smithi071 ceph-mon[33293]: pgmap v614: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-05-11T13:53:20.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.778+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65ac06a240 con 0x7f65ac0760a0 2024-05-11T13:53:20.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.779+0000 7f65a8c00700 1 -- 172.21.15.63:0/2234259507 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+11936 (secure 0 0 0) 0x7f65940a9df0 con 0x7f65ac0760a0 2024-05-11T13:53:20.768 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:53:20.768 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:52:58.859384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65900b2a70 msgr2=0x7f65900b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65900b2a70 0x7f65900b4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f659c009c60 tx=0x7f659c009260 comp rx=0 tx=0).stop 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac0760a0 msgr2=0x7f65ac075200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac0760a0 0x7f65ac075200 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f659400e9a0 tx=0x7f659400ecb0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 shutdown_connections 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f65900b2a70 0x7f65900b4f30 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f659c009c60 tx=0x7f659c009260 comp rx=0 tx=0).stop 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65ac10ffb0 0x7f65ac06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ac075780 0x7f65ac074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 --2- 172.21.15.63:0/2234259507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65ac0760a0 0x7f65ac075200 secure :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f659400e9a0 tx=0x7f659400ecb0 comp rx=0 tx=0).stop 2024-05-11T13:53:20.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.783+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 >> 172.21.15.63:0/2234259507 conn(0x7f65ac07a6b0 msgr2=0x7f65ac1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:20.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.785+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 shutdown_connections 2024-05-11T13:53:20.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:20.785+0000 7f65b2600700 1 -- 172.21.15.63:0/2234259507 wait complete. 2024-05-11T13:53:20.776 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 74 2024-05-11T13:53:20.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:20 smithi165 ceph-mon[33235]: pgmap v614: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-05-11T13:53:21.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 715.0 s, 418.0 tps, lat 239.783 ms stddev 53.930 2024-05-11T13:53:21.565 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:21 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2234259507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:53:21.648 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0' 2024-05-11T13:53:21.649 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0 2024-05-11T13:53:21.693 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:21 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2234259507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:53:21.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:21 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2234259507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:53:22.388 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:53:22.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:22 smithi063 ceph-mon[29809]: pgmap v615: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-05-11T13:53:22.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:22 smithi071 ceph-mon[33293]: pgmap v615: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-05-11T13:53:22.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:22 smithi165 ceph-mon[33235]: pgmap v615: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-05-11T13:53:24.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:24 smithi063 ceph-mon[29809]: pgmap v616: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-05-11T13:53:24.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:24 smithi071 ceph-mon[33293]: pgmap v616: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-05-11T13:53:24.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:24 smithi165 ceph-mon[33235]: pgmap v616: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-05-11T13:53:25.383 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:25 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:53:25.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:25 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:53:25.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:25 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:53:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 720.0 s, 389.0 tps, lat 258.302 ms stddev 66.966 2024-05-11T13:53:26.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:26 smithi063 ceph-mon[29809]: pgmap v617: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 5.0 MiB/s wr, 405 op/s 2024-05-11T13:53:26.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:26 smithi071 ceph-mon[33293]: pgmap v617: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 5.0 MiB/s wr, 405 op/s 2024-05-11T13:53:26.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:26 smithi165 ceph-mon[33235]: pgmap v617: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 5.0 MiB/s wr, 405 op/s 2024-05-11T13:53:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.096+0000 7f7239600700 1 -- 172.21.15.63:0/1986725734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723406b1a0 msgr2=0x7f7234116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.096+0000 7f7239600700 1 --2- 172.21.15.63:0/1986725734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723406b1a0 0x7f7234116ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f722c00b0a0 tx=0x7f722c00b3b0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.096+0000 7f7239600700 1 -- 172.21.15.63:0/1986725734 shutdown_connections 2024-05-11T13:53:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.096+0000 7f7239600700 1 --2- 172.21.15.63:0/1986725734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723406b1a0 0x7f7234116ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.096+0000 7f7239600700 1 --2- 172.21.15.63:0/1986725734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f723410a7b0 0x7f723410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.096+0000 7f7239600700 1 --2- 172.21.15.63:0/1986725734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410b160 0x7f723410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.096+0000 7f7239600700 1 -- 172.21.15.63:0/1986725734 >> 172.21.15.63:0/1986725734 conn(0x7f723406eb40 msgr2=0x7f723406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.097+0000 7f7239600700 1 -- 172.21.15.63:0/1986725734 shutdown_connections 2024-05-11T13:53:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.097+0000 7f7239600700 1 -- 172.21.15.63:0/1986725734 wait complete. 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.100+0000 7f7239600700 1 Processor -- start 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.100+0000 7f7239600700 1 -- start start 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.101+0000 7f7239600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f723406b1a0 0x7f7234116690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.101+0000 7f7239600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410a7b0 0x7f7234116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.101+0000 7f7239600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723410b160 0x7f7234113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.101+0000 7f7239600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7234119c40 con 0x7f723406b1a0 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.101+0000 7f7239600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7234119ac0 con 0x7f723410a7b0 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.101+0000 7f7239600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7234119dc0 con 0x7f723410b160 2024-05-11T13:53:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410a7b0 0x7f7234116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410a7b0 0x7f7234116bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:37478/0 (socket says 172.21.15.63:37478) 2024-05-11T13:53:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 -- 172.21.15.63:0/1661635602 learned_addr learned my addr 172.21.15.63:0/1661635602 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7233400700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723410b160 0x7f7234113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 -- 172.21.15.63:0/1661635602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723410b160 msgr2=0x7f7234113840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723410b160 0x7f7234113840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 -- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f723406b1a0 msgr2=0x7f7234116690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f723406b1a0 0x7f7234116690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.102+0000 7f7232000700 1 -- 172.21.15.63:0/1661635602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7234114120 con 0x7f723410a7b0 2024-05-11T13:53:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.103+0000 7f7232000700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410a7b0 0x7f7234116bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f722000e9f0 tx=0x7f722000edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.103+0000 7f722b400700 1 -- 172.21.15.63:0/1661635602 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722000cbd0 con 0x7f723410a7b0 2024-05-11T13:53:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.104+0000 7f722b400700 1 -- 172.21.15.63:0/1661635602 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f722000cd70 con 0x7f723410a7b0 2024-05-11T13:53:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.104+0000 7f722b400700 1 -- 172.21.15.63:0/1661635602 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72200055f0 con 0x7f723410a7b0 2024-05-11T13:53:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.104+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7234114410 con 0x7f723410a7b0 2024-05-11T13:53:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.104+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72341b55c0 con 0x7f723410a7b0 2024-05-11T13:53:28.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.111+0000 7f722b400700 1 -- 172.21.15.63:0/1661635602 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7220004750 con 0x7f723410a7b0 2024-05-11T13:53:28.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.115+0000 7f722b400700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f72180b2cb0 0x7f72180b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.116+0000 7f722b400700 1 -- 172.21.15.63:0/1661635602 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f72200e2cd0 con 0x7f723410a7b0 2024-05-11T13:53:28.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.116+0000 7f7232a00700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f72180b2cb0 0x7f72180b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.118+0000 7f7232a00700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f72180b2cb0 0x7f72180b5170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7224005fd0 tx=0x7f7224005f60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:28.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.126+0000 7f7244018b80 1 Processor -- start 2024-05-11T13:53:28.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.127+0000 7f7244018b80 1 -- start start 2024-05-11T13:53:28.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.127+0000 7f7244018b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.127+0000 7f7244018b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e13f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.127+0000 7f7244018b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e13f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.127+0000 7f7244018b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5603e1344bb0 con 0x5603e13f8230 2024-05-11T13:53:28.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.127+0000 7f7244018b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5603e12e28f0 con 0x5603e13f8b50 2024-05-11T13:53:28.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.127+0000 7f7244018b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5603e1340b70 con 0x5603e13f2d50 2024-05-11T13:53:28.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.128+0000 7f7232a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.128+0000 7f7232a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39278/0 (socket says 172.21.15.63:39278) 2024-05-11T13:53:28.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.128+0000 7f7232a00700 1 -- 172.21.15.63:0/638099904 learned_addr learned my addr 172.21.15.63:0/638099904 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:28.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.128+0000 7f7232000700 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e13f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.129+0000 7f7232a00700 1 -- 172.21.15.63:0/638099904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 msgr2=0x5603e13f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.129+0000 7f7232a00700 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e13f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.129+0000 7f7232a00700 1 -- 172.21.15.63:0/638099904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 msgr2=0x5603e13f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.129+0000 7f7232a00700 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e13f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.129+0000 7f7232a00700 1 -- 172.21.15.63:0/638099904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e1291a60 con 0x5603e13f8230 2024-05-11T13:53:28.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.130+0000 7f7232a00700 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f8610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7224004400 tx=0x7f7224010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce349090816df4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:28.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.130+0000 7f721fe00700 1 -- 172.21.15.63:0/638099904 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224023070 con 0x5603e13f8230 2024-05-11T13:53:28.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.130+0000 7f721fe00700 1 -- 172.21.15.63:0/638099904 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7224019050 con 0x5603e13f8230 2024-05-11T13:53:28.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.130+0000 7f721fe00700 1 -- 172.21.15.63:0/638099904 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224010930 con 0x5603e13f8230 2024-05-11T13:53:28.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.131+0000 7f7244018b80 1 -- 172.21.15.63:0/638099904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 msgr2=0x5603e13f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.131+0000 7f7244018b80 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f8610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7224004400 tx=0x7f7224010000 comp rx=0 tx=0).stop 2024-05-11T13:53:28.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.132+0000 7f7244018b80 1 -- 172.21.15.63:0/638099904 shutdown_connections 2024-05-11T13:53:28.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.132+0000 7f7244018b80 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e13f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.132+0000 7f7244018b80 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f8610 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.132+0000 7f7244018b80 1 --2- 172.21.15.63:0/638099904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e13f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.132+0000 7f7244018b80 1 -- 172.21.15.63:0/638099904 >> 172.21.15.63:0/638099904 conn(0x5603e1320cf0 msgr2=0x5603e13ed350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:28.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.133+0000 7f7244018b80 1 -- 172.21.15.63:0/638099904 shutdown_connections 2024-05-11T13:53:28.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.133+0000 7f7244018b80 1 -- 172.21.15.63:0/638099904 wait complete. 2024-05-11T13:53:28.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.133+0000 7f7244018b80 1 -- start start 2024-05-11T13:53:28.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.134+0000 7f7244018b80 1 Processor -- start 2024-05-11T13:53:28.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7244018b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f2d50 0x5603e13fe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7244018b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e13fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7244018b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f8b50 0x5603e13fc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7244018b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5603e13ff670 con 0x5603e13f2d50 2024-05-11T13:53:28.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7244018b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5603e12aacb0 con 0x5603e13f8230 2024-05-11T13:53:28.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7244018b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5603e13ff7f0 con 0x5603e13f8b50 2024-05-11T13:53:28.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7244018b80 10 client.0 _ll_register_callbacks cb 0x5603e12d8510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:53:28.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7232a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f2d50 0x5603e13fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7232000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e13fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7232a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f2d50 0x5603e13fe990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39288/0 (socket says 172.21.15.63:39288) 2024-05-11T13:53:28.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.135+0000 7f7232a00700 1 -- 172.21.15.63:0/2717216464 learned_addr learned my addr 172.21.15.63:0/2717216464 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:28.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.136+0000 7f7233400700 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f8b50 0x5603e13fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.136+0000 7f7232000700 1 -- 172.21.15.63:0/2717216464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f8b50 msgr2=0x5603e13fc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.136+0000 7f7232000700 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f8b50 0x5603e13fc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.136+0000 7f7232000700 1 -- 172.21.15.63:0/2717216464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f2d50 msgr2=0x5603e13fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.136+0000 7f7232000700 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f2d50 0x5603e13fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.136+0000 7f7232000700 1 -- 172.21.15.63:0/2717216464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e130a5b0 con 0x5603e13f8230 2024-05-11T13:53:28.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.137+0000 7f7233400700 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f8b50 0x5603e13fc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:53:28.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.137+0000 7f7232000700 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e13fbd30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72200dffd0 tx=0x7f7220004cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af18e168588c6281 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:28.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.137+0000 7f721cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:53:28.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.137+0000 7f721cc00700 1 -- 172.21.15.63:0/2717216464 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220016540 con 0x5603e13f8230 2024-05-11T13:53:28.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.137+0000 7f721cc00700 1 -- 172.21.15.63:0/2717216464 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f72200a6150 con 0x5603e13f8230 2024-05-11T13:53:28.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.137+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7224019050 con 0x5603e13f8230 2024-05-11T13:53:28.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.138+0000 7f721cc00700 1 -- 172.21.15.63:0/2717216464 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f72200a9670 con 0x5603e13f8230 2024-05-11T13:53:28.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.138+0000 7f7244018b80 10 client.35759 fetch_fsmap learned FSMap version 75 2024-05-11T13:53:28.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.138+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5603e140a040 con 0x5603e13f8230 2024-05-11T13:53:28.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.138+0000 7f721cc00700 1 -- 172.21.15.63:0/2717216464 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7224019050 con 0x5603e13f8230 2024-05-11T13:53:28.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.139+0000 7f721cc00700 1 -- 172.21.15.63:0/2717216464 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 75) ==== 8362+0+0 (secure 0 0 0) 0x7f72200a9dc0 con 0x5603e13f8230 2024-05-11T13:53:28.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.140+0000 7f7244018b80 10 client.35759 fetch_fsmap finished waiting for FSMap version 75 2024-05-11T13:53:28.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.140+0000 7f7244018b80 10 client.35759 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:53:28.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.140+0000 7f7244018b80 20 client.35759 populate_metadata read hostname 'smithi063' 2024-05-11T13:53:28.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.140+0000 7f7244018b80 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e140c5d0 0x5603e140e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.140+0000 7f7244018b80 4 client.35759 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:53:28.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.140+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e13bf1f0 con 0x5603e140c5d0 2024-05-11T13:53:28.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.141+0000 7f7232a00700 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e140c5d0 0x5603e140e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.141+0000 7f7232a00700 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e140c5d0 0x5603e140e9c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af710f9791b5467 server_cookie=15eeebea19c947ca in_seq=0 out_seq=0 2024-05-11T13:53:28.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.141+0000 7f721cc00700 10 client.35759 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:28.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.142+0000 7f721cc00700 1 -- 172.21.15.63:0/2717216464 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603e13bf1f0 con 0x5603e140c5d0 2024-05-11T13:53:28.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.142+0000 7f721cc00700 10 client.35759 handle_command_reply: tid=0 2024-05-11T13:53:28.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.142+0000 7f7244018b80 1 client.35759 shutdown 2024-05-11T13:53:28.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 msgr2=0x5603e13fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e13fbd30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72200dffd0 tx=0x7f7220004cc0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 shutdown_connections 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f8b50 0x5603e13fc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f2d50 0x5603e13fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e140c5d0 0x5603e140e9c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 --2- 172.21.15.63:0/2717216464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e13fbd30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 >> 172.21.15.63:0/2717216464 conn(0x5603e1320cf0 msgr2=0x5603e13f1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.143+0000 7f721cc00700 0 client.35759 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.144+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 shutdown_connections 2024-05-11T13:53:28.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.144+0000 7f7244018b80 1 -- 172.21.15.63:0/2717216464 wait complete. 2024-05-11T13:53:28.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.144+0000 7f7244018b80 20 client.35759 trim_cache size 0 max 16384 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.186+0000 7f7244018b80 1 Processor -- start 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.186+0000 7f7244018b80 1 -- start start 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.186+0000 7f7244018b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e13f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7244018b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7244018b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e1408760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7244018b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72341be3a0 con 0x5603e13f8230 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7244018b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f72341bfee0 con 0x5603e13f8b50 2024-05-11T13:53:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7244018b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72341bea70 con 0x5603e13f2d50 2024-05-11T13:53:28.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7233400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e1408760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7233400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e1408760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:37506/0 (socket says 172.21.15.63:37506) 2024-05-11T13:53:28.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7233400700 1 -- 172.21.15.63:0/2669662036 learned_addr learned my addr 172.21.15.63:0/2669662036 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:28.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7232000700 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.187+0000 7f7232a00700 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e13f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.188+0000 7f7232000700 1 -- 172.21.15.63:0/2669662036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 msgr2=0x5603e13f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.188+0000 7f7232000700 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e13f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.188+0000 7f7232000700 1 -- 172.21.15.63:0/2669662036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 msgr2=0x5603e1408760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.188+0000 7f7232000700 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e1408760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.188+0000 7f7232000700 1 -- 172.21.15.63:0/2669662036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e140efa0 con 0x5603e13f8230 2024-05-11T13:53:28.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.188+0000 7f7232000700 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f7570 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f72200a6dd0 tx=0x7f7220009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8485385e7a600e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:28.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.189+0000 7f721f400700 1 -- 172.21.15.63:0/2669662036 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72200a8660 con 0x5603e13f8230 2024-05-11T13:53:28.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.189+0000 7f721f400700 1 -- 172.21.15.63:0/2669662036 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72200a8660 con 0x5603e13f8230 2024-05-11T13:53:28.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.189+0000 7f721f400700 1 -- 172.21.15.63:0/2669662036 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72200b29c0 con 0x5603e13f8230 2024-05-11T13:53:28.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 -- 172.21.15.63:0/2669662036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 msgr2=0x5603e13f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f7570 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f72200a6dd0 tx=0x7f7220009810 comp rx=0 tx=0).stop 2024-05-11T13:53:28.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 -- 172.21.15.63:0/2669662036 shutdown_connections 2024-05-11T13:53:28.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8b50 0x5603e1408760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e13f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 --2- 172.21.15.63:0/2669662036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8230 0x5603e13f7570 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 -- 172.21.15.63:0/2669662036 >> 172.21.15.63:0/2669662036 conn(0x5603e1320cf0 msgr2=0x5603e13f6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:28.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 -- 172.21.15.63:0/2669662036 shutdown_connections 2024-05-11T13:53:28.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.190+0000 7f7244018b80 1 -- 172.21.15.63:0/2669662036 wait complete. 2024-05-11T13:53:28.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.191+0000 7f7244018b80 1 -- start start 2024-05-11T13:53:28.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.191+0000 7f7244018b80 1 Processor -- start 2024-05-11T13:53:28.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7244018b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e1405190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7244018b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e1401ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7244018b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8b50 0x5603e14021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7244018b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72341bf810 con 0x5603e13f8b50 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7244018b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f72341bf140 con 0x5603e13f8230 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7244018b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72341bc360 con 0x5603e13f2d50 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7244018b80 10 client.0 _ll_register_callbacks cb 0x5603e12d8510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.192+0000 7f7233400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8b50 0x5603e14021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e1405190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7233400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8b50 0x5603e14021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39306/0 (socket says 172.21.15.63:39306) 2024-05-11T13:53:28.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e1405190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:40038/0 (socket says 172.21.15.63:40038) 2024-05-11T13:53:28.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7233400700 1 -- 172.21.15.63:0/1281644935 learned_addr learned my addr 172.21.15.63:0/1281644935 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:28.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232000700 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e1401ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232000700 1 -- 172.21.15.63:0/1281644935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 msgr2=0x5603e1405190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232000700 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e1405190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232000700 1 -- 172.21.15.63:0/1281644935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8b50 msgr2=0x5603e14021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232000700 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8b50 0x5603e14021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.193+0000 7f7232000700 1 -- 172.21.15.63:0/1281644935 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e1476df0 con 0x5603e13f8230 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.194+0000 7f7232000700 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e1401ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72200c43c0 tx=0x7f72200b27e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc990ddfd65a7165 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.194+0000 7f721cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.194+0000 7f721cc00700 1 -- 172.21.15.63:0/1281644935 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220016540 con 0x5603e13f8230 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.194+0000 7f721cc00700 1 -- 172.21.15.63:0/1281644935 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f72200a8fd0 con 0x5603e13f8230 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.194+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72200a8660 con 0x5603e13f8230 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f721cc00700 1 -- 172.21.15.63:0/1281644935 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f72200adf10 con 0x5603e13f8230 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 10 client.35765 fetch_fsmap learned FSMap version 75 2024-05-11T13:53:28.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5603e1477f60 con 0x5603e13f8230 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f721cc00700 1 -- 172.21.15.63:0/1281644935 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72200a8660 con 0x5603e13f8230 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f721cc00700 1 -- 172.21.15.63:0/1281644935 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 75) ==== 8362+0+0 (secure 0 0 0) 0x7f72200a8830 con 0x5603e13f8230 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 10 client.35765 fetch_fsmap finished waiting for FSMap version 75 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 10 client.35765 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 20 client.35765 populate_metadata read hostname 'smithi063' 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e1478430 0x5603e147a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 4 client.35765 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}] 2024-05-11T13:53:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.195+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}) -- 0x5603e1352590 con 0x5603e1478430 2024-05-11T13:53:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.197+0000 7f7232a00700 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e1478430 0x5603e147a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.199+0000 7f7232a00700 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e1478430 0x5603e147a820 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36acdf90448ba34 server_cookie=c33bb1e37b4c2f03 in_seq=0 out_seq=0 2024-05-11T13:53:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.199+0000 7f721cc00700 10 client.35765 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f721cc00700 1 -- 172.21.15.63:0/1281644935 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5603e1352590 con 0x5603e1478430 2024-05-11T13:53:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f721cc00700 10 client.35765 handle_command_reply: tid=0 2024-05-11T13:53:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f7244018b80 1 client.35765 shutdown 2024-05-11T13:53:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 msgr2=0x5603e1401ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f7244018b80 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e1401ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72200c43c0 tx=0x7f72200b27e0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 shutdown_connections 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f7244018b80 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603e13f8b50 0x5603e14021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.517+0000 7f7244018b80 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603e13f2d50 0x5603e1405190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.518+0000 7f7244018b80 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5603e1478430 0x5603e147a820 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.518+0000 7f7244018b80 1 --2- 172.21.15.63:0/1281644935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5603e13f8230 0x5603e1401ab0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.518+0000 7f721cc00700 0 client.35765 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.518+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 >> 172.21.15.63:0/1281644935 conn(0x5603e1320cf0 msgr2=0x5603e13f76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.518+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 shutdown_connections 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.518+0000 7f7244018b80 1 -- 172.21.15.63:0/1281644935 wait complete. 2024-05-11T13:53:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.518+0000 7f7244018b80 20 client.35765 trim_cache size 0 max 16384 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 75, 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 58, 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "2fccb18f": { 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 58, 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:53:28.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.2, 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:53:28.509 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f72180b2cb0 msgr2=0x7f72180b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f72180b2cb0 0x7f72180b5170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7224005fd0 tx=0x7f7224005f60 comp rx=0 tx=0).stop 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410a7b0 msgr2=0x7f7234116bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410a7b0 0x7f7234116bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f722000e9f0 tx=0x7f722000edb0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 shutdown_connections 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f72180b2cb0 0x7f72180b5170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f723410b160 0x7f7234113840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f723406b1a0 0x7f7234116690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 --2- 172.21.15.63:0/1661635602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f723410a7b0 0x7f7234116bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:28.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.519+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 >> 172.21.15.63:0/1661635602 conn(0x7f723406eb40 msgr2=0x7f7234073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:28.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.520+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 shutdown_connections 2024-05-11T13:53:28.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:28.520+0000 7f7239600700 1 -- 172.21.15.63:0/1661635602 wait complete. 2024-05-11T13:53:28.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:28 smithi063 ceph-mon[29809]: pgmap v618: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-05-11T13:53:28.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:28 smithi071 ceph-mon[33293]: pgmap v618: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-05-11T13:53:28.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:28 smithi165 ceph-mon[33235]: pgmap v618: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-05-11T13:53:29.328 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2fccb18f', quiesce took 14.2 seconds. Will release after: 25.15 2024-05-11T13:53:30.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:30 smithi063 ceph-mon[29809]: pgmap v619: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 6.0 MiB/s wr, 502 op/s 2024-05-11T13:53:30.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:30 smithi071 ceph-mon[33293]: pgmap v619: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 6.0 MiB/s wr, 502 op/s 2024-05-11T13:53:30.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:30 smithi165 ceph-mon[33235]: pgmap v619: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 6.0 MiB/s wr, 502 op/s 2024-05-11T13:53:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 725.0 s, 212.7 tps, lat 212.321 ms stddev 49.022 2024-05-11T13:53:32.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:32 smithi063 ceph-mon[29809]: pgmap v620: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.8 MiB/s wr, 422 op/s 2024-05-11T13:53:32.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:32 smithi071 ceph-mon[33293]: pgmap v620: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.8 MiB/s wr, 422 op/s 2024-05-11T13:53:32.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:32 smithi165 ceph-mon[33235]: pgmap v620: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.8 MiB/s wr, 422 op/s 2024-05-11T13:53:34.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:34 smithi063 ceph-mon[29809]: pgmap v621: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.6 MiB/s wr, 396 op/s 2024-05-11T13:53:34.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:34 smithi071 ceph-mon[33293]: pgmap v621: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.6 MiB/s wr, 396 op/s 2024-05-11T13:53:34.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:34 smithi165 ceph-mon[33235]: pgmap v621: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.6 MiB/s wr, 396 op/s 2024-05-11T13:53:35.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:35 smithi071 ceph-mon[33293]: pgmap v622: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.7 MiB/s wr, 413 op/s 2024-05-11T13:53:35.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:35 smithi165 ceph-mon[33235]: pgmap v622: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.7 MiB/s wr, 413 op/s 2024-05-11T13:53:35.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:35 smithi063 ceph-mon[29809]: pgmap v622: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.7 MiB/s wr, 413 op/s 2024-05-11T13:53:36.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 730.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:37.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:37 smithi063 ceph-mon[29809]: pgmap v623: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-05-11T13:53:37.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:37 smithi071 ceph-mon[33293]: pgmap v623: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-05-11T13:53:38.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:37 smithi165 ceph-mon[33235]: pgmap v623: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-05-11T13:53:39.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:39 smithi063 ceph-mon[29809]: pgmap v624: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-05-11T13:53:39.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:39 smithi071 ceph-mon[33293]: pgmap v624: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-05-11T13:53:40.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:39 smithi165 ceph-mon[33235]: pgmap v624: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-05-11T13:53:41.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 735.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:41.857 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:41 smithi165 ceph-mon[33235]: pgmap v625: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 608 KiB/s wr, 66 op/s 2024-05-11T13:53:41.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:41 smithi063 ceph-mon[29809]: pgmap v625: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 608 KiB/s wr, 66 op/s 2024-05-11T13:53:41.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:41 smithi071 ceph-mon[33293]: pgmap v625: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 608 KiB/s wr, 66 op/s 2024-05-11T13:53:43.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:43 smithi063 ceph-mon[29809]: pgmap v626: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 181 KiB/s wr, 31 op/s 2024-05-11T13:53:43.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:43 smithi071 ceph-mon[33293]: pgmap v626: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 181 KiB/s wr, 31 op/s 2024-05-11T13:53:44.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:43 smithi165 ceph-mon[33235]: pgmap v626: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 181 KiB/s wr, 31 op/s 2024-05-11T13:53:45.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:45 smithi063 ceph-mon[29809]: pgmap v627: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 263 KiB/s wr, 45 op/s 2024-05-11T13:53:45.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:45 smithi071 ceph-mon[33293]: pgmap v627: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 263 KiB/s wr, 45 op/s 2024-05-11T13:53:46.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:45 smithi165 ceph-mon[33235]: pgmap v627: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 263 KiB/s wr, 45 op/s 2024-05-11T13:53:46.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 740.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:47.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:47 smithi063 ceph-mon[29809]: pgmap v628: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 161 KiB/s wr, 28 op/s 2024-05-11T13:53:47.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:47 smithi071 ceph-mon[33293]: pgmap v628: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 161 KiB/s wr, 28 op/s 2024-05-11T13:53:48.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:47 smithi165 ceph-mon[33235]: pgmap v628: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 161 KiB/s wr, 28 op/s 2024-05-11T13:53:49.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:49 smithi063 ceph-mon[29809]: pgmap v629: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 227 KiB/s wr, 40 op/s 2024-05-11T13:53:49.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:49 smithi071 ceph-mon[33293]: pgmap v629: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 227 KiB/s wr, 40 op/s 2024-05-11T13:53:50.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:49 smithi165 ceph-mon[33235]: pgmap v629: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 227 KiB/s wr, 40 op/s 2024-05-11T13:53:51.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 745.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:51.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:51 smithi063 ceph-mon[29809]: pgmap v630: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-05-11T13:53:51.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:51 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:53:51.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:51 smithi071 ceph-mon[33293]: pgmap v630: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-05-11T13:53:51.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:51 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:53:52.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:51 smithi165 ceph-mon[33235]: pgmap v630: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-05-11T13:53:52.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:51 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:53:53.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:53 smithi063 ceph-mon[29809]: pgmap v631: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 156 KiB/s wr, 27 op/s 2024-05-11T13:53:53.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:53 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:53:53.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:53 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:53:53.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:53 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:53:53.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:53 smithi071 ceph-mon[33293]: pgmap v631: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 156 KiB/s wr, 27 op/s 2024-05-11T13:53:53.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:53 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:53:53.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:53 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:53:53.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:53 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:53:54.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:53 smithi165 ceph-mon[33235]: pgmap v631: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 156 KiB/s wr, 27 op/s 2024-05-11T13:53:54.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:53 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:53:54.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:53 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:53:54.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:53 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:53:54.480 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:53:54.952 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:53:55.804 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:55 smithi063 ceph-mon[29809]: pgmap v632: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 240 KiB/s wr, 41 op/s 2024-05-11T13:53:55.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:55 smithi071 ceph-mon[33293]: pgmap v632: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 240 KiB/s wr, 41 op/s 2024-05-11T13:53:56.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:55 smithi165 ceph-mon[33235]: pgmap v632: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 240 KiB/s wr, 41 op/s 2024-05-11T13:53:56.144 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 750.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:53:57.735 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:57 smithi063 ceph-mon[29809]: pgmap v633: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 158 KiB/s wr, 27 op/s 2024-05-11T13:53:57.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:57 smithi071 ceph-mon[33293]: pgmap v633: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 158 KiB/s wr, 27 op/s 2024-05-11T13:53:58.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:57 smithi165 ceph-mon[33235]: pgmap v633: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 158 KiB/s wr, 27 op/s 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 -- 172.21.15.63:0/4085747880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc574075780 msgr2=0x7fc574075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 --2- 172.21.15.63:0/4085747880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc574075780 0x7fc574075b60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc564009990 tx=0x7fc564009ca0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 -- 172.21.15.63:0/4085747880 shutdown_connections 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 --2- 172.21.15.63:0/4085747880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc57410ffb0 0x7fc5741123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 --2- 172.21.15.63:0/4085747880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc574075780 0x7fc574075b60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 --2- 172.21.15.63:0/4085747880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5740760a0 0x7fc57410fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 -- 172.21.15.63:0/4085747880 >> 172.21.15.63:0/4085747880 conn(0x7fc57407a6b0 msgr2=0x7fc57407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 -- 172.21.15.63:0/4085747880 shutdown_connections 2024-05-11T13:53:58.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.580+0000 7fc57b200700 1 -- 172.21.15.63:0/4085747880 wait complete. 2024-05-11T13:53:58.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 Processor -- start 2024-05-11T13:53:58.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 -- start start 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc574075780 0x7fc57419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5740760a0 0x7fc57419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc57410ffb0 0x7fc5741a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc574115880 con 0x7fc5740760a0 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc574115700 con 0x7fc57410ffb0 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.582+0000 7fc57b200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc574115a00 con 0x7fc574075780 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.583+0000 7fc56b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5740760a0 0x7fc57419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.583+0000 7fc56b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5740760a0 0x7fc57419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57448/0 (socket says 172.21.15.63:57448) 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.583+0000 7fc56b400700 1 -- 172.21.15.63:0/2165217547 learned_addr learned my addr 172.21.15.63:0/2165217547 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.583+0000 7fc573e00700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc574075780 0x7fc57419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.583+0000 7fc579200700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc57410ffb0 0x7fc5741a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:58.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc56b400700 1 -- 172.21.15.63:0/2165217547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc574075780 msgr2=0x7fc57419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc56b400700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc574075780 0x7fc57419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc56b400700 1 -- 172.21.15.63:0/2165217547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc57410ffb0 msgr2=0x7fc5741a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc56b400700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc57410ffb0 0x7fc5741a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc56b400700 1 -- 172.21.15.63:0/2165217547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5741a0d40 con 0x7fc5740760a0 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc573e00700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc574075780 0x7fc57419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc56b400700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5740760a0 0x7fc57419c270 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00d980 tx=0x7fc55c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc571600700 1 -- 172.21.15.63:0/2165217547 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c017030 con 0x7fc5740760a0 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc571600700 1 -- 172.21.15.63:0/2165217547 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc55c0044b0 con 0x7fc5740760a0 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc571600700 1 -- 172.21.15.63:0/2165217547 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0038e0 con 0x7fc5740760a0 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.584+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5741a1030 con 0x7fc5740760a0 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.585+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741a88b0 con 0x7fc5740760a0 2024-05-11T13:53:58.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.586+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57406a240 con 0x7fc5740760a0 2024-05-11T13:53:58.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.589+0000 7fc571600700 1 -- 172.21.15.63:0/2165217547 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc55c01a070 con 0x7fc5740760a0 2024-05-11T13:53:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.590+0000 7fc571600700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc5580b29f0 0x7fc5580b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:53:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.591+0000 7fc571600700 1 -- 172.21.15.63:0/2165217547 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fc55c0e2770 con 0x7fc5740760a0 2024-05-11T13:53:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.591+0000 7fc573e00700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc5580b29f0 0x7fc5580b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:53:58.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.592+0000 7fc573e00700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc5580b29f0 0x7fc5580b4eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b5c0 tx=0x7fc56401a040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:53:58.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.594+0000 7fc571600700 1 -- 172.21.15.63:0/2165217547 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc55c0a9d00 con 0x7fc5740760a0 2024-05-11T13:53:58.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.927+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc57402e570 con 0x7fc5740760a0 2024-05-11T13:53:58.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.928+0000 7fc571600700 1 -- 172.21.15.63:0/2165217547 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+11935 (secure 0 0 0) 0x7fc55c0a9600 con 0x7fc5740760a0 2024-05-11T13:53:58.917 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:53:58.918 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":75,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:53:23.861111+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34451,34472,34427,34403,24472]},"id":1}]} 2024-05-11T13:53:58.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc5580b29f0 msgr2=0x7fc5580b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc5580b29f0 0x7fc5580b4eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b5c0 tx=0x7fc56401a040 comp rx=0 tx=0).stop 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5740760a0 msgr2=0x7fc57419c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5740760a0 0x7fc57419c270 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00d980 tx=0x7fc55c00dc90 comp rx=0 tx=0).stop 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 shutdown_connections 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc5580b29f0 0x7fc5580b4eb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc57410ffb0 0x7fc5741a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc574075780 0x7fc57419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 --2- 172.21.15.63:0/2165217547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5740760a0 0x7fc57419c270 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:53:58.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.931+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 >> 172.21.15.63:0/2165217547 conn(0x7fc57407a6b0 msgr2=0x7fc574100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:53:58.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.932+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 shutdown_connections 2024-05-11T13:53:58.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:53:58.932+0000 7fc57b200700 1 -- 172.21.15.63:0/2165217547 wait complete. 2024-05-11T13:53:58.923 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 75 2024-05-11T13:53:59.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:59 smithi063 ceph-mon[29809]: pgmap v634: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 228 KiB/s wr, 40 op/s 2024-05-11T13:53:59.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:53:59 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2165217547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:53:59.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:59 smithi071 ceph-mon[33293]: pgmap v634: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 228 KiB/s wr, 40 op/s 2024-05-11T13:53:59.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:53:59 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2165217547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:54:00.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:59 smithi165 ceph-mon[33235]: pgmap v634: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 228 KiB/s wr, 40 op/s 2024-05-11T13:54:00.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:53:59 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2165217547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:54:00.204 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id 2fccb18f --release --await' 2024-05-11T13:54:00.204 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id 2fccb18f --release --await 2024-05-11T13:54:00.747 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:54:01.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 755.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:54:01.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:01 smithi063 ceph-mon[29809]: pgmap v635: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 166 KiB/s wr, 29 op/s 2024-05-11T13:54:01.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:01 smithi071 ceph-mon[33293]: pgmap v635: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 166 KiB/s wr, 29 op/s 2024-05-11T13:54:02.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:01 smithi165 ceph-mon[33235]: pgmap v635: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 166 KiB/s wr, 29 op/s 2024-05-11T13:54:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 -- 172.21.15.63:0/1828764242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 msgr2=0x7f7cf4109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 --2- 172.21.15.63:0/1828764242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf4109a60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40099c0 tx=0x7f7ce4009cd0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 -- 172.21.15.63:0/1828764242 shutdown_connections 2024-05-11T13:54:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 --2- 172.21.15.63:0/1828764242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf4103fc0 0x7f7cf4110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 --2- 172.21.15.63:0/1828764242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf4109a60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 --2- 172.21.15.63:0/1828764242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cf4103600 0x7f7cf4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 -- 172.21.15.63:0/1828764242 >> 172.21.15.63:0/1828764242 conn(0x7f7cf407a6a0 msgr2=0x7f7cf407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 -- 172.21.15.63:0/1828764242 shutdown_connections 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.262+0000 7f7cf9600700 1 -- 172.21.15.63:0/1828764242 wait complete. 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 Processor -- start 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 -- start start 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf4103600 0x7f7cf4075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cf4103fc0 0x7f7cf4075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf40762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf4113650 con 0x7f7cf4109680 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41134d0 con 0x7f7cf4103fc0 2024-05-11T13:54:03.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.264+0000 7f7cf9600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41137d0 con 0x7f7cf4103600 2024-05-11T13:54:03.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf40762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf40762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45972/0 (socket says 172.21.15.63:45972) 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 -- 172.21.15.63:0/208286864 learned_addr learned my addr 172.21.15.63:0/208286864 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf4103600 0x7f7cf4075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cebe00700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cf4103fc0 0x7f7cf4075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 -- 172.21.15.63:0/208286864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf4103600 msgr2=0x7f7cf4075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf4103600 0x7f7cf4075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 -- 172.21.15.63:0/208286864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cf4103fc0 msgr2=0x7f7cf4075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cf4103fc0 0x7f7cf4075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.265+0000 7f7cf3400700 1 -- 172.21.15.63:0/208286864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf41a81a0 con 0x7f7cf4109680 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.266+0000 7f7cf3400700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf40762d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007ee0 tx=0x7f7cec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.266+0000 7f7ceb400700 1 -- 172.21.15.63:0/208286864 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec004bb0 con 0x7f7cf4109680 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.266+0000 7f7ceb400700 1 -- 172.21.15.63:0/208286864 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cec004d50 con 0x7f7cf4109680 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.266+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf41a8490 con 0x7f7cf4109680 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.266+0000 7f7ceb400700 1 -- 172.21.15.63:0/208286864 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0058b0 con 0x7f7cf4109680 2024-05-11T13:54:03.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.266+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf41a89d0 con 0x7f7cf4109680 2024-05-11T13:54:03.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.270+0000 7f7ceb400700 1 -- 172.21.15.63:0/208286864 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7cec01a020 con 0x7f7cf4109680 2024-05-11T13:54:03.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.272+0000 7f7d04086b80 1 Processor -- start 2024-05-11T13:54:03.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.272+0000 7f7d04086b80 1 -- start start 2024-05-11T13:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7ceb400700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7cd80b2a70 0x7f7cd80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1a5a0 0x5593c0f1aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f26d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7d04086b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5593c0e6cb20 con 0x5593c0f1af60 2024-05-11T13:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7d04086b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5593c0e0a860 con 0x5593c0f1ffd0 2024-05-11T13:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7d04086b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5593c0e68ae0 con 0x5593c0f1a5a0 2024-05-11T13:54:03.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7ceb400700 1 -- 172.21.15.63:0/208286864 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f7cec0e1fe0 con 0x7f7cf4109680 2024-05-11T13:54:03.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7cebe00700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7cd80b2a70 0x7f7cd80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7cf3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f26d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.273+0000 7f7cf2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f26d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45986/0 (socket says 172.21.15.63:45986) 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 -- 172.21.15.63:0/3191744966 learned_addr learned my addr 172.21.15.63:0/3191744966 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cebe00700 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1a5a0 0x5593c0f1aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 -- 172.21.15.63:0/3191744966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1a5a0 msgr2=0x5593c0f1aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1a5a0 0x5593c0f1aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 -- 172.21.15.63:0/3191744966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 msgr2=0x5593c0f203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 -- 172.21.15.63:0/3191744966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c0db99d0 con 0x5593c0f1af60 2024-05-11T13:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cebe00700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7cd80b2a70 0x7f7cd80b4f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00b6c0 tx=0x7f7cdc012890 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.274+0000 7f7cf3400700 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f26d90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0a8bd0 tx=0x7f7cec0a8940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e8ce7accf7da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.275+0000 7f7ce3e00700 1 -- 172.21.15.63:0/3191744966 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0a95d0 con 0x5593c0f1af60 2024-05-11T13:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.275+0000 7f7ce3e00700 1 -- 172.21.15.63:0/3191744966 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cec0cb070 con 0x5593c0f1af60 2024-05-11T13:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.275+0000 7f7ce3e00700 1 -- 172.21.15.63:0/3191744966 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0a9130 con 0x5593c0f1af60 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 -- 172.21.15.63:0/3191744966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 msgr2=0x5593c0f26d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f26d90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0a8bd0 tx=0x7f7cec0a8940 comp rx=0 tx=0).stop 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 -- 172.21.15.63:0/3191744966 shutdown_connections 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f26d90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3191744966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1a5a0 0x5593c0f1aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 -- 172.21.15.63:0/3191744966 >> 172.21.15.63:0/3191744966 conn(0x5593c0e48c60 msgr2=0x5593c0f14e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 -- 172.21.15.63:0/3191744966 shutdown_connections 2024-05-11T13:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.276+0000 7f7d04086b80 1 -- 172.21.15.63:0/3191744966 wait complete. 2024-05-11T13:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 -- start start 2024-05-11T13:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 Processor -- start 2024-05-11T13:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f26a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f23df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f24330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5593c0f27390 con 0x5593c0f1af60 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5593c0dd2c20 con 0x5593c0f1a5a0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5593c0f27510 con 0x5593c0f1ffd0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.277+0000 7f7d04086b80 10 client.0 _ll_register_callbacks cb 0x5593c0e13d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f23df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f23df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45998/0 (socket says 172.21.15.63:45998) 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 -- 172.21.15.63:0/1906095176 learned_addr learned my addr 172.21.15.63:0/1906095176 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f26a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cf3400700 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f24330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 -- 172.21.15.63:0/1906095176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 msgr2=0x5593c0f24330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f24330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 -- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 msgr2=0x5593c0f26a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f26a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.278+0000 7f7cebe00700 1 -- 172.21.15.63:0/1906095176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c0e32520 con 0x5593c0f1af60 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7cebe00700 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f23df0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00f800 tx=0x7f7cdc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ecf0c4d5b3228f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7ce0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:54:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7ce0c00700 1 -- 172.21.15.63:0/1906095176 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc01c8b0 con 0x5593c0f1af60 2024-05-11T13:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7ce0c00700 1 -- 172.21.15.63:0/1906095176 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f7cdc01ca50 con 0x5593c0f1af60 2024-05-11T13:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cec0cb070 con 0x5593c0f1af60 2024-05-11T13:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7ce0c00700 1 -- 172.21.15.63:0/1906095176 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f7cdc01cc50 con 0x5593c0f1af60 2024-05-11T13:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7d04086b80 10 client.16185 fetch_fsmap learned FSMap version 75 2024-05-11T13:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.279+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5593c0f31c40 con 0x5593c0f1af60 2024-05-11T13:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7ce0c00700 1 -- 172.21.15.63:0/1906095176 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cec0cb070 con 0x5593c0f1af60 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7ce0c00700 1 -- 172.21.15.63:0/1906095176 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 75) ==== 8362+0+0 (secure 0 0 0) 0x7f7cdc01f3f0 con 0x5593c0f1af60 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7d04086b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 75 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7d04086b80 10 client.16185 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7d04086b80 20 client.16185 populate_metadata read hostname 'smithi063' 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7d04086b80 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0f34280 0x5593c0f36670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7d04086b80 4 client.16185 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.280+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593c0ee72b0 con 0x5593c0f34280 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.281+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0f34280 0x5593c0f36670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.282+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0f34280 0x5593c0f36670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b45a939c7e5e3f server_cookie=c9da707920b5cd84 in_seq=0 out_seq=0 2024-05-11T13:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.282+0000 7f7ce0c00700 10 client.16185 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7ce0c00700 1 -- 172.21.15.63:0/1906095176 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593c0ee72b0 con 0x5593c0f34280 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7ce0c00700 10 client.16185 handle_command_reply: tid=0 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 client.16185 shutdown 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 msgr2=0x5593c0f23df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f23df0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00f800 tx=0x7f7cdc01a000 comp rx=0 tx=0).stop 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 shutdown_connections 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f24330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0f34280 0x5593c0f36670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f26a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.284+0000 7f7d04086b80 1 --2- 172.21.15.63:0/1906095176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f23df0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.285+0000 7f7ce0c00700 0 client.16185 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.285+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 >> 172.21.15.63:0/1906095176 conn(0x5593c0e48c60 msgr2=0x5593c0f0f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:54:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.285+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 shutdown_connections 2024-05-11T13:54:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.285+0000 7f7d04086b80 1 -- 172.21.15.63:0/1906095176 wait complete. 2024-05-11T13:54:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.285+0000 7f7d04086b80 20 client.16185 trim_cache size 0 max 16384 2024-05-11T13:54:03.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.311+0000 7f7d04086b80 1 Processor -- start 2024-05-11T13:54:03.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.312+0000 7f7d04086b80 1 -- start start 2024-05-11T13:54:03.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.312+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f303e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.312+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.312+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f3be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.312+0000 7f7d04086b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41b18a0 con 0x5593c0f1af60 2024-05-11T13:54:03.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.312+0000 7f7d04086b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41b33e0 con 0x5593c0f1a5a0 2024-05-11T13:54:03.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.312+0000 7f7d04086b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41b1f70 con 0x5593c0f1ffd0 2024-05-11T13:54:03.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f1e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f1e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46000/0 (socket says 172.21.15.63:46000) 2024-05-11T13:54:03.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 -- 172.21.15.63:0/610357849 learned_addr learned my addr 172.21.15.63:0/610357849 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:54:03.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cf3400700 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f3be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 -- 172.21.15.63:0/610357849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 msgr2=0x5593c0f3be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f3be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 -- 172.21.15.63:0/610357849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 msgr2=0x5593c0f303e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f303e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.313+0000 7f7cebe00700 1 -- 172.21.15.63:0/610357849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c0f23bc0 con 0x5593c0f1af60 2024-05-11T13:54:03.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.314+0000 7f7cebe00700 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f1e160 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc01f860 tx=0x7f7cdc00baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e976e9e9563210f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.314+0000 7f7ce3400700 1 -- 172.21.15.63:0/610357849 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc039070 con 0x5593c0f1af60 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.314+0000 7f7ce3400700 1 -- 172.21.15.63:0/610357849 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cdc020eb0 con 0x5593c0f1af60 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.314+0000 7f7ce3400700 1 -- 172.21.15.63:0/610357849 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc005040 con 0x5593c0f1af60 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 -- 172.21.15.63:0/610357849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 msgr2=0x5593c0f1e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f1e160 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc01f860 tx=0x7f7cdc00baf0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 -- 172.21.15.63:0/610357849 shutdown_connections 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1ffd0 0x5593c0f3be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1a5a0 0x5593c0f303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 --2- 172.21.15.63:0/610357849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1af60 0x5593c0f1e160 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 -- 172.21.15.63:0/610357849 >> 172.21.15.63:0/610357849 conn(0x5593c0e48c60 msgr2=0x5593c0f1da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 -- 172.21.15.63:0/610357849 shutdown_connections 2024-05-11T13:54:03.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 -- 172.21.15.63:0/610357849 wait complete. 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.315+0000 7f7d04086b80 1 -- start start 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 1 Processor -- start 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1a5a0 0x5593c0f29af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1af60 0x5593c0f2a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f9ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41b2d10 con 0x5593c0f1a5a0 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41b2640 con 0x5593c0f1ffd0 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf41af860 con 0x5593c0f1af60 2024-05-11T13:54:03.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.316+0000 7f7d04086b80 10 client.0 _ll_register_callbacks cb 0x5593c0e13d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:54:03.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1a5a0 0x5593c0f29af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1a5a0 0x5593c0f29af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46002/0 (socket says 172.21.15.63:46002) 2024-05-11T13:54:03.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 -- 172.21.15.63:0/3246558695 learned_addr learned my addr 172.21.15.63:0/3246558695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:54:03.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cebe00700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1af60 0x5593c0f2a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf3400700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f9ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 -- 172.21.15.63:0/3246558695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1af60 msgr2=0x5593c0f2a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1af60 0x5593c0f2a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 -- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 msgr2=0x5593c0f9ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f9ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 -- 172.21.15.63:0/3246558695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c0f9f090 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf3400700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f9ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7cf2a00700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1a5a0 0x5593c0f29af0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4000c00 tx=0x7f7ce40054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c8d020068d2f34 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.317+0000 7f7ce0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.318+0000 7f7ce0c00700 1 -- 172.21.15.63:0/3246558695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40056c0 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.318+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5593c0e7a500 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.318+0000 7f7ce0c00700 1 -- 172.21.15.63:0/3246558695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f7ce400b9a0 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.318+0000 7f7ce0c00700 1 -- 172.21.15.63:0/3246558695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f7ce4034050 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.318+0000 7f7d04086b80 10 client.16191 fetch_fsmap learned FSMap version 75 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.318+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5593c0fa0410 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7ce0c00700 1 -- 172.21.15.63:0/3246558695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5593c0e7a500 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7ce0c00700 1 -- 172.21.15.63:0/3246558695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 75) ==== 8362+0+0 (secure 0 0 0) 0x7f7ce400f460 con 0x5593c0f1a5a0 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7d04086b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 75 2024-05-11T13:54:03.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7d04086b80 10 client.16191 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:54:03.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7d04086b80 20 client.16191 populate_metadata read hostname 'smithi063' 2024-05-11T13:54:03.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0fa0b20 0x5593c0fa2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:54:03.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7d04086b80 4 client.16191 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2fccb18f", "release": true, "await": true}] 2024-05-11T13:54:03.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.319+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2fccb18f", "release": true, "await": true}) -- 0x5593c0e745a0 con 0x5593c0fa0b20 2024-05-11T13:54:03.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.320+0000 7f7cebe00700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0fa0b20 0x5593c0fa2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:54:03.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.320+0000 7f7cebe00700 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0fa0b20 0x5593c0fa2f40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cb034d0761bc59 server_cookie=20557e0d1837bb39 in_seq=0 out_seq=0 2024-05-11T13:54:03.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.320+0000 7f7ce0c00700 10 client.16191 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:54:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.391+0000 7f7ce0c00700 1 -- 172.21.15.63:0/3246558695 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5593c0e745a0 con 0x5593c0fa0b20 2024-05-11T13:54:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.391+0000 7f7ce0c00700 10 client.16191 handle_command_reply: tid=0 2024-05-11T13:54:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.391+0000 7f7d04086b80 1 client.16191 shutdown 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1a5a0 msgr2=0x5593c0f29af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1a5a0 0x5593c0f29af0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4000c00 tx=0x7f7ce40054a0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 shutdown_connections 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5593c0fa0b20 0x5593c0fa2f40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593c0f1ffd0 0x5593c0f9ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5593c0f1a5a0 0x5593c0f29af0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 --2- 172.21.15.63:0/3246558695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5593c0f1af60 0x5593c0f2a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 >> 172.21.15.63:0/3246558695 conn(0x5593c0e48c60 msgr2=0x5593c0f1e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:54:03.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.393+0000 7f7ce0c00700 0 client.16191 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:54:03.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.394+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 shutdown_connections 2024-05-11T13:54:03.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.394+0000 7f7d04086b80 1 -- 172.21.15.63:0/3246558695 wait complete. 2024-05-11T13:54:03.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.394+0000 7f7d04086b80 20 client.16191 trim_cache size 0 max 16384 2024-05-11T13:54:03.384 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:54:03.384 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 75, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 60, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "2fccb18f": { 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 60, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 85.2, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:54:03.385 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:54:03.386 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.396+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7cd80b2a70 msgr2=0x7f7cd80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.396+0000 7f7cf9600700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7cd80b2a70 0x7f7cd80b4f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00b6c0 tx=0x7f7cdc012890 comp rx=0 tx=0).stop 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.396+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 msgr2=0x7f7cf40762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.396+0000 7f7cf9600700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf40762d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007ee0 tx=0x7f7cec00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 shutdown_connections 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f7cd80b2a70 0x7f7cd80b4f30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf4109680 0x7f7cf40762d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf4103600 0x7f7cf4075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 --2- 172.21.15.63:0/208286864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cf4103fc0 0x7f7cf4075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:54:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 >> 172.21.15.63:0/208286864 conn(0x7f7cf407a6a0 msgr2=0x7f7cf4102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:54:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 shutdown_connections 2024-05-11T13:54:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:54:03.397+0000 7f7cf9600700 1 -- 172.21.15.63:0/208286864 wait complete. 2024-05-11T13:54:03.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:03 smithi063 ceph-mon[29809]: pgmap v636: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 158 KiB/s wr, 27 op/s 2024-05-11T13:54:03.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:03 smithi071 ceph-mon[33293]: pgmap v636: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 158 KiB/s wr, 27 op/s 2024-05-11T13:54:04.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:03 smithi165 ceph-mon[33235]: pgmap v636: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 158 KiB/s wr, 27 op/s 2024-05-11T13:54:05.025 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2fccb18f', seconds elapsed: 49.9 2024-05-11T13:54:05.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:05 smithi071 ceph-mon[33293]: pgmap v637: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 336 KiB/s wr, 61 op/s 2024-05-11T13:54:05.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:05 smithi071 ceph-mon[33293]: osdmap e120: 12 total, 12 up, 12 in 2024-05-11T13:54:05.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:05 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:54:06.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:05 smithi165 ceph-mon[33235]: pgmap v637: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 336 KiB/s wr, 61 op/s 2024-05-11T13:54:06.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:05 smithi165 ceph-mon[33235]: osdmap e120: 12 total, 12 up, 12 in 2024-05-11T13:54:06.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:05 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:54:06.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 760.0 s, 119.1 tps, lat 6311.945 ms stddev 13006.117 2024-05-11T13:54:06.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:05 smithi063 ceph-mon[29809]: pgmap v637: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 336 KiB/s wr, 61 op/s 2024-05-11T13:54:06.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:05 smithi063 ceph-mon[29809]: osdmap e120: 12 total, 12 up, 12 in 2024-05-11T13:54:06.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:05 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:54:07.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:07 smithi071 ceph-mon[33293]: pgmap v639: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 302 KiB/s wr, 57 op/s 2024-05-11T13:54:08.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:07 smithi165 ceph-mon[33235]: pgmap v639: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 302 KiB/s wr, 57 op/s 2024-05-11T13:54:08.165 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:07 smithi063 ceph-mon[29809]: pgmap v639: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 302 KiB/s wr, 57 op/s 2024-05-11T13:54:10.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:09 smithi165 ceph-mon[33235]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-11T13:54:10.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:09 smithi063 ceph-mon[29809]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-11T13:54:10.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:09 smithi071 ceph-mon[33293]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-11T13:54:11.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:10 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:54:11.092 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:10 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:54:11.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 765.0 s, 231.6 tps, lat 434.202 ms stddev 60.159 2024-05-11T13:54:11.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:10 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:54:12.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:11 smithi165 ceph-mon[33235]: pgmap v641: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-11T13:54:12.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:11 smithi063 ceph-mon[29809]: pgmap v641: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-11T13:54:12.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:11 smithi071 ceph-mon[33293]: pgmap v641: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-11T13:54:14.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:13 smithi165 ceph-mon[33235]: pgmap v642: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-11T13:54:14.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:13 smithi063 ceph-mon[29809]: pgmap v642: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-11T13:54:14.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:13 smithi071 ceph-mon[33293]: pgmap v642: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-11T13:54:16.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 770.0 s, 221.4 tps, lat 438.772 ms stddev 82.172 2024-05-11T13:54:16.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:16 smithi063 ceph-mon[29809]: pgmap v643: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 3.3 MiB/s wr, 356 op/s 2024-05-11T13:54:16.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:16 smithi063 ceph-mon[29809]: osdmap e121: 12 total, 12 up, 12 in 2024-05-11T13:54:16.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:16 smithi071 ceph-mon[33293]: pgmap v643: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 3.3 MiB/s wr, 356 op/s 2024-05-11T13:54:16.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:16 smithi071 ceph-mon[33293]: osdmap e121: 12 total, 12 up, 12 in 2024-05-11T13:54:16.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:16 smithi165 ceph-mon[33235]: pgmap v643: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 3.3 MiB/s wr, 356 op/s 2024-05-11T13:54:16.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:16 smithi165 ceph-mon[33235]: osdmap e121: 12 total, 12 up, 12 in 2024-05-11T13:54:18.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:18 smithi063 ceph-mon[29809]: pgmap v645: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 3.3 MiB/s wr, 356 op/s 2024-05-11T13:54:18.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:18 smithi071 ceph-mon[33293]: pgmap v645: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 3.3 MiB/s wr, 356 op/s 2024-05-11T13:54:18.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:18 smithi165 ceph-mon[33235]: pgmap v645: 129 pgs: 129 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 3.3 MiB/s wr, 356 op/s 2024-05-11T13:54:20.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:20 smithi063 ceph-mon[29809]: pgmap v646: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-05-11T13:54:20.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:20 smithi071 ceph-mon[33293]: pgmap v646: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-05-11T13:54:20.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:20 smithi165 ceph-mon[33235]: pgmap v646: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-05-11T13:54:21.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 775.0 s, 224.4 tps, lat 463.242 ms stddev 151.758 2024-05-11T13:54:22.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:22 smithi063 ceph-mon[29809]: pgmap v647: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:22.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:22 smithi071 ceph-mon[33293]: pgmap v647: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:22.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:22 smithi165 ceph-mon[33235]: pgmap v647: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:24.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:24 smithi063 ceph-mon[29809]: pgmap v648: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:24.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:24 smithi071 ceph-mon[33293]: pgmap v648: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:24.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:24 smithi165 ceph-mon[33235]: pgmap v648: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 780.0 s, 246.2 tps, lat 401.655 ms stddev 93.220 2024-05-11T13:54:26.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:26 smithi063 ceph-mon[29809]: pgmap v649: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-05-11T13:54:26.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:26 smithi071 ceph-mon[33293]: pgmap v649: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-05-11T13:54:26.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:26 smithi165 ceph-mon[33235]: pgmap v649: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-05-11T13:54:28.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:28 smithi063 ceph-mon[29809]: pgmap v650: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-05-11T13:54:28.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:28 smithi071 ceph-mon[33293]: pgmap v650: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-05-11T13:54:28.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:28 smithi165 ceph-mon[33235]: pgmap v650: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-05-11T13:54:30.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:30 smithi063 ceph-mon[29809]: pgmap v651: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-05-11T13:54:30.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:30 smithi071 ceph-mon[33293]: pgmap v651: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-05-11T13:54:30.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:30 smithi165 ceph-mon[33235]: pgmap v651: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-05-11T13:54:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 785.0 s, 232.4 tps, lat 428.256 ms stddev 57.100 2024-05-11T13:54:32.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:32 smithi063 ceph-mon[29809]: pgmap v652: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-05-11T13:54:32.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:32 smithi071 ceph-mon[33293]: pgmap v652: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-05-11T13:54:32.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:32 smithi165 ceph-mon[33235]: pgmap v652: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-05-11T13:54:34.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:34 smithi063 ceph-mon[29809]: pgmap v653: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 289 op/s 2024-05-11T13:54:34.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:34 smithi071 ceph-mon[33293]: pgmap v653: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 289 op/s 2024-05-11T13:54:34.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:34 smithi165 ceph-mon[33235]: pgmap v653: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 289 op/s 2024-05-11T13:54:35.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:35 smithi063 ceph-mon[29809]: pgmap v654: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:35.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:35 smithi071 ceph-mon[33293]: pgmap v654: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:35.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:35 smithi165 ceph-mon[33235]: pgmap v654: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-11T13:54:36.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 790.0 s, 207.6 tps, lat 479.076 ms stddev 95.837 2024-05-11T13:54:37.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:37 smithi063 ceph-mon[29809]: pgmap v655: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:54:37.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:37 smithi071 ceph-mon[33293]: pgmap v655: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:54:38.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:37 smithi165 ceph-mon[33235]: pgmap v655: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:54:39.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:39 smithi063 ceph-mon[29809]: pgmap v656: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-05-11T13:54:39.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:39 smithi071 ceph-mon[33293]: pgmap v656: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-05-11T13:54:40.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:39 smithi165 ceph-mon[33235]: pgmap v656: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-05-11T13:54:41.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 795.0 s, 200.8 tps, lat 497.840 ms stddev 164.486 2024-05-11T13:54:41.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:41 smithi063 ceph-mon[29809]: pgmap v657: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-05-11T13:54:41.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:41 smithi071 ceph-mon[33293]: pgmap v657: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-05-11T13:54:42.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:41 smithi165 ceph-mon[33235]: pgmap v657: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-05-11T13:54:43.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:43 smithi063 ceph-mon[29809]: pgmap v658: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-05-11T13:54:43.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:43 smithi071 ceph-mon[33293]: pgmap v658: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-05-11T13:54:44.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:43 smithi165 ceph-mon[33235]: pgmap v658: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-05-11T13:54:45.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:45 smithi063 ceph-mon[29809]: pgmap v659: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-11T13:54:45.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:45 smithi071 ceph-mon[33293]: pgmap v659: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-11T13:54:46.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:45 smithi165 ceph-mon[33235]: pgmap v659: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-11T13:54:46.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 800.0 s, 197.8 tps, lat 511.933 ms stddev 167.788 2024-05-11T13:54:47.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:47 smithi063 ceph-mon[29809]: pgmap v660: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-05-11T13:54:47.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:47 smithi071 ceph-mon[33293]: pgmap v660: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-05-11T13:54:48.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:47 smithi165 ceph-mon[33235]: pgmap v660: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-05-11T13:54:49.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:49 smithi063 ceph-mon[29809]: pgmap v661: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-11T13:54:49.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:49 smithi071 ceph-mon[33293]: pgmap v661: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-11T13:54:50.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:49 smithi165 ceph-mon[33235]: pgmap v661: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-11T13:54:51.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 805.0 s, 249.2 tps, lat 399.767 ms stddev 60.008 2024-05-11T13:54:51.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:51 smithi063 ceph-mon[29809]: pgmap v662: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 3.1 MiB/s wr, 349 op/s 2024-05-11T13:54:51.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:51 smithi071 ceph-mon[33293]: pgmap v662: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 3.1 MiB/s wr, 349 op/s 2024-05-11T13:54:52.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:51 smithi165 ceph-mon[33235]: pgmap v662: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 3.1 MiB/s wr, 349 op/s 2024-05-11T13:54:53.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:53 smithi063 ceph-mon[29809]: pgmap v663: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-11T13:54:53.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:53 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:54:53.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:53 smithi071 ceph-mon[33293]: pgmap v663: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-11T13:54:53.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:53 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:54:54.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:53 smithi165 ceph-mon[33235]: pgmap v663: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-11T13:54:54.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:53 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:54:55.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:55 smithi063 ceph-mon[29809]: pgmap v664: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.0 MiB/s wr, 472 op/s 2024-05-11T13:54:55.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:55 smithi071 ceph-mon[33293]: pgmap v664: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.0 MiB/s wr, 472 op/s 2024-05-11T13:54:56.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:55 smithi165 ceph-mon[33235]: pgmap v664: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.0 MiB/s wr, 472 op/s 2024-05-11T13:54:56.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 810.0 s, 230.8 tps, lat 436.496 ms stddev 70.080 2024-05-11T13:54:57.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:57 smithi063 ceph-mon[29809]: pgmap v665: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-05-11T13:54:57.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:57 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:54:57.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:57 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:54:57.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:57 smithi071 ceph-mon[33293]: pgmap v665: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-05-11T13:54:57.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:57 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:54:57.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:57 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:54:58.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:57 smithi165 ceph-mon[33235]: pgmap v665: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-05-11T13:54:58.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:57 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:54:58.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:57 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:54:58.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:58 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:58 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:54:58.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:58 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:54:58.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:58 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:58.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:58 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:54:58.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:58 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:54:59.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:59.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:59.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:59.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:58 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:54:59.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:58 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi165 to 176.4M 2024-05-11T13:54:59.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:58 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi165 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-11T13:54:59.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:54:59 smithi063 ceph-mon[29809]: pgmap v666: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-05-11T13:54:59.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:54:59 smithi071 ceph-mon[33293]: pgmap v666: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-05-11T13:55:00.056 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:54:59 smithi165 ceph-mon[33235]: pgmap v666: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-05-11T13:55:01.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 815.0 s, 210.4 tps, lat 464.652 ms stddev 98.968 2024-05-11T13:55:01.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:01.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:01 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:01.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:01.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:01 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:01.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:01.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:01 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: pgmap v667: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-05-11T13:55:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:55:02.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:55:02.555 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: pgmap v667: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:55:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: pgmap v667: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:55:02.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:55:03.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:03 smithi165 ceph-mon[33235]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:55:03.554 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:03 smithi165 ceph-mon[33235]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:55:03.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:03 smithi063 ceph-mon[29809]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:55:03.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:03 smithi063 ceph-mon[29809]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:55:03.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:03 smithi071 ceph-mon[33293]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-11T13:55:03.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:03 smithi071 ceph-mon[33293]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-11T13:55:04.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:04 smithi165 ceph-mon[33235]: pgmap v668: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.8 MiB/s wr, 333 op/s 2024-05-11T13:55:04.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:04 smithi063 ceph-mon[29809]: pgmap v668: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.8 MiB/s wr, 333 op/s 2024-05-11T13:55:04.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:04 smithi071 ceph-mon[33293]: pgmap v668: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.8 MiB/s wr, 333 op/s 2024-05-11T13:55:05.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:05 smithi165 ceph-mon[33235]: osdmap e122: 12 total, 12 up, 12 in 2024-05-11T13:55:05.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:05 smithi063 ceph-mon[29809]: osdmap e122: 12 total, 12 up, 12 in 2024-05-11T13:55:05.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:05 smithi071 ceph-mon[33293]: osdmap e122: 12 total, 12 up, 12 in 2024-05-11T13:55:06.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 820.0 s, 183.6 tps, lat 543.323 ms stddev 169.127 2024-05-11T13:55:06.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:06 smithi063 ceph-mon[29809]: pgmap v670: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-05-11T13:55:06.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:06 smithi071 ceph-mon[33293]: pgmap v670: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-05-11T13:55:06.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:06 smithi165 ceph-mon[33235]: pgmap v670: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-05-11T13:55:08.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:08 smithi063 ceph-mon[29809]: pgmap v671: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-05-11T13:55:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:08 smithi071 ceph-mon[33293]: pgmap v671: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-05-11T13:55:08.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:08 smithi165 ceph-mon[33235]: pgmap v671: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-05-11T13:55:10.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:10 smithi063 ceph-mon[29809]: pgmap v672: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-11T13:55:10.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:10 smithi071 ceph-mon[33293]: pgmap v672: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-11T13:55:10.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:10 smithi165 ceph-mon[33235]: pgmap v672: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-11T13:55:11.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 825.0 s, 206.0 tps, lat 492.601 ms stddev 74.551 2024-05-11T13:55:12.606 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:12 smithi165 ceph-mon[33235]: pgmap v673: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:55:12.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:12 smithi063 ceph-mon[29809]: pgmap v673: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:55:12.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:12 smithi071 ceph-mon[33293]: pgmap v673: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:55:14.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:14 smithi063 ceph-mon[29809]: pgmap v674: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:55:14.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:14 smithi071 ceph-mon[33293]: pgmap v674: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:55:14.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:14 smithi165 ceph-mon[33235]: pgmap v674: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-05-11T13:55:16.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 830.0 s, 243.4 tps, lat 416.575 ms stddev 61.919 2024-05-11T13:55:16.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:16 smithi063 ceph-mon[29809]: pgmap v675: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 2.2 MiB/s wr, 264 op/s 2024-05-11T13:55:16.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:16 smithi063 ceph-mon[29809]: osdmap e123: 12 total, 12 up, 12 in 2024-05-11T13:55:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:16 smithi071 ceph-mon[33293]: pgmap v675: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 2.2 MiB/s wr, 264 op/s 2024-05-11T13:55:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:16 smithi071 ceph-mon[33293]: osdmap e123: 12 total, 12 up, 12 in 2024-05-11T13:55:16.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:16 smithi165 ceph-mon[33235]: pgmap v675: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 2.2 MiB/s wr, 264 op/s 2024-05-11T13:55:16.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:16 smithi165 ceph-mon[33235]: osdmap e123: 12 total, 12 up, 12 in 2024-05-11T13:55:18.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:18 smithi063 ceph-mon[29809]: pgmap v677: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-05-11T13:55:18.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:18 smithi071 ceph-mon[33293]: pgmap v677: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-05-11T13:55:19.034 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:18 smithi165 ceph-mon[33235]: pgmap v677: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-05-11T13:55:20.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:20 smithi063 ceph-mon[29809]: pgmap v678: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-11T13:55:20.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:20 smithi071 ceph-mon[33293]: pgmap v678: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-11T13:55:20.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:20 smithi165 ceph-mon[33235]: pgmap v678: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-11T13:55:21.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 835.0 s, 251.6 tps, lat 393.800 ms stddev 56.483 2024-05-11T13:55:22.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:22 smithi063 ceph-mon[29809]: pgmap v679: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-05-11T13:55:22.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:22 smithi071 ceph-mon[33293]: pgmap v679: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-05-11T13:55:22.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:22 smithi165 ceph-mon[33235]: pgmap v679: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-05-11T13:55:24.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:24 smithi063 ceph-mon[29809]: pgmap v680: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-05-11T13:55:24.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:24 smithi071 ceph-mon[33293]: pgmap v680: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-05-11T13:55:24.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:24 smithi165 ceph-mon[33235]: pgmap v680: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-05-11T13:55:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 840.0 s, 237.8 tps, lat 424.267 ms stddev 112.464 2024-05-11T13:55:26.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:26 smithi063 ceph-mon[29809]: pgmap v681: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-11T13:55:26.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:26 smithi071 ceph-mon[33293]: pgmap v681: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-11T13:55:26.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:26 smithi165 ceph-mon[33235]: pgmap v681: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-11T13:55:28.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:28 smithi063 ceph-mon[29809]: pgmap v682: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-05-11T13:55:28.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:28 smithi071 ceph-mon[33293]: pgmap v682: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-05-11T13:55:28.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:28 smithi165 ceph-mon[33235]: pgmap v682: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-05-11T13:55:30.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:30 smithi063 ceph-mon[29809]: pgmap v683: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.8 MiB/s wr, 443 op/s 2024-05-11T13:55:30.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:30 smithi071 ceph-mon[33293]: pgmap v683: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.8 MiB/s wr, 443 op/s 2024-05-11T13:55:30.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:30 smithi165 ceph-mon[33235]: pgmap v683: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.8 MiB/s wr, 443 op/s 2024-05-11T13:55:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 845.0 s, 211.2 tps, lat 470.983 ms stddev 148.941 2024-05-11T13:55:32.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:32 smithi063 ceph-mon[29809]: pgmap v684: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-05-11T13:55:32.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:32 smithi071 ceph-mon[33293]: pgmap v684: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-05-11T13:55:32.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:32 smithi165 ceph-mon[33235]: pgmap v684: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-05-11T13:55:34.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:34 smithi071 ceph-mon[33293]: pgmap v685: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-05-11T13:55:34.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:34 smithi165 ceph-mon[33235]: pgmap v685: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-05-11T13:55:34.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:34 smithi063 ceph-mon[29809]: pgmap v685: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-05-11T13:55:36.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 850.0 s, 210.8 tps, lat 473.622 ms stddev 123.610 2024-05-11T13:55:36.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:36 smithi071 ceph-mon[33293]: pgmap v686: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-05-11T13:55:36.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:36 smithi165 ceph-mon[33235]: pgmap v686: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-05-11T13:55:36.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:36 smithi063 ceph-mon[29809]: pgmap v686: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-05-11T13:55:37.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:37 smithi165 ceph-mon[33235]: pgmap v687: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 2.4 MiB/s wr, 296 op/s 2024-05-11T13:55:37.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:37 smithi063 ceph-mon[29809]: pgmap v687: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 2.4 MiB/s wr, 296 op/s 2024-05-11T13:55:37.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:37 smithi071 ceph-mon[33293]: pgmap v687: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 2.4 MiB/s wr, 296 op/s 2024-05-11T13:55:38.440 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-11T13:55:38.441 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.85 2024-05-11T13:55:38.441 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:55:39.117 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:55:39.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:39 smithi063 ceph-mon[29809]: pgmap v688: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-05-11T13:55:39.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:39 smithi071 ceph-mon[33293]: pgmap v688: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-05-11T13:55:40.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:39 smithi165 ceph-mon[33235]: pgmap v688: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-05-11T13:55:41.140 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 855.0 s, 203.7 tps, lat 486.125 ms stddev 94.842 2024-05-11T13:55:41.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:41 smithi063 ceph-mon[29809]: pgmap v689: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.9 MiB/s wr, 340 op/s 2024-05-11T13:55:41.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:41 smithi071 ceph-mon[33293]: pgmap v689: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.9 MiB/s wr, 340 op/s 2024-05-11T13:55:42.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:41 smithi165 ceph-mon[33235]: pgmap v689: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.9 MiB/s wr, 340 op/s 2024-05-11T13:55:42.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.009+0000 7f8819800700 1 -- 172.21.15.63:0/666772582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 msgr2=0x7f881410a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:42.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.009+0000 7f8819800700 1 --2- 172.21.15.63:0/666772582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 0x7f881410a1c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f88040099c0 tx=0x7f8804009cd0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.010+0000 7f8819800700 1 -- 172.21.15.63:0/666772582 shutdown_connections 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.010+0000 7f8819800700 1 --2- 172.21.15.63:0/666772582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f881410eec0 0x7f8814116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.010+0000 7f8819800700 1 --2- 172.21.15.63:0/666772582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 0x7f881410a1c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.010+0000 7f8819800700 1 --2- 172.21.15.63:0/666772582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 0x7f881410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.010+0000 7f8819800700 1 -- 172.21.15.63:0/666772582 >> 172.21.15.63:0/666772582 conn(0x7f8814076d20 msgr2=0x7f8814108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.010+0000 7f8819800700 1 -- 172.21.15.63:0/666772582 shutdown_connections 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.010+0000 7f8819800700 1 -- 172.21.15.63:0/666772582 wait complete. 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.012+0000 7f8819800700 1 Processor -- start 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.012+0000 7f8819800700 1 -- start start 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.012+0000 7f8819800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 0x7f88141a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.012+0000 7f8819800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 0x7f88141a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.012+0000 7f8819800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f881410eec0 0x7f88141a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8819800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8814119d20 con 0x7f8814109de0 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8819800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8814119ba0 con 0x7f881410a790 2024-05-11T13:55:42.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8819800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8814119ea0 con 0x7f881410eec0 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8812000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 0x7f88141a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8812000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 0x7f88141a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:36292/0 (socket says 172.21.15.63:36292) 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8812000700 1 -- 172.21.15.63:0/1926977649 learned_addr learned my addr 172.21.15.63:0/1926977649 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8812a00700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 0x7f88141a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.013+0000 7f8813400700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f881410eec0 0x7f88141a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.014+0000 7f8812000700 1 -- 172.21.15.63:0/1926977649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f881410eec0 msgr2=0x7f88141a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.014+0000 7f8812000700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f881410eec0 0x7f88141a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.014+0000 7f8812000700 1 -- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 msgr2=0x7f88141a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.014+0000 7f8812000700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 0x7f88141a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.014+0000 7f8812000700 1 -- 172.21.15.63:0/1926977649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88141a5240 con 0x7f881410a790 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.015+0000 7f8812000700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 0x7f88141a0770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f880000ea30 tx=0x7f880000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.015+0000 7f880b400700 1 -- 172.21.15.63:0/1926977649 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880000cc90 con 0x7f881410a790 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.015+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141a5530 con 0x7f881410a790 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.015+0000 7f880b400700 1 -- 172.21.15.63:0/1926977649 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f880000ce30 con 0x7f881410a790 2024-05-11T13:55:42.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.015+0000 7f880b400700 1 -- 172.21.15.63:0/1926977649 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800003e20 con 0x7f881410a790 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.015+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141acdb0 con 0x7f881410a790 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.018+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881406a240 con 0x7f881410a790 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.019+0000 7f880b400700 1 -- 172.21.15.63:0/1926977649 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8800004000 con 0x7f881410a790 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.021+0000 7f880b400700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87f80b2aa0 0x7f87f80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.021+0000 7f880b400700 1 -- 172.21.15.63:0/1926977649 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f88000e2d80 con 0x7f881410a790 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.022+0000 7f8812a00700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87f80b2aa0 0x7f87f80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.023+0000 7f8812a00700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87f80b2aa0 0x7f87f80b4f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8804006010 tx=0x7f88040036e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:42.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.028+0000 7f880b400700 1 -- 172.21.15.63:0/1926977649 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88000aa310 con 0x7f881410a790 2024-05-11T13:55:42.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.490+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f881402e570 con 0x7f881410a790 2024-05-11T13:55:42.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.492+0000 7f880b400700 1 -- 172.21.15.63:0/1926977649 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+11942 (secure 0 0 0) 0x7f88000a9c10 con 0x7f881410a790 2024-05-11T13:55:42.481 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:55:42.481 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:54:09.149495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:55:42.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.494+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87f80b2aa0 msgr2=0x7f87f80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:42.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.494+0000 7f8819800700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87f80b2aa0 0x7f87f80b4f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8804006010 tx=0x7f88040036e0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.494+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 msgr2=0x7f88141a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.494+0000 7f8819800700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 0x7f88141a0770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f880000ea30 tx=0x7f880000edf0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 shutdown_connections 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87f80b2aa0 0x7f87f80b4f60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f881410eec0 0x7f88141a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8814109de0 0x7f88141a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 --2- 172.21.15.63:0/1926977649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f881410a790 0x7f88141a0770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 >> 172.21.15.63:0/1926977649 conn(0x7f8814076d20 msgr2=0x7f881410db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 shutdown_connections 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:42.495+0000 7f8819800700 1 -- 172.21.15.63:0/1926977649 wait complete. 2024-05-11T13:55:42.484 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 77 2024-05-11T13:55:42.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:42 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1926977649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:55:42.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:42 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1926977649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:55:42.954 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1926977649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:55:43.867 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:43 smithi063 ceph-mon[29809]: pgmap v690: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-05-11T13:55:43.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:43 smithi071 ceph-mon[33293]: pgmap v690: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-05-11T13:55:44.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:43 smithi165 ceph-mon[33235]: pgmap v690: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-05-11T13:55:44.301 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 77.85 --await-for 42.0' 2024-05-11T13:55:44.301 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 77.85 --await-for 42.0 2024-05-11T13:55:44.931 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:55:45.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:45 smithi071 ceph-mon[33293]: pgmap v691: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-05-11T13:55:46.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:45 smithi165 ceph-mon[33235]: pgmap v691: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-05-11T13:55:46.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 860.0 s, 188.9 tps, lat 536.601 ms stddev 154.510 2024-05-11T13:55:46.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:45 smithi063 ceph-mon[29809]: pgmap v691: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-05-11T13:55:47.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:47 smithi071 ceph-mon[33293]: pgmap v692: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-05-11T13:55:48.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:47 smithi165 ceph-mon[33235]: pgmap v692: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-05-11T13:55:48.080 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:47 smithi063 ceph-mon[29809]: pgmap v692: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-05-11T13:55:49.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:49 smithi063 ceph-mon[29809]: pgmap v693: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-11T13:55:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 -- 172.21.15.63:0/827035024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8108910 msgr2=0x7f5ac8110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 --2- 172.21.15.63:0/827035024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8108910 0x7f5ac8110490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000b0d0 tx=0x7f5ac000b3e0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 -- 172.21.15.63:0/827035024 shutdown_connections 2024-05-11T13:55:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 --2- 172.21.15.63:0/827035024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8108910 0x7f5ac8110490 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 --2- 172.21.15.63:0/827035024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac806b490 0x7f5ac806b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 --2- 172.21.15.63:0/827035024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac806be40 0x7f5ac81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 -- 172.21.15.63:0/827035024 >> 172.21.15.63:0/827035024 conn(0x7f5ac80793f0 msgr2=0x7f5ac8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/827035024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac806b490 0x7f5ac806b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.550+0000 7f5acec00700 1 -- 172.21.15.63:0/827035024 shutdown_connections 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.551+0000 7f5acec00700 1 -- 172.21.15.63:0/827035024 wait complete. 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.552+0000 7f5acec00700 1 Processor -- start 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.553+0000 7f5acec00700 1 -- start start 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.553+0000 7f5acec00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac806be40 0x7f5ac8102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5acec00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac8108910 0x7f5ac8102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5acec00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8102fa0 0x7f5ac81a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5acec00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8113680 con 0x7f5ac8108910 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5acec00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8113500 con 0x7f5ac8102fa0 2024-05-11T13:55:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5acec00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8113800 con 0x7f5ac806be40 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac8108910 0x7f5ac8102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac8108910 0x7f5ac8102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39936/0 (socket says 172.21.15.63:39936) 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5ac7400700 1 -- 172.21.15.63:0/1118507736 learned_addr learned my addr 172.21.15.63:0/1118507736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.554+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac806be40 0x7f5ac8102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.555+0000 7f5ac7400700 1 -- 172.21.15.63:0/1118507736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac806be40 msgr2=0x7f5ac8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.555+0000 7f5ac7400700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac806be40 0x7f5ac8102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.555+0000 7f5ac7400700 1 -- 172.21.15.63:0/1118507736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8102fa0 msgr2=0x7f5ac81a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.555+0000 7f5accc00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8102fa0 0x7f5ac81a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.555+0000 7f5ac7400700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8102fa0 0x7f5ac81a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.555+0000 7f5ac7400700 1 -- 172.21.15.63:0/1118507736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac81a84a0 con 0x7f5ac8108910 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.555+0000 7f5accc00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8102fa0 0x7f5ac81a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.556+0000 7f5ac7400700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac8108910 0x7f5ac8102a60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800e9a0 tx=0x7f5ab800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.556+0000 7f5ac4c00700 1 -- 172.21.15.63:0/1118507736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab800cbd0 con 0x7f5ac8108910 2024-05-11T13:55:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.556+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81a8790 con 0x7f5ac8108910 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.556+0000 7f5ac4c00700 1 -- 172.21.15.63:0/1118507736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab800cd70 con 0x7f5ac8108910 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.556+0000 7f5ac4c00700 1 -- 172.21.15.63:0/1118507736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8018680 con 0x7f5ac8108910 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.556+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81a8d20 con 0x7f5ac8108910 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.560+0000 7f5ac4c00700 1 -- 172.21.15.63:0/1118507736 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5ab8010950 con 0x7f5ac8108910 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.562+0000 7f5ac4c00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5aac0b2b20 0x7f5aac0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.562+0000 7f5ac4c00700 1 -- 172.21.15.63:0/1118507736 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f5ab80e2d20 con 0x7f5ac8108910 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.562+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5aac0b2b20 0x7f5aac0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.563+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5aac0b2b20 0x7f5aac0b4fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005fd0 tx=0x7f5ab4005da0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.982+0000 7f5ad9524b80 1 Processor -- start 2024-05-11T13:55:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.982+0000 7f5ad9524b80 1 -- start start 2024-05-11T13:55:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.982+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d33e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.982+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33ea380 0x5613d33e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.982+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e4560 0x5613d33e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5613d332dbb0 con 0x5613d33e9a60 2024-05-11T13:55:49.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5613d32cb8f0 con 0x5613d33ea380 2024-05-11T13:55:49.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5613d3329b70 con 0x5613d33e4560 2024-05-11T13:55:49.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33ea380 0x5613d33e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33ea380 0x5613d33e3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:36920/0 (socket says 172.21.15.63:36920) 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 -- 172.21.15.63:0/202886971 learned_addr learned my addr 172.21.15.63:0/202886971 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5accc00700 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e4560 0x5613d33e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 -- 172.21.15.63:0/202886971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e4560 msgr2=0x5613d33e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e4560 0x5613d33e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 -- 172.21.15.63:0/202886971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 msgr2=0x5613d33e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d33e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5ac7400700 1 -- 172.21.15.63:0/202886971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d327aa60 con 0x5613d33ea380 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.983+0000 7f5accc00700 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e4560 0x5613d33e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.984+0000 7f5ac7400700 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33ea380 0x5613d33e3f40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80a9b40 tx=0x7f5ab80a9ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4f6e6574748a97 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.984+0000 7f5abd600700 1 -- 172.21.15.63:0/202886971 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80cae60 con 0x5613d33ea380 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.984+0000 7f5abd600700 1 -- 172.21.15.63:0/202886971 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab80aa2b0 con 0x5613d33ea380 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.984+0000 7f5abd600700 1 -- 172.21.15.63:0/202886971 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80b27e0 con 0x5613d33ea380 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 -- 172.21.15.63:0/202886971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33ea380 msgr2=0x5613d33e3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:49.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33ea380 0x5613d33e3f40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80a9b40 tx=0x7f5ab80a9ab0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 -- 172.21.15.63:0/202886971 shutdown_connections 2024-05-11T13:55:49.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e4560 0x5613d33e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d33e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/202886971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33ea380 0x5613d33e3f40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 -- 172.21.15.63:0/202886971 >> 172.21.15.63:0/202886971 conn(0x5613d3309cf0 msgr2=0x5613d33d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 -- 172.21.15.63:0/202886971 shutdown_connections 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.985+0000 7f5ad9524b80 1 -- 172.21.15.63:0/202886971 wait complete. 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.986+0000 7f5ad9524b80 1 -- start start 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.987+0000 7f5ad9524b80 1 Processor -- start 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.988+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.988+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e9a60 0x5613d33ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.988+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33ea380 0x5613d33edc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.988+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5613d33f0e20 con 0x5613d33ea380 2024-05-11T13:55:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.988+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5613d3293cb0 con 0x5613d33e4560 2024-05-11T13:55:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.988+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5613d33f0fa0 con 0x5613d33e9a60 2024-05-11T13:55:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.988+0000 7f5ad9524b80 10 client.0 _ll_register_callbacks cb 0x5613d32c1510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:55:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.989+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e9a60 0x5613d33ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.989+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e9a60 0x5613d33ed680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:45934/0 (socket says 172.21.15.63:45934) 2024-05-11T13:55:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.989+0000 7f5ac7400700 1 -- 172.21.15.63:0/1262401632 learned_addr learned my addr 172.21.15.63:0/1262401632 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:55:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.989+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.989+0000 7f5accc00700 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33ea380 0x5613d33edc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.990+0000 7f5ac7400700 1 -- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 msgr2=0x5613d33f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:55:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.990+0000 7f5ac7400700 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.990+0000 7f5ac7400700 1 -- 172.21.15.63:0/1262401632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33ea380 msgr2=0x5613d33edc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:55:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.990+0000 7f5ac7400700 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33ea380 0x5613d33edc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.990+0000 7f5ac7400700 1 -- 172.21.15.63:0/1262401632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d32f35b0 con 0x5613d33e9a60 2024-05-11T13:55:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.991+0000 7f5ac7400700 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e9a60 0x5613d33ed680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800e970 tx=0x7f5ab80a9c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10764bf04b0e578f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.991+0000 7f5ab2000700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-05-11T13:55:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.991+0000 7f5ab2000700 1 -- 172.21.15.63:0/1262401632 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80cae60 con 0x5613d33e9a60 2024-05-11T13:55:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.992+0000 7f5ab2000700 1 -- 172.21.15.63:0/1262401632 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f5ab80b2790 con 0x5613d33e9a60 2024-05-11T13:55:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.992+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab80aa2b0 con 0x5613d33e9a60 2024-05-11T13:55:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.992+0000 7f5ab2000700 1 -- 172.21.15.63:0/1262401632 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f5ab80b2a10 con 0x5613d33e9a60 2024-05-11T13:55:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.992+0000 7f5ad9524b80 10 client.25756 fetch_fsmap learned FSMap version 77 2024-05-11T13:55:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.992+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5613d33f0e20 con 0x5613d33e9a60 2024-05-11T13:55:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.993+0000 7f5ab2000700 1 -- 172.21.15.63:0/1262401632 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab80aa2b0 con 0x5613d33e9a60 2024-05-11T13:55:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.993+0000 7f5ab2000700 1 -- 172.21.15.63:0/1262401632 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f5ab8086020 con 0x5613d33e9a60 2024-05-11T13:55:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.993+0000 7f5ad9524b80 10 client.25756 fetch_fsmap finished waiting for FSMap version 77 2024-05-11T13:55:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.993+0000 7f5ad9524b80 10 client.25756 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:55:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.993+0000 7f5ad9524b80 20 client.25756 populate_metadata read hostname 'smithi063' 2024-05-11T13:55:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.993+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d340eb30 0x5613d3410f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.993+0000 7f5ad9524b80 4 client.25756 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:55:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.994+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d33a81f0 con 0x5613d340eb30 2024-05-11T13:55:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.994+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d340eb30 0x5613d3410f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:49.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.996+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d340eb30 0x5613d3410f20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85a6421cb61c63e server_cookie=4d99c64a1eec8596 in_seq=0 out_seq=0 2024-05-11T13:55:49.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.996+0000 7f5ab2000700 10 client.25756 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:55:49.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.997+0000 7f5ab2000700 1 -- 172.21.15.63:0/1262401632 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5613d33a81f0 con 0x5613d340eb30 2024-05-11T13:55:49.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.997+0000 7f5ab2000700 10 client.25756 handle_command_reply: tid=0 2024-05-11T13:55:49.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.997+0000 7f5ad9524b80 1 client.25756 shutdown 2024-05-11T13:55:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e9a60 msgr2=0x5613d33ed680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e9a60 0x5613d33ed680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800e970 tx=0x7f5ab80a9c30 comp rx=0 tx=0).stop 2024-05-11T13:55:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 shutdown_connections 2024-05-11T13:55:49.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33ea380 0x5613d33edc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d340eb30 0x5613d3410f20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ab2000700 0 client.25756 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:55:49.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/1262401632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33e9a60 0x5613d33ed680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:49.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.998+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 >> 172.21.15.63:0/1262401632 conn(0x5613d3309cf0 msgr2=0x5613d33da720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:49.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.999+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 shutdown_connections 2024-05-11T13:55:49.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.999+0000 7f5ad9524b80 1 -- 172.21.15.63:0/1262401632 wait complete. 2024-05-11T13:55:49.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:49.999+0000 7f5ad9524b80 20 client.25756 trim_cache size 0 max 16384 2024-05-11T13:55:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.040+0000 7f5ad9524b80 1 Processor -- start 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.040+0000 7f5ad9524b80 1 -- start start 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d3416640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d3416b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d33f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80718b0 con 0x5613d33e9a60 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80734b0 con 0x5613d33e4560 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8071f80 con 0x5613d33ea380 2024-05-11T13:55:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d3416b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d3416b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55402/0 (socket says 172.21.15.63:55402) 2024-05-11T13:55:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.041+0000 7f5ac7400700 1 -- 172.21.15.63:0/311056790 learned_addr learned my addr 172.21.15.63:0/311056790 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:55:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.042+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d3416640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.042+0000 7f5ac7400700 1 -- 172.21.15.63:0/311056790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 msgr2=0x5613d33f6220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.042+0000 7f5ac7400700 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d33f6220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.042+0000 7f5ac7400700 1 -- 172.21.15.63:0/311056790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 msgr2=0x5613d3416640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.042+0000 7f5ac7400700 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d3416640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.042+0000 7f5ac7400700 1 -- 172.21.15.63:0/311056790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d33e4d30 con 0x5613d33e9a60 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.043+0000 7f5ac7400700 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d3416b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80a9b40 tx=0x7f5ab801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5750934778c57e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.043+0000 7f5abcc00700 1 -- 172.21.15.63:0/311056790 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80b2a10 con 0x5613d33e9a60 2024-05-11T13:55:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.043+0000 7f5abcc00700 1 -- 172.21.15.63:0/311056790 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab80b2790 con 0x5613d33e9a60 2024-05-11T13:55:50.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.043+0000 7f5abcc00700 1 -- 172.21.15.63:0/311056790 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8010430 con 0x5613d33e9a60 2024-05-11T13:55:50.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.044+0000 7f5ad9524b80 1 -- 172.21.15.63:0/311056790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 msgr2=0x5613d3416b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:50.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.044+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d3416b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80a9b40 tx=0x7f5ab801f000 comp rx=0 tx=0).stop 2024-05-11T13:55:50.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.045+0000 7f5ad9524b80 1 -- 172.21.15.63:0/311056790 shutdown_connections 2024-05-11T13:55:50.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.045+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d33f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.045+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d3416640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.045+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/311056790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d3416b80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.045+0000 7f5ad9524b80 1 -- 172.21.15.63:0/311056790 >> 172.21.15.63:0/311056790 conn(0x5613d3309cf0 msgr2=0x5613d33ecf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:50.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.045+0000 7f5ad9524b80 1 -- 172.21.15.63:0/311056790 shutdown_connections 2024-05-11T13:55:50.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.045+0000 7f5ad9524b80 1 -- 172.21.15.63:0/311056790 wait complete. 2024-05-11T13:55:50.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.046+0000 7f5ad9524b80 1 -- start start 2024-05-11T13:55:50.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.046+0000 7f5ad9524b80 1 Processor -- start 2024-05-11T13:55:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.047+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.048+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d33f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.048+0000 7f5ad9524b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d3479200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.049+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8072d20 con 0x5613d33e9a60 2024-05-11T13:55:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.049+0000 7f5ac7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.049+0000 7f5ac7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.63:36936/0 (socket says 172.21.15.63:36936) 2024-05-11T13:55:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.049+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8072650 con 0x5613d33e4560 2024-05-11T13:55:50.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.049+0000 7f5ad9524b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac806f830 con 0x5613d33ea380 2024-05-11T13:55:50.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.049+0000 7f5ad9524b80 10 client.0 _ll_register_callbacks cb 0x5613d32c1510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:55:50.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.049+0000 7f5accc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d3479200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:50.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.050+0000 7f5accc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d3479200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:45942/0 (socket says 172.21.15.63:45942) 2024-05-11T13:55:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.050+0000 7f5accc00700 1 -- 172.21.15.63:0/967287372 learned_addr learned my addr 172.21.15.63:0/967287372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:55:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.051+0000 7f5ac7e00700 1 -- 172.21.15.63:0/967287372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 msgr2=0x5613d3479200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:55:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.051+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d3479200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.051+0000 7f5ac7e00700 1 -- 172.21.15.63:0/967287372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 msgr2=0x5613d33f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:55:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.051+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d33f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.051+0000 7f5ac7e00700 1 -- 172.21.15.63:0/967287372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d33f4010 con 0x5613d33e4560 2024-05-11T13:55:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.052+0000 7f5ac7e00700 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f3200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5613d3411580 tx=0x7f5ab4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46756af273ac5078 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:55:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.052+0000 7f5ab2000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-11T13:55:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.052+0000 7f5ab2000700 1 -- 172.21.15.63:0/967287372 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40257e0 con 0x5613d33e4560 2024-05-11T13:55:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.052+0000 7f5ab2000700 1 -- 172.21.15.63:0/967287372 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f5ab4025980 con 0x5613d33e4560 2024-05-11T13:55:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.053+0000 7f5accc00700 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d3479200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T13:55:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.054+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab80b2790 con 0x5613d33e4560 2024-05-11T13:55:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.055+0000 7f5ab2000700 1 -- 172.21.15.63:0/967287372 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f5ab402f430 con 0x5613d33e4560 2024-05-11T13:55:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.055+0000 7f5ad9524b80 10 client.35807 fetch_fsmap learned FSMap version 77 2024-05-11T13:55:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.055+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5613d347a880 con 0x5613d33e4560 2024-05-11T13:55:50.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.058+0000 7f5ab2000700 1 -- 172.21.15.63:0/967287372 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab80b2790 con 0x5613d33e4560 2024-05-11T13:55:50.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.058+0000 7f5ab2000700 1 -- 172.21.15.63:0/967287372 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f5ab40105d0 con 0x5613d33e4560 2024-05-11T13:55:50.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.060+0000 7f5ad9524b80 10 client.35807 fetch_fsmap finished waiting for FSMap version 77 2024-05-11T13:55:50.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.060+0000 7f5ad9524b80 10 client.35807 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:55:50.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.060+0000 7f5ad9524b80 20 client.35807 populate_metadata read hostname 'smithi063' 2024-05-11T13:55:50.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.061+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d347b070 0x5613d347d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:55:50.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.061+0000 7f5ad9524b80 4 client.35807 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.85, "await_for": 42.0}] 2024-05-11T13:55:50.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.061+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.85, "await_for": 42.0}) -- 0x5613d333b590 con 0x5613d347b070 2024-05-11T13:55:50.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.062+0000 7f5ac7400700 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d347b070 0x5613d347d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:55:50.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.064+0000 7f5ac7400700 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d347b070 0x5613d347d490 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd8e0ba1bb3d976 server_cookie=d38ad5c2b187bb33 in_seq=0 out_seq=0 2024-05-11T13:55:50.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.064+0000 7f5ab2000700 10 client.35807 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:55:50.055 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:49 smithi165 ceph-mon[33235]: pgmap v693: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-11T13:55:50.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:49 smithi071 ceph-mon[33293]: pgmap v693: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-11T13:55:50.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.363+0000 7f5ab2000700 1 -- 172.21.15.63:0/967287372 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5613d333b590 con 0x5613d347b070 2024-05-11T13:55:50.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.363+0000 7f5ab2000700 10 client.35807 handle_command_reply: tid=0 2024-05-11T13:55:50.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.363+0000 7f5ad9524b80 1 client.35807 shutdown 2024-05-11T13:55:50.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.363+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 msgr2=0x5613d33f3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.363+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f3200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5613d3411580 tx=0x7f5ab4004750 comp rx=0 tx=0).stop 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 shutdown_connections 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613d33ea380 0x5613d3479200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5613d347b070 0x5613d347d490 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613d33e4560 0x5613d33f3200 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ab2000700 0 client.35807 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 --2- 172.21.15.63:0/967287372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613d33e9a60 0x5613d33f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 >> 172.21.15.63:0/967287372 conn(0x5613d3309cf0 msgr2=0x5613d33e74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 shutdown_connections 2024-05-11T13:55:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 1 -- 172.21.15.63:0/967287372 wait complete. 2024-05-11T13:55:50.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.364+0000 7f5ad9524b80 20 client.35807 trim_cache size 0 max 16384 2024-05-11T13:55:50.354 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:55:50.354 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 77, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 62, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "d6d8ed9e": { 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 62, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 77.8, 2024-05-11T13:55:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:55:50.356 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:55:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5aac0b2b20 msgr2=0x7f5aac0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5aac0b2b20 0x7f5aac0b4fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005fd0 tx=0x7f5ab4005da0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac8108910 msgr2=0x7f5ac8102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:55:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac8108910 0x7f5ac8102a60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800e9a0 tx=0x7f5ab800ed60 comp rx=0 tx=0).stop 2024-05-11T13:55:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 shutdown_connections 2024-05-11T13:55:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f5aac0b2b20 0x7f5aac0b4fe0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ac8102fa0 0x7f5ac81a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac806be40 0x7f5ac8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 --2- 172.21.15.63:0/1118507736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac8108910 0x7f5ac8102a60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:55:50.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 >> 172.21.15.63:0/1118507736 conn(0x7f5ac80793f0 msgr2=0x7f5ac8107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:55:50.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 shutdown_connections 2024-05-11T13:55:50.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:55:50.366+0000 7f5acec00700 1 -- 172.21.15.63:0/1118507736 wait complete. 2024-05-11T13:55:51.139 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 865.0 s, 156.8 tps, lat 510.019 ms stddev 119.530 2024-05-11T13:55:51.951 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd6d8ed9e', quiesce took 13.5 seconds. Will release after: 17.85 2024-05-11T13:55:52.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:51 smithi165 ceph-mon[33235]: pgmap v694: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 4.1 MiB/s wr, 359 op/s 2024-05-11T13:55:52.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:51 smithi063 ceph-mon[29809]: pgmap v694: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 4.1 MiB/s wr, 359 op/s 2024-05-11T13:55:52.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:51 smithi071 ceph-mon[33293]: pgmap v694: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 4.1 MiB/s wr, 359 op/s 2024-05-11T13:55:54.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:53 smithi165 ceph-mon[33235]: pgmap v695: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 3.9 MiB/s wr, 331 op/s 2024-05-11T13:55:54.068 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:53 smithi063 ceph-mon[29809]: pgmap v695: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 3.9 MiB/s wr, 331 op/s 2024-05-11T13:55:54.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:53 smithi071 ceph-mon[33293]: pgmap v695: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 3.9 MiB/s wr, 331 op/s 2024-05-11T13:55:56.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:55 smithi165 ceph-mon[33235]: pgmap v696: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-05-11T13:55:56.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 870.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:55:56.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:55 smithi063 ceph-mon[29809]: pgmap v696: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-05-11T13:55:56.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:55 smithi071 ceph-mon[33293]: pgmap v696: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-05-11T13:55:58.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:57 smithi165 ceph-mon[33235]: pgmap v697: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-05-11T13:55:58.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:57 smithi063 ceph-mon[29809]: pgmap v697: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-05-11T13:55:58.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:57 smithi071 ceph-mon[33293]: pgmap v697: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-05-11T13:56:00.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:55:59 smithi165 ceph-mon[33235]: pgmap v698: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 3.1 MiB/s wr, 290 op/s 2024-05-11T13:56:00.173 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:55:59 smithi063 ceph-mon[29809]: pgmap v698: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 3.1 MiB/s wr, 290 op/s 2024-05-11T13:56:00.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:55:59 smithi071 ceph-mon[33293]: pgmap v698: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 3.1 MiB/s wr, 290 op/s 2024-05-11T13:56:01.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 875.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:56:02.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:01 smithi165 ceph-mon[33235]: pgmap v699: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-11T13:56:02.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:01 smithi063 ceph-mon[29809]: pgmap v699: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-11T13:56:02.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:01 smithi071 ceph-mon[33293]: pgmap v699: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-11T13:56:03.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:02 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:56:03.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:02 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:56:03.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:02 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:56:04.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:03 smithi165 ceph-mon[33235]: pgmap v700: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-11T13:56:04.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:03 smithi063 ceph-mon[29809]: pgmap v700: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-11T13:56:04.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:03 smithi071 ceph-mon[33293]: pgmap v700: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-11T13:56:06.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:05 smithi165 ceph-mon[33235]: pgmap v701: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-11T13:56:06.077 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:05 smithi063 ceph-mon[29809]: pgmap v701: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-11T13:56:06.142 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 880.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:56:06.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:05 smithi071 ceph-mon[33293]: pgmap v701: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-11T13:56:08.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:07 smithi165 ceph-mon[33235]: pgmap v702: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-11T13:56:08.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:07 smithi063 ceph-mon[29809]: pgmap v702: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-11T13:56:08.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:07 smithi071 ceph-mon[33293]: pgmap v702: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-11T13:56:09.802 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:56:09.914 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:09 smithi063 ceph-mon[29809]: pgmap v703: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-05-11T13:56:10.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:09 smithi165 ceph-mon[33235]: pgmap v703: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-05-11T13:56:10.193 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:09 smithi071 ceph-mon[33293]: pgmap v703: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-05-11T13:56:10.499 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:56:10.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:10 smithi063 ceph-mon[29809]: osdmap e124: 12 total, 12 up, 12 in 2024-05-11T13:56:10.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:10 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:10.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:10 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:10.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:10 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:56:10.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:10 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:56:10.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:10 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:11.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:10 smithi165 ceph-mon[33235]: osdmap e124: 12 total, 12 up, 12 in 2024-05-11T13:56:11.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:10 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:11.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:10 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:11.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:10 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:56:11.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:10 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:56:11.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:10 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:11.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 885.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:56:11.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:10 smithi071 ceph-mon[33293]: osdmap e124: 12 total, 12 up, 12 in 2024-05-11T13:56:11.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:10 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:11.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:10 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:11.192 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:10 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:56:11.192 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:10 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:56:11.192 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:10 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:56:12.336 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:12 smithi063 ceph-mon[29809]: pgmap v705: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-05-11T13:56:12.442 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:12 smithi071 ceph-mon[33293]: pgmap v705: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-05-11T13:56:12.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:12 smithi165 ceph-mon[33235]: pgmap v705: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-05-11T13:56:12.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 -- 172.21.15.63:0/4219747966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c806c650 msgr2=0x7f66c8110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 --2- 172.21.15.63:0/4219747966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c806c650 0x7f66c8110430 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f66c00099c0 tx=0x7f66c0009cd0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 -- 172.21.15.63:0/4219747966 shutdown_connections 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 --2- 172.21.15.63:0/4219747966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c806c650 0x7f66c8110430 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 --2- 172.21.15.63:0/4219747966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c81096f0 0x7f66c8109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 --2- 172.21.15.63:0/4219747966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c806bc90 0x7f66c806c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 -- 172.21.15.63:0/4219747966 >> 172.21.15.63:0/4219747966 conn(0x7f66c8079250 msgr2=0x7f66c8079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 -- 172.21.15.63:0/4219747966 shutdown_connections 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.697+0000 7f66d0000700 1 -- 172.21.15.63:0/4219747966 wait complete. 2024-05-11T13:56:12.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.698+0000 7f66d0000700 1 Processor -- start 2024-05-11T13:56:12.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.699+0000 7f66d0000700 1 -- start start 2024-05-11T13:56:12.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.699+0000 7f66d0000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c806bc90 0x7f66c819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.699+0000 7f66d0000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c806c650 0x7f66c819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.699+0000 7f66d0000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c81096f0 0x7f66c8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.699+0000 7f66d0000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8113680 con 0x7f66c81096f0 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.699+0000 7f66d0000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8113500 con 0x7f66c806c650 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.699+0000 7f66d0000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8113800 con 0x7f66c806bc90 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c81096f0 0x7f66c8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c81096f0 0x7f66c8199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46610/0 (socket says 172.21.15.63:46610) 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 -- 172.21.15.63:0/305004665 learned_addr learned my addr 172.21.15.63:0/305004665 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ccc00700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c806c650 0x7f66c819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 -- 172.21.15.63:0/305004665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c806bc90 msgr2=0x7f66c819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c806bc90 0x7f66c819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 -- 172.21.15.63:0/305004665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c806c650 msgr2=0x7f66c819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c806c650 0x7f66c819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ce000700 1 -- 172.21.15.63:0/305004665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c819a0a0 con 0x7f66c81096f0 2024-05-11T13:56:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.700+0000 7f66ccc00700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c806c650 0x7f66c819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.701+0000 7f66ce000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c81096f0 0x7f66c8199940 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b580 tx=0x7f66c0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.701+0000 7f66c6000700 1 -- 172.21.15.63:0/305004665 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0027070 con 0x7f66c81096f0 2024-05-11T13:56:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.701+0000 7f66c6000700 1 -- 172.21.15.63:0/305004665 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66c000fce0 con 0x7f66c81096f0 2024-05-11T13:56:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.701+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c819a330 con 0x7f66c81096f0 2024-05-11T13:56:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.701+0000 7f66c6000700 1 -- 172.21.15.63:0/305004665 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0022d30 con 0x7f66c81096f0 2024-05-11T13:56:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.701+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c81a6650 con 0x7f66c81096f0 2024-05-11T13:56:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.702+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c806a240 con 0x7f66c81096f0 2024-05-11T13:56:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.705+0000 7f66c6000700 1 -- 172.21.15.63:0/305004665 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f66c002a070 con 0x7f66c81096f0 2024-05-11T13:56:12.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.707+0000 7f66c6000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f66b00b29f0 0x7f66b00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:12.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.707+0000 7f66c6000700 1 -- 172.21.15.63:0/305004665 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f66c00e3950 con 0x7f66c81096f0 2024-05-11T13:56:12.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.707+0000 7f66cd600700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f66b00b29f0 0x7f66b00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:12.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.708+0000 7f66cd600700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f66b00b29f0 0x7f66b00b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0060b0 tx=0x7f66bc006040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:12.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:12.710+0000 7f66c6000700 1 -- 172.21.15.63:0/305004665 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66c0031080 con 0x7f66c81096f0 2024-05-11T13:56:12.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.003+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66c802e570 con 0x7f66c81096f0 2024-05-11T13:56:12.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.005+0000 7f66c6000700 1 -- 172.21.15.63:0/305004665 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+11942 (secure 0 0 0) 0x7f66c0013070 con 0x7f66c81096f0 2024-05-11T13:56:12.994 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:56:12.994 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:54:09.149495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:56:12.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f66b00b29f0 msgr2=0x7f66b00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:12.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f66b00b29f0 0x7f66b00b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0060b0 tx=0x7f66bc006040 comp rx=0 tx=0).stop 2024-05-11T13:56:12.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c81096f0 msgr2=0x7f66c8199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:12.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c81096f0 0x7f66c8199940 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b580 tx=0x7f66c0005ef0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 shutdown_connections 2024-05-11T13:56:12.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f66b00b29f0 0x7f66b00b4eb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66c81096f0 0x7f66c8199940 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c806bc90 0x7f66c819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 --2- 172.21.15.63:0/305004665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c806c650 0x7f66c819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:12.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.008+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 >> 172.21.15.63:0/305004665 conn(0x7f66c8079250 msgr2=0x7f66c8101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:12.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.009+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 shutdown_connections 2024-05-11T13:56:12.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:13.009+0000 7f66d0000700 1 -- 172.21.15.63:0/305004665 wait complete. 2024-05-11T13:56:12.999 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 77 2024-05-11T13:56:13.264 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:13 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/305004665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:13.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:13 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/305004665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:13.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:13 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/305004665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:14.311 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id d6d8ed9e --release --await' 2024-05-11T13:56:14.312 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id d6d8ed9e --release --await 2024-05-11T13:56:14.388 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:14 smithi063 ceph-mon[29809]: pgmap v706: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-05-11T13:56:14.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:14 smithi071 ceph-mon[33293]: pgmap v706: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-05-11T13:56:14.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:14 smithi165 ceph-mon[33235]: pgmap v706: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-05-11T13:56:14.794 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:56:16.143 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 890.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-11T13:56:16.562 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:16 smithi063 ceph-mon[29809]: pgmap v707: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-05-11T13:56:16.563 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:16 smithi063 ceph-mon[29809]: osdmap e125: 12 total, 12 up, 12 in 2024-05-11T13:56:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:16 smithi071 ceph-mon[33293]: pgmap v707: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-05-11T13:56:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:16 smithi071 ceph-mon[33293]: osdmap e125: 12 total, 12 up, 12 in 2024-05-11T13:56:16.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:16 smithi165 ceph-mon[33235]: pgmap v707: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-05-11T13:56:16.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:16 smithi165 ceph-mon[33235]: osdmap e125: 12 total, 12 up, 12 in 2024-05-11T13:56:17.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.517+0000 7fa865600700 1 -- 172.21.15.63:0/2712041180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 msgr2=0x7fa86010a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.517+0000 7fa865600700 1 --2- 172.21.15.63:0/2712041180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa86010a200 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa8500099c0 tx=0x7fa850009cd0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.517+0000 7fa865600700 1 -- 172.21.15.63:0/2712041180 shutdown_connections 2024-05-11T13:56:17.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.517+0000 7fa865600700 1 --2- 172.21.15.63:0/2712041180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa86010eeb0 0x7fa860116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.517+0000 7fa865600700 1 --2- 172.21.15.63:0/2712041180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa86010a200 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.517+0000 7fa865600700 1 --2- 172.21.15.63:0/2712041180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa86010a7d0 0x7fa86010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.517+0000 7fa865600700 1 -- 172.21.15.63:0/2712041180 >> 172.21.15.63:0/2712041180 conn(0x7fa860076d20 msgr2=0x7fa860079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.518+0000 7fa865600700 1 -- 172.21.15.63:0/2712041180 shutdown_connections 2024-05-11T13:56:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.518+0000 7fa865600700 1 -- 172.21.15.63:0/2712041180 wait complete. 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 Processor -- start 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 -- start start 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa8601a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa86010a7d0 0x7fa8601a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa86010eeb0 0x7fa8601a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa860119c60 con 0x7fa860109e20 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa860119ae0 con 0x7fa86010a7d0 2024-05-11T13:56:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.519+0000 7fa865600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa860119de0 con 0x7fa86010eeb0 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa8601a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa8601a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46640/0 (socket says 172.21.15.63:46640) 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 -- 172.21.15.63:0/2906151266 learned_addr learned my addr 172.21.15.63:0/2906151266 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85e000700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa86010a7d0 0x7fa8601a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85f400700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa86010eeb0 0x7fa8601a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 -- 172.21.15.63:0/2906151266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa86010eeb0 msgr2=0x7fa8601a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa86010eeb0 0x7fa8601a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 -- 172.21.15.63:0/2906151266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa86010a7d0 msgr2=0x7fa8601a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa86010a7d0 0x7fa8601a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85ea00700 1 -- 172.21.15.63:0/2906151266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8601a51f0 con 0x7fa860109e20 2024-05-11T13:56:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.520+0000 7fa85e000700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa86010a7d0 0x7fa8601a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.521+0000 7fa85ea00700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa8601a0230 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b580 tx=0x7fa85000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.521+0000 7fa857400700 1 -- 172.21.15.63:0/2906151266 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850026070 con 0x7fa860109e20 2024-05-11T13:56:17.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.521+0000 7fa857400700 1 -- 172.21.15.63:0/2906151266 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8500054c0 con 0x7fa860109e20 2024-05-11T13:56:17.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.521+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8601a5480 con 0x7fa860109e20 2024-05-11T13:56:17.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.521+0000 7fa857400700 1 -- 172.21.15.63:0/2906151266 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850004c00 con 0x7fa860109e20 2024-05-11T13:56:17.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.521+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8601acd60 con 0x7fa860109e20 2024-05-11T13:56:17.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.525+0000 7fa857400700 1 -- 172.21.15.63:0/2906151266 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa850029070 con 0x7fa860109e20 2024-05-11T13:56:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.526+0000 7fa857400700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8440b2a70 0x7fa8440b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.526+0000 7fa857400700 1 -- 172.21.15.63:0/2906151266 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fa8500e3550 con 0x7fa860109e20 2024-05-11T13:56:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.527+0000 7fa85e000700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8440b2a70 0x7fa8440b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.527+0000 7fa85e000700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8440b2a70 0x7fa8440b4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa860108e30 tx=0x7fa84c005f50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:17.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.545+0000 7fa86ff11b80 1 Processor -- start 2024-05-11T13:56:17.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.545+0000 7fa86ff11b80 1 -- start start 2024-05-11T13:56:17.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa6599a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa65995150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aa658e6b20 con 0x55aa6599a1a0 2024-05-11T13:56:17.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aa65884860 con 0x55aa65994cc0 2024-05-11T13:56:17.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55aa658e2ae0 con 0x55aa6599aac0 2024-05-11T13:56:17.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa6599a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa6599a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46646/0 (socket says 172.21.15.63:46646) 2024-05-11T13:56:17.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85ea00700 1 -- 172.21.15.63:0/1084788847 learned_addr learned my addr 172.21.15.63:0/1084788847 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:17.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85f400700 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa65995150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85ea00700 1 -- 172.21.15.63:0/1084788847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 msgr2=0x55aa659946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85ea00700 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85ea00700 1 -- 172.21.15.63:0/1084788847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 msgr2=0x55aa65995150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.546+0000 7fa85ea00700 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa65995150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.547+0000 7fa85ea00700 1 -- 172.21.15.63:0/1084788847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa658339d0 con 0x55aa6599a1a0 2024-05-11T13:56:17.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.547+0000 7fa85f400700 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa65995150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:56:17.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.547+0000 7fa85ea00700 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa6599a580 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa8500aa400 tx=0x7fa8500046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd5f52cfc503c72 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.547+0000 7fa84be00700 1 -- 172.21.15.63:0/1084788847 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500ae820 con 0x55aa6599a1a0 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.547+0000 7fa84be00700 1 -- 172.21.15.63:0/1084788847 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8500b3240 con 0x55aa6599a1a0 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.547+0000 7fa84be00700 1 -- 172.21.15.63:0/1084788847 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500aa930 con 0x55aa6599a1a0 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1084788847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 msgr2=0x55aa6599a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa6599a580 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa8500aa400 tx=0x7fa8500046e0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1084788847 shutdown_connections 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa65995150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa6599a580 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1084788847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1084788847 >> 172.21.15.63:0/1084788847 conn(0x55aa658c2c60 msgr2=0x55aa6598f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1084788847 shutdown_connections 2024-05-11T13:56:17.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.548+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1084788847 wait complete. 2024-05-11T13:56:17.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.549+0000 7fa86ff11b80 1 -- start start 2024-05-11T13:56:17.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.549+0000 7fa86ff11b80 1 Processor -- start 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa65994cc0 0x55aa659a0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa6599a1a0 0x55aa6599dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa6599e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aa659a13a0 con 0x55aa6599aac0 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aa6584cc20 con 0x55aa6599a1a0 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55aa659a1520 con 0x55aa65994cc0 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa86ff11b80 10 client.0 _ll_register_callbacks cb 0x55aa6587a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:56:17.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa85f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa6599e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa85f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa6599e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46662/0 (socket says 172.21.15.63:46662) 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa85f400700 1 -- 172.21.15.63:0/1588714762 learned_addr learned my addr 172.21.15.63:0/1588714762 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa85e000700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa6599a1a0 0x55aa6599dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.550+0000 7fa85ea00700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa65994cc0 0x55aa659a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa85f400700 1 -- 172.21.15.63:0/1588714762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa65994cc0 msgr2=0x55aa659a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa85f400700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa65994cc0 0x55aa659a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa85f400700 1 -- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa6599a1a0 msgr2=0x55aa6599dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa85f400700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa6599a1a0 0x55aa6599dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa85f400700 1 -- 172.21.15.63:0/1588714762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa658ac520 con 0x55aa6599aac0 2024-05-11T13:56:17.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa85e000700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa6599a1a0 0x55aa6599dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:17.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa85f400700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa6599e1e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa858008bd0 tx=0x7fa858008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc583c8c38906cd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:17.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa848c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa848c00700 1 -- 172.21.15.63:0/1588714762 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858004bb0 con 0x55aa6599aac0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa848c00700 1 -- 172.21.15.63:0/1588714762 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fa858004d50 con 0x55aa6599aac0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.551+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8500b3240 con 0x55aa6599aac0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.552+0000 7fa848c00700 1 -- 172.21.15.63:0/1588714762 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fa858022050 con 0x55aa6599aac0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.552+0000 7fa86ff11b80 10 client.16227 fetch_fsmap learned FSMap version 77 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.552+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55aa659abc00 con 0x55aa6599aac0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.552+0000 7fa848c00700 1 -- 172.21.15.63:0/1588714762 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8500b3240 con 0x55aa6599aac0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.553+0000 7fa848c00700 1 -- 172.21.15.63:0/1588714762 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7fa858015a40 con 0x55aa6599aac0 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.553+0000 7fa86ff11b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 77 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.553+0000 7fa86ff11b80 10 client.16227 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.553+0000 7fa86ff11b80 20 client.16227 populate_metadata read hostname 'smithi063' 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.553+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa659ae2c0 0x55aa659b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.553+0000 7fa86ff11b80 4 client.16227 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:56:17.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.553+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa65961160 con 0x55aa659ae2c0 2024-05-11T13:56:17.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.554+0000 7fa85ea00700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa659ae2c0 0x55aa659b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.555+0000 7fa85ea00700 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa659ae2c0 0x55aa659b06e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8191b3f706aaa7 server_cookie=99c56e947aaa5db3 in_seq=0 out_seq=0 2024-05-11T13:56:17.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.555+0000 7fa848c00700 10 client.16227 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:17.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.556+0000 7fa848c00700 1 -- 172.21.15.63:0/1588714762 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa65961160 con 0x55aa659ae2c0 2024-05-11T13:56:17.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.556+0000 7fa848c00700 10 client.16227 handle_command_reply: tid=0 2024-05-11T13:56:17.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 client.16227 shutdown 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 msgr2=0x55aa6599e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa6599e1e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa858008bd0 tx=0x7fa858008f90 comp rx=0 tx=0).stop 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 shutdown_connections 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa6599e1e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa65994cc0 0x55aa659a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa659ae2c0 0x55aa659b06e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/1588714762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa6599a1a0 0x55aa6599dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 >> 172.21.15.63:0/1588714762 conn(0x55aa658c2c60 msgr2=0x55aa65993690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa848c00700 0 client.16227 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 shutdown_connections 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 1 -- 172.21.15.63:0/1588714762 wait complete. 2024-05-11T13:56:17.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.557+0000 7fa86ff11b80 20 client.16227 trim_cache size 0 max 16384 2024-05-11T13:56:17.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 Processor -- start 2024-05-11T13:56:17.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 -- start start 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa659996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601b5ab0 con 0x55aa6599a1a0 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601b76b0 con 0x55aa65994cc0 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.580+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601b6180 con 0x55aa6599aac0 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa659996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa659996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46666/0 (socket says 172.21.15.63:46666) 2024-05-11T13:56:17.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 -- 172.21.15.63:0/3949198644 learned_addr learned my addr 172.21.15.63:0/3949198644 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:17.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85ea00700 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85f400700 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 -- 172.21.15.63:0/3949198644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 msgr2=0x55aa659aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 -- 172.21.15.63:0/3949198644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 msgr2=0x55aa659946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659946a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 -- 172.21.15.63:0/3949198644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa65989880 con 0x55aa6599a1a0 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85f400700 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.581+0000 7fa85e000700 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa659996e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00ec40 tx=0x7fa84c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930ce05d988b15e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.582+0000 7fa84b400700 1 -- 172.21.15.63:0/3949198644 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c010cd0 con 0x55aa6599a1a0 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.582+0000 7fa84b400700 1 -- 172.21.15.63:0/3949198644 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa84c01e8c0 con 0x55aa6599a1a0 2024-05-11T13:56:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.582+0000 7fa84b400700 1 -- 172.21.15.63:0/3949198644 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01d6f0 con 0x55aa6599a1a0 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.582+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3949198644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 msgr2=0x55aa659996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.582+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa659996e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00ec40 tx=0x7fa84c0047c0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3949198644 shutdown_connections 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599aac0 0x55aa659aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3949198644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599a1a0 0x55aa659996e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3949198644 >> 172.21.15.63:0/3949198644 conn(0x55aa658c2c60 msgr2=0x55aa65998290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3949198644 shutdown_connections 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3949198644 wait complete. 2024-05-11T13:56:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 -- start start 2024-05-11T13:56:17.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.583+0000 7fa86ff11b80 1 Processor -- start 2024-05-11T13:56:17.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599a1a0 0x55aa659a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa86ff11b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa659a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601b6f20 con 0x55aa6599aac0 2024-05-11T13:56:17.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601b6850 con 0x55aa65994cc0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa86ff11b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601b3a70 con 0x55aa6599a1a0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa86ff11b80 10 client.0 _ll_register_callbacks cb 0x55aa6587a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.584+0000 7fa85f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa659a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa659a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46680/0 (socket says 172.21.15.63:46680) 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 -- 172.21.15.63:0/3221547136 learned_addr learned my addr 172.21.15.63:0/3221547136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85ea00700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85e000700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599a1a0 0x55aa659a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 -- 172.21.15.63:0/3221547136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599a1a0 msgr2=0x55aa659a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599a1a0 0x55aa659a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 -- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 msgr2=0x55aa659a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 -- 172.21.15.63:0/3221547136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa659a56c0 con 0x55aa6599aac0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85e000700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599a1a0 0x55aa659a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa85f400700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa659a4e80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa858015db0 tx=0x7fa8580050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba17b87d6e4658da server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:17.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.585+0000 7fa848c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:56:17.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.586+0000 7fa848c00700 1 -- 172.21.15.63:0/3221547136 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858005e70 con 0x55aa6599aac0 2024-05-11T13:56:17.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.586+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c01e8c0 con 0x55aa6599aac0 2024-05-11T13:56:17.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.586+0000 7fa848c00700 1 -- 172.21.15.63:0/3221547136 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fa858004a90 con 0x55aa6599aac0 2024-05-11T13:56:17.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.586+0000 7fa848c00700 1 -- 172.21.15.63:0/3221547136 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fa858035050 con 0x55aa6599aac0 2024-05-11T13:56:17.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.586+0000 7fa86ff11b80 10 client.16233 fetch_fsmap learned FSMap version 77 2024-05-11T13:56:17.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.586+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55aa65a197a0 con 0x55aa6599aac0 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.586+0000 7fa848c00700 1 -- 172.21.15.63:0/3221547136 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa84c01e8c0 con 0x55aa6599aac0 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa848c00700 1 -- 172.21.15.63:0/3221547136 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7fa85800fcd0 con 0x55aa6599aac0 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa86ff11b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 77 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa86ff11b80 10 client.16233 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa86ff11b80 20 client.16233 populate_metadata read hostname 'smithi063' 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa65a19f70 0x55aa65a1c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa86ff11b80 4 client.16233 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d6d8ed9e", "release": true, "await": true}] 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d6d8ed9e", "release": true, "await": true}) -- 0x55aa658f4500 con 0x55aa65a19f70 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.587+0000 7fa85ea00700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa65a19f70 0x55aa65a1c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.588+0000 7fa85ea00700 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa65a19f70 0x55aa65a1c390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36168b6a2bdc8711 server_cookie=18199b4c11e0edf in_seq=0 out_seq=0 2024-05-11T13:56:17.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.588+0000 7fa848c00700 10 client.16233 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:17.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa848c00700 1 -- 172.21.15.63:0/3221547136 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55aa658f4500 con 0x55aa65a19f70 2024-05-11T13:56:17.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa848c00700 10 client.16233 handle_command_reply: tid=0 2024-05-11T13:56:17.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 client.16233 shutdown 2024-05-11T13:56:17.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 msgr2=0x55aa659a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa659a4e80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa858015db0 tx=0x7fa8580050d0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 shutdown_connections 2024-05-11T13:56:17.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa6599aac0 0x55aa659a4e80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55aa65a19f70 0x55aa65a1c390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aa65994cc0 0x55aa659a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 --2- 172.21.15.63:0/3221547136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa6599a1a0 0x55aa659a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 >> 172.21.15.63:0/3221547136 conn(0x55aa658c2c60 msgr2=0x55aa6599fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:17.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa848c00700 0 client.16233 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:17.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.608+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 shutdown_connections 2024-05-11T13:56:17.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.609+0000 7fa86ff11b80 1 -- 172.21.15.63:0/3221547136 wait complete. 2024-05-11T13:56:17.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.609+0000 7fa86ff11b80 20 client.16233 trim_cache size 0 max 16384 2024-05-11T13:56:17.597 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 77, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 64, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "d6d8ed9e": { 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 64, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 77.8, 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:56:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:17.599 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8440b2a70 msgr2=0x7fa8440b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8440b2a70 0x7fa8440b4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa860108e30 tx=0x7fa84c005f50 comp rx=0 tx=0).stop 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 msgr2=0x7fa8601a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa8601a0230 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b580 tx=0x7fa85000f740 comp rx=0 tx=0).stop 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 shutdown_connections 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8440b2a70 0x7fa8440b4f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa86010eeb0 0x7fa8601a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa860109e20 0x7fa8601a0230 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 --2- 172.21.15.63:0/2906151266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa86010a7d0 0x7fa8601a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 >> 172.21.15.63:0/2906151266 conn(0x7fa860076d20 msgr2=0x7fa8600780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 shutdown_connections 2024-05-11T13:56:17.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:17.610+0000 7fa865600700 1 -- 172.21.15.63:0/2906151266 wait complete. 2024-05-11T13:56:18.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:18 smithi063 ceph-mon[29809]: pgmap v709: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-11T13:56:18.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:18 smithi071 ceph-mon[33293]: pgmap v709: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-11T13:56:18.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:18 smithi165 ceph-mon[33235]: pgmap v709: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-11T13:56:18.912 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd6d8ed9e', seconds elapsed: 40.5 2024-05-11T13:56:18.913 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.85 2024-05-11T13:56:18.913 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:56:19.617 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:56:20.618 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:20 smithi063 ceph-mon[29809]: pgmap v710: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 349 KiB/s wr, 39 op/s 2024-05-11T13:56:20.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:20 smithi071 ceph-mon[33293]: pgmap v710: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 349 KiB/s wr, 39 op/s 2024-05-11T13:56:20.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:20 smithi165 ceph-mon[33235]: pgmap v710: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 349 KiB/s wr, 39 op/s 2024-05-11T13:56:21.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 895.0 s, 129.9 tps, lat 4751.229 ms stddev 9932.770 2024-05-11T13:56:22.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:22 smithi063 ceph-mon[29809]: pgmap v711: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 307 KiB/s wr, 35 op/s 2024-05-11T13:56:22.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:22 smithi071 ceph-mon[33293]: pgmap v711: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 307 KiB/s wr, 35 op/s 2024-05-11T13:56:22.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:22 smithi165 ceph-mon[33235]: pgmap v711: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 307 KiB/s wr, 35 op/s 2024-05-11T13:56:24.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 -- 172.21.15.63:0/1115890106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 msgr2=0x7efda0112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:24.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 --2- 172.21.15.63:0/1115890106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0112660 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efd9800b0a0 tx=0x7efd9800b3b0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 -- 172.21.15.63:0/1115890106 shutdown_connections 2024-05-11T13:56:24.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 --2- 172.21.15.63:0/1115890106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0112660 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 --2- 172.21.15.63:0/1115890106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda010b790 0x7efda010bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 --2- 172.21.15.63:0/1115890106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efda0105790 0x7efda0105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 -- 172.21.15.63:0/1115890106 >> 172.21.15.63:0/1115890106 conn(0x7efda0101280 msgr2=0x7efda01036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:24.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 -- 172.21.15.63:0/1115890106 shutdown_connections 2024-05-11T13:56:24.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.158+0000 7efda8400700 1 -- 172.21.15.63:0/1115890106 wait complete. 2024-05-11T13:56:24.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 Processor -- start 2024-05-11T13:56:24.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 -- start start 2024-05-11T13:56:24.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efda0105790 0x7efda0077a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:24.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0077f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:24.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda010b790 0x7efda0078490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:24.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efda01158d0 con 0x7efda0106150 2024-05-11T13:56:24.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efda0115750 con 0x7efda0105790 2024-05-11T13:56:24.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.160+0000 7efda8400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efda0115a50 con 0x7efda010b790 2024-05-11T13:56:24.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0077f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:24.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0077f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54990/0 (socket says 172.21.15.63:54990) 2024-05-11T13:56:24.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 -- 172.21.15.63:0/3317320747 learned_addr learned my addr 172.21.15.63:0/3317320747 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:24.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5a00700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efda0105790 0x7efda0077a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:24.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 -- 172.21.15.63:0/3317320747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda010b790 msgr2=0x7efda0078490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:24.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda010b790 0x7efda0078490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 -- 172.21.15.63:0/3317320747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efda0105790 msgr2=0x7efda0077a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:24.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efda0105790 0x7efda0077a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 -- 172.21.15.63:0/3317320747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda0073e50 con 0x7efda0106150 2024-05-11T13:56:24.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.161+0000 7efda5000700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0077f50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00d950 tx=0x7efd8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:24.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.162+0000 7efd9e000700 1 -- 172.21.15.63:0/3317320747 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c016070 con 0x7efda0106150 2024-05-11T13:56:24.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.162+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda0074140 con 0x7efda0106150 2024-05-11T13:56:24.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.162+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda00746d0 con 0x7efda0106150 2024-05-11T13:56:24.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.162+0000 7efd9e000700 1 -- 172.21.15.63:0/3317320747 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd8c009dd0 con 0x7efda0106150 2024-05-11T13:56:24.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.162+0000 7efd9e000700 1 -- 172.21.15.63:0/3317320747 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c020060 con 0x7efda0106150 2024-05-11T13:56:24.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.163+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd80005330 con 0x7efda0106150 2024-05-11T13:56:24.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.165+0000 7efd9e000700 1 -- 172.21.15.63:0/3317320747 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7efd8c019070 con 0x7efda0106150 2024-05-11T13:56:24.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.167+0000 7efd9e000700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd880b2a20 0x7efd880b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:24.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.167+0000 7efd9e000700 1 -- 172.21.15.63:0/3317320747 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7efd8c0e25a0 con 0x7efda0106150 2024-05-11T13:56:24.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.167+0000 7efda5a00700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd880b2a20 0x7efd880b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:24.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.169+0000 7efda5a00700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd880b2a20 0x7efd880b4ee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efd94009be0 tx=0x7efd94009260 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:24.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.175+0000 7efd9e000700 1 -- 172.21.15.63:0/3317320747 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd8c010e20 con 0x7efda0106150 2024-05-11T13:56:24.653 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:56:24.654 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:54:09.149495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:56:24.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.664+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd800058f0 con 0x7efda0106150 2024-05-11T13:56:24.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.665+0000 7efd9e000700 1 -- 172.21.15.63:0/3317320747 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+11942 (secure 0 0 0) 0x7efd8c0a9b90 con 0x7efda0106150 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.669+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd880b2a20 msgr2=0x7efd880b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.669+0000 7efda8400700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd880b2a20 0x7efd880b4ee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efd94009be0 tx=0x7efd94009260 comp rx=0 tx=0).stop 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.669+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 msgr2=0x7efda0077f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.669+0000 7efda8400700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0077f50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00d950 tx=0x7efd8c00dc60 comp rx=0 tx=0).stop 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.669+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 shutdown_connections 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.670+0000 7efda8400700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efd880b2a20 0x7efd880b4ee0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.670+0000 7efda8400700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda010b790 0x7efda0078490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.670+0000 7efda8400700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efda0105790 0x7efda0077a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.670+0000 7efda8400700 1 --2- 172.21.15.63:0/3317320747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efda0106150 0x7efda0077f50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:24.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.670+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 >> 172.21.15.63:0/3317320747 conn(0x7efda0101280 msgr2=0x7efda0101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:24.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.670+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 shutdown_connections 2024-05-11T13:56:24.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:24.674+0000 7efda8400700 1 -- 172.21.15.63:0/3317320747 wait complete. 2024-05-11T13:56:24.664 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 77 2024-05-11T13:56:24.665 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:24 smithi063 ceph-mon[29809]: pgmap v712: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 307 KiB/s wr, 35 op/s 2024-05-11T13:56:24.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:24 smithi071 ceph-mon[33293]: pgmap v712: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 307 KiB/s wr, 35 op/s 2024-05-11T13:56:24.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:24 smithi165 ceph-mon[33235]: pgmap v712: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 307 KiB/s wr, 35 op/s 2024-05-11T13:56:25.435 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db / --timeout 60 --expiration 77.85 --await-for 42.0' 2024-05-11T13:56:25.435 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db / --timeout 60 --expiration 77.85 --await-for 42.0 2024-05-11T13:56:25.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:25 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3317320747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:25.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:25 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3317320747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:25.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:25 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3317320747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:56:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:progress: 900.0 s, 204.2 tps, lat 499.851 ms stddev 87.715 2024-05-11T13:56:26.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:26 smithi063 ceph-mon[29809]: pgmap v713: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-05-11T13:56:26.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:26 smithi071 ceph-mon[33293]: pgmap v713: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-05-11T13:56:26.799 INFO:teuthology.orchestra.run.smithi063.stdout:transaction type: 2024-05-11T13:56:26.799 INFO:teuthology.orchestra.run.smithi063.stdout:scaling factor: 32 2024-05-11T13:56:26.799 INFO:teuthology.orchestra.run.smithi063.stdout:query mode: simple 2024-05-11T13:56:26.799 INFO:teuthology.orchestra.run.smithi063.stdout:number of clients: 100 2024-05-11T13:56:26.799 INFO:teuthology.orchestra.run.smithi063.stdout:number of threads: 4 2024-05-11T13:56:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:duration: 900 s 2024-05-11T13:56:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:number of transactions actually processed: 215844 2024-05-11T13:56:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:latency average = 416.821 ms 2024-05-11T13:56:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:latency stddev = 2257.448 ms 2024-05-11T13:56:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:tps = 239.650937 (including connections establishing) 2024-05-11T13:56:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:tps = 239.656542 (excluding connections establishing) 2024-05-11T13:56:26.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:26 smithi165 ceph-mon[33235]: pgmap v713: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-05-11T13:56:26.820 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-11T13:56:28.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:28 smithi071 ceph-mon[33293]: pgmap v714: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-05-11T13:56:28.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:28 smithi165 ceph-mon[33235]: pgmap v714: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-05-11T13:56:28.821 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:28 smithi063 ceph-mon[29809]: pgmap v714: 129 pgs: 129 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 -- 172.21.15.63:0/615709633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 msgr2=0x7f1d58105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/615709633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 0x7f1d58105470 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1d400099c0 tx=0x7f1d40009cd0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 -- 172.21.15.63:0/615709633 shutdown_connections 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/615709633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d581059b0 0x7f1d58110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/615709633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 0x7f1d58103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/615709633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 0x7f1d58105470 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 -- 172.21.15.63:0/615709633 >> 172.21.15.63:0/615709633 conn(0x7f1d580feeb0 msgr2=0x7f1d581012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:30.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.223+0000 7f1d5ce00700 1 -- 172.21.15.63:0/615709633 shutdown_connections 2024-05-11T13:56:30.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.224+0000 7f1d5ce00700 1 -- 172.21.15.63:0/615709633 wait complete. 2024-05-11T13:56:30.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.225+0000 7f1d5ce00700 1 Processor -- start 2024-05-11T13:56:30.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.225+0000 7f1d5ce00700 1 -- start start 2024-05-11T13:56:30.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d5ce00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 0x7f1d581a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d5ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 0x7f1d581a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d5ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d581059b0 0x7f1d5819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d5ce00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d58113560 con 0x7f1d58104ff0 2024-05-11T13:56:30.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d5ce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581133e0 con 0x7f1d581059b0 2024-05-11T13:56:30.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d5ce00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581136e0 con 0x7f1d58103700 2024-05-11T13:56:30.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d57400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 0x7f1d581a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.226+0000 7f1d4ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 0x7f1d581a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d57400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 0x7f1d581a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.63:52876/0 (socket says 172.21.15.63:52876) 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d57400700 1 -- 172.21.15.63:0/2265025721 learned_addr learned my addr 172.21.15.63:0/2265025721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d57e00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d581059b0 0x7f1d5819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d4ea00700 1 -- 172.21.15.63:0/2265025721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 msgr2=0x7f1d581a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 0x7f1d581a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d4ea00700 1 -- 172.21.15.63:0/2265025721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d581059b0 msgr2=0x7f1d5819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d581059b0 0x7f1d5819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d4ea00700 1 -- 172.21.15.63:0/2265025721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5819c110 con 0x7f1d58104ff0 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d57e00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d581059b0 0x7f1d5819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:30.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.227+0000 7f1d57400700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 0x7f1d581a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:30.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.228+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 0x7f1d581a1d30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1d400098e0 tx=0x7f1d400056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:30.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.228+0000 7f1d54c00700 1 -- 172.21.15.63:0/2265025721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40026070 con 0x7f1d58104ff0 2024-05-11T13:56:30.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.228+0000 7f1d54c00700 1 -- 172.21.15.63:0/2265025721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d4000f460 con 0x7f1d58104ff0 2024-05-11T13:56:30.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.228+0000 7f1d54c00700 1 -- 172.21.15.63:0/2265025721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40009e60 con 0x7f1d58104ff0 2024-05-11T13:56:30.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.228+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5819c3a0 con 0x7f1d58104ff0 2024-05-11T13:56:30.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.229+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d581a86c0 con 0x7f1d58104ff0 2024-05-11T13:56:30.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.232+0000 7f1d54c00700 1 -- 172.21.15.63:0/2265025721 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1d40029070 con 0x7f1d58104ff0 2024-05-11T13:56:30.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.234+0000 7f1d54c00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1d3c0b2b20 0x7f1d3c0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.234+0000 7f1d57400700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1d3c0b2b20 0x7f1d3c0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.234+0000 7f1d54c00700 1 -- 172.21.15.63:0/2265025721 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f1d400e3250 con 0x7f1d58104ff0 2024-05-11T13:56:30.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.235+0000 7f1d57400700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1d3c0b2b20 0x7f1d3c0b4fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d5819caf0 tx=0x7f1d4800b040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:30.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.386+0000 7f1d68d17b80 1 Processor -- start 2024-05-11T13:56:30.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.386+0000 7f1d68d17b80 1 -- start start 2024-05-11T13:56:30.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d171e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d1718c50 0x55b7d17190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b7d166abb0 con 0x55b7d171e100 2024-05-11T13:56:30.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b7d16088f0 con 0x55b7d171ea20 2024-05-11T13:56:30.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b7d1666b70 con 0x55b7d1718c50 2024-05-11T13:56:30.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d171e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.387+0000 7f1d57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d171e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57194/0 (socket says 172.21.15.63:57194) 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d57400700 1 -- 172.21.15.63:0/3493722766 learned_addr learned my addr 172.21.15.63:0/3493722766 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d57e00700 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d1718c50 0x55b7d17190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d57400700 1 -- 172.21.15.63:0/3493722766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d1718c50 msgr2=0x55b7d17190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d57400700 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d1718c50 0x55b7d17190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d57400700 1 -- 172.21.15.63:0/3493722766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 msgr2=0x55b7d1718630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d57400700 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.388+0000 7f1d57400700 1 -- 172.21.15.63:0/3493722766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d15b7a60 con 0x55b7d171e100 2024-05-11T13:56:30.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.389+0000 7f1d57400700 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d171e4e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1d480166e0 tx=0x7f1d48012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99584dd68393034 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:30.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.389+0000 7f1d4cc00700 1 -- 172.21.15.63:0/3493722766 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48003c00 con 0x55b7d171e100 2024-05-11T13:56:30.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.390+0000 7f1d4cc00700 1 -- 172.21.15.63:0/3493722766 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d48003da0 con 0x55b7d171e100 2024-05-11T13:56:30.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.390+0000 7f1d4cc00700 1 -- 172.21.15.63:0/3493722766 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48021780 con 0x55b7d171e100 2024-05-11T13:56:30.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.391+0000 7f1d68d17b80 1 -- 172.21.15.63:0/3493722766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 msgr2=0x55b7d171e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.391+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d171e4e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1d480166e0 tx=0x7f1d48012000 comp rx=0 tx=0).stop 2024-05-11T13:56:30.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.391+0000 7f1d68d17b80 1 -- 172.21.15.63:0/3493722766 shutdown_connections 2024-05-11T13:56:30.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.391+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d1718c50 0x55b7d17190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.391+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d171e4e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.391+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/3493722766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.391+0000 7f1d68d17b80 1 -- 172.21.15.63:0/3493722766 >> 172.21.15.63:0/3493722766 conn(0x55b7d1646cf0 msgr2=0x55b7d1713190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:30.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.392+0000 7f1d68d17b80 1 -- 172.21.15.63:0/3493722766 shutdown_connections 2024-05-11T13:56:30.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.392+0000 7f1d68d17b80 1 -- 172.21.15.63:0/3493722766 wait complete. 2024-05-11T13:56:30.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.393+0000 7f1d68d17b80 1 -- start start 2024-05-11T13:56:30.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.393+0000 7f1d68d17b80 1 Processor -- start 2024-05-11T13:56:30.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.394+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d1718c50 0x55b7d1724760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.394+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d1721b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.394+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d171ea20 0x55b7d1722040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.394+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b7d1725540 con 0x55b7d1718c50 2024-05-11T13:56:30.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.394+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b7d15d0cb0 con 0x55b7d171e100 2024-05-11T13:56:30.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.394+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b7d17256c0 con 0x55b7d171ea20 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d68d17b80 10 client.0 _ll_register_callbacks cb 0x55b7d1611e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d1718c50 0x55b7d1724760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d1718c50 0x55b7d1724760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57208/0 (socket says 172.21.15.63:57208) 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 -- 172.21.15.63:0/362478371 learned_addr learned my addr 172.21.15.63:0/362478371 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d1721b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57e00700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d171ea20 0x55b7d1722040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 -- 172.21.15.63:0/362478371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d171ea20 msgr2=0x55b7d1722040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:30.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d171ea20 0x55b7d1722040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 -- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 msgr2=0x55b7d1721b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:30.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d1721b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57400700 1 -- 172.21.15.63:0/362478371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d16305b0 con 0x55b7d1718c50 2024-05-11T13:56:30.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.395+0000 7f1d57e00700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d171ea20 0x55b7d1722040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:30.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.396+0000 7f1d57400700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d1718c50 0x55b7d1724760 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1d480095c0 tx=0x7f1d48003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ec0de7e35d3651 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:30.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.396+0000 7f1d45600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.396+0000 7f1d45600700 1 -- 172.21.15.63:0/362478371 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4802f070 con 0x55b7d1718c50 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.396+0000 7f1d45600700 1 -- 172.21.15.63:0/362478371 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f1d48004600 con 0x55b7d1718c50 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.396+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d48003da0 con 0x55b7d1718c50 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d45600700 1 -- 172.21.15.63:0/362478371 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1d4803b050 con 0x55b7d1718c50 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d68d17b80 10 client.16251 fetch_fsmap learned FSMap version 77 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55b7d172fd80 con 0x55b7d1718c50 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d45600700 1 -- 172.21.15.63:0/362478371 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d48003da0 con 0x55b7d1718c50 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d45600700 1 -- 172.21.15.63:0/362478371 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f1d48021400 con 0x55b7d1718c50 2024-05-11T13:56:30.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d68d17b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 77 2024-05-11T13:56:30.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d68d17b80 10 client.16251 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:56:30.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d68d17b80 20 client.16251 populate_metadata read hostname 'smithi063' 2024-05-11T13:56:30.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.397+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d1732250 0x55b7d1734640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.398+0000 7f1d68d17b80 4 client.16251 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:56:30.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.398+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7d16e5340 con 0x55b7d1732250 2024-05-11T13:56:30.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.398+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d1732250 0x55b7d1734640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.399+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d1732250 0x55b7d1734640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a1476ad435d4 server_cookie=5ba7430415a6116a in_seq=0 out_seq=0 2024-05-11T13:56:30.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.399+0000 7f1d45600700 10 client.16251 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:30.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d45600700 1 -- 172.21.15.63:0/362478371 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7d16e5340 con 0x55b7d1732250 2024-05-11T13:56:30.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d45600700 10 client.16251 handle_command_reply: tid=0 2024-05-11T13:56:30.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 client.16251 shutdown 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d1718c50 msgr2=0x55b7d1724760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d1718c50 0x55b7d1724760 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1d480095c0 tx=0x7f1d48003f50 comp rx=0 tx=0).stop 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 shutdown_connections 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d171ea20 0x55b7d1722040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d1718c50 0x55b7d1724760 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d1732250 0x55b7d1734640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/362478371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d1721b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.402+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 >> 172.21.15.63:0/362478371 conn(0x55b7d1646cf0 msgr2=0x55b7d1717620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.403+0000 7f1d45600700 0 client.16251 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.403+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 shutdown_connections 2024-05-11T13:56:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.403+0000 7f1d68d17b80 1 -- 172.21.15.63:0/362478371 wait complete. 2024-05-11T13:56:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.403+0000 7f1d68d17b80 20 client.16251 trim_cache size 0 max 16384 2024-05-11T13:56:30.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 Processor -- start 2024-05-11T13:56:30.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 -- start start 2024-05-11T13:56:30.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d172e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 0x55b7d1718550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581b1380 con 0x55b7d171e100 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581b2f80 con 0x55b7d171ea20 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.430+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581b1a50 con 0x55b7d17221d0 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d172e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d172e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57224/0 (socket says 172.21.15.63:57224) 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57400700 1 -- 172.21.15.63:0/887682274 learned_addr learned my addr 172.21.15.63:0/887682274 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57e00700 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 0x55b7d1718550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57400700 1 -- 172.21.15.63:0/887682274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 msgr2=0x55b7d1718550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57400700 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 0x55b7d1718550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57400700 1 -- 172.21.15.63:0/887682274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 msgr2=0x55b7d1718010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.431+0000 7f1d57400700 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.432+0000 7f1d57400700 1 -- 172.21.15.63:0/887682274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d170fac0 con 0x55b7d171e100 2024-05-11T13:56:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.432+0000 7f1d57400700 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d172e5d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1d48019040 tx=0x7f1d48009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d847d1d94544dab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.432+0000 7f1d47e00700 1 -- 172.21.15.63:0/887682274 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4802f040 con 0x55b7d171e100 2024-05-11T13:56:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.432+0000 7f1d47e00700 1 -- 172.21.15.63:0/887682274 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d48038030 con 0x55b7d171e100 2024-05-11T13:56:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.432+0000 7f1d47e00700 1 -- 172.21.15.63:0/887682274 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48021cc0 con 0x55b7d171e100 2024-05-11T13:56:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 -- 172.21.15.63:0/887682274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 msgr2=0x55b7d172e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d172e5d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1d48019040 tx=0x7f1d48009b00 comp rx=0 tx=0).stop 2024-05-11T13:56:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 -- 172.21.15.63:0/887682274 shutdown_connections 2024-05-11T13:56:30.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 0x55b7d1718550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171e100 0x55b7d172e5d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/887682274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171ea20 0x55b7d1718010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 -- 172.21.15.63:0/887682274 >> 172.21.15.63:0/887682274 conn(0x55b7d1646cf0 msgr2=0x55b7d1721ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:30.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 -- 172.21.15.63:0/887682274 shutdown_connections 2024-05-11T13:56:30.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.434+0000 7f1d68d17b80 1 -- 172.21.15.63:0/887682274 wait complete. 2024-05-11T13:56:30.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.435+0000 7f1d68d17b80 1 -- start start 2024-05-11T13:56:30.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.435+0000 7f1d68d17b80 1 Processor -- start 2024-05-11T13:56:30.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d172adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171ea20 0x55b7d1727920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d68d17b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 0x55b7d1727e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581b27f0 con 0x55b7d171ea20 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581b2120 con 0x55b7d171e100 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d68d17b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581af340 con 0x55b7d17221d0 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d68d17b80 10 client.0 _ll_register_callbacks cb 0x55b7d1611e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d4ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171ea20 0x55b7d1727920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d4ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171ea20 0x55b7d1727920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57240/0 (socket says 172.21.15.63:57240) 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d4ea00700 1 -- 172.21.15.63:0/1878110082 learned_addr learned my addr 172.21.15.63:0/1878110082 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:30.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d57400700 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d172adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d4ea00700 1 -- 172.21.15.63:0/1878110082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 msgr2=0x55b7d1727e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.436+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 0x55b7d1727e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d4ea00700 1 -- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 msgr2=0x55b7d172adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d172adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d4ea00700 1 -- 172.21.15.63:0/1878110082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d1728560 con 0x55b7d171ea20 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171ea20 0x55b7d1727920 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1d400aa100 tx=0x7f1d400aa970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6a1d92edd18490 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d45600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d45600700 1 -- 172.21.15.63:0/1878110082 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400b2f40 con 0x55b7d171ea20 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d48038030 con 0x55b7d171ea20 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d45600700 1 -- 172.21.15.63:0/1878110082 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f1d400c46a0 con 0x55b7d171ea20 2024-05-11T13:56:30.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.437+0000 7f1d45600700 1 -- 172.21.15.63:0/1878110082 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1d400ae160 con 0x55b7d171ea20 2024-05-11T13:56:30.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 10 client.16257 fetch_fsmap learned FSMap version 77 2024-05-11T13:56:30.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55b7d17a60d0 con 0x55b7d171ea20 2024-05-11T13:56:30.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d45600700 1 -- 172.21.15.63:0/1878110082 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d48038030 con 0x55b7d171ea20 2024-05-11T13:56:30.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d45600700 1 -- 172.21.15.63:0/1878110082 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f1d4000f930 con 0x55b7d171ea20 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 77 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 10 client.16257 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 20 client.16257 populate_metadata read hostname 'smithi063' 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d17a6870 0x55b7d17a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 4 client.16257 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.85, "await_for": 42.0}] 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.438+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.85, "await_for": 42.0}) -- 0x55b7d1678590 con 0x55b7d17a6870 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.439+0000 7f1d57400700 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d17a6870 0x55b7d17a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.440+0000 7f1d57400700 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d17a6870 0x55b7d17a8c90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa72e14a0e80b96b server_cookie=f0b6a5a68b93dead in_seq=0 out_seq=0 2024-05-11T13:56:30.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.440+0000 7f1d45600700 10 client.16257 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:30.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:30 smithi063 ceph-mon[29809]: pgmap v715: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 4.0 MiB/s wr, 377 op/s 2024-05-11T13:56:30.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:30 smithi165 ceph-mon[33235]: pgmap v715: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 4.0 MiB/s wr, 377 op/s 2024-05-11T13:56:30.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.887+0000 7f1d45600700 1 -- 172.21.15.63:0/1878110082 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b7d1678590 con 0x55b7d17a6870 2024-05-11T13:56:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.887+0000 7f1d45600700 10 client.16257 handle_command_reply: tid=0 2024-05-11T13:56:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.887+0000 7f1d68d17b80 1 client.16257 shutdown 2024-05-11T13:56:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.887+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171ea20 msgr2=0x55b7d1727920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.887+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171ea20 0x55b7d1727920 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1d400aa100 tx=0x7f1d400aa970 comp rx=0 tx=0).stop 2024-05-11T13:56:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.887+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 shutdown_connections 2024-05-11T13:56:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7d17221d0 0x55b7d1727e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x55b7d17a6870 0x55b7d17a8c90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7d171e100 0x55b7d172adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d68d17b80 1 --2- 172.21.15.63:0/1878110082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b7d171ea20 0x55b7d1727920 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d45600700 0 client.16257 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 >> 172.21.15.63:0/1878110082 conn(0x55b7d1646cf0 msgr2=0x55b7d171bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 shutdown_connections 2024-05-11T13:56:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.888+0000 7f1d68d17b80 1 -- 172.21.15.63:0/1878110082 wait complete. 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.889+0000 7f1d68d17b80 20 client.16257 trim_cache size 0 max 16384 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 77, 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 66, 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout: "c88e3984": { 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 66, 2024-05-11T13:56:30.877 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 77.8, 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "QUIESCED", 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:56:30.878 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:30.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:30.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:30.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:30.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:30.879 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1d3c0b2b20 msgr2=0x7f1d3c0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1d3c0b2b20 0x7f1d3c0b4fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d5819caf0 tx=0x7f1d4800b040 comp rx=0 tx=0).stop 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 msgr2=0x7f1d581a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 0x7f1d581a1d30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1d400098e0 tx=0x7f1d400056a0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 shutdown_connections 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f1d3c0b2b20 0x7f1d3c0b4fe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d581059b0 0x7f1d5819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d58103700 0x7f1d581a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 --2- 172.21.15.63:0/2265025721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58104ff0 0x7f1d581a1d30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.890+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 >> 172.21.15.63:0/2265025721 conn(0x7f1d580feeb0 msgr2=0x7f1d580ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.891+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 shutdown_connections 2024-05-11T13:56:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:30.891+0000 7f1d5ce00700 1 -- 172.21.15.63:0/2265025721 wait complete. 2024-05-11T13:56:30.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:30 smithi071 ceph-mon[33293]: pgmap v715: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 4.0 MiB/s wr, 377 op/s 2024-05-11T13:56:31.597 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:31 smithi063 ceph-mon[29809]: pgmap v716: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.0 MiB/s wr, 393 op/s 2024-05-11T13:56:31.659 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c88e3984', quiesce took 12.7 seconds. Will release after: 17.85 2024-05-11T13:56:31.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:31 smithi165 ceph-mon[33235]: pgmap v716: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.0 MiB/s wr, 393 op/s 2024-05-11T13:56:31.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:31 smithi071 ceph-mon[33293]: pgmap v716: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.0 MiB/s wr, 393 op/s 2024-05-11T13:56:33.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:33 smithi063 ceph-mon[29809]: pgmap v717: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-05-11T13:56:33.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:33 smithi071 ceph-mon[33293]: pgmap v717: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-05-11T13:56:34.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:33 smithi165 ceph-mon[33235]: pgmap v717: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-05-11T13:56:35.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:35 smithi063 ceph-mon[29809]: pgmap v718: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 4.9 MiB/s wr, 493 op/s 2024-05-11T13:56:35.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:35 smithi071 ceph-mon[33293]: pgmap v718: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 4.9 MiB/s wr, 493 op/s 2024-05-11T13:56:36.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:35 smithi165 ceph-mon[33235]: pgmap v718: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 4.9 MiB/s wr, 493 op/s 2024-05-11T13:56:37.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:37 smithi063 ceph-mon[29809]: pgmap v719: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.1 MiB/s wr, 340 op/s 2024-05-11T13:56:37.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:37 smithi071 ceph-mon[33293]: pgmap v719: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.1 MiB/s wr, 340 op/s 2024-05-11T13:56:38.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:37 smithi165 ceph-mon[33235]: pgmap v719: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.1 MiB/s wr, 340 op/s 2024-05-11T13:56:39.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:39 smithi063 ceph-mon[29809]: pgmap v720: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-05-11T13:56:39.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:39 smithi071 ceph-mon[33293]: pgmap v720: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-05-11T13:56:40.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:39 smithi165 ceph-mon[33235]: pgmap v720: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-05-11T13:56:41.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:41 smithi063 ceph-mon[29809]: pgmap v721: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-11T13:56:41.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:41 smithi071 ceph-mon[33293]: pgmap v721: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-11T13:56:42.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:41 smithi165 ceph-mon[33235]: pgmap v721: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-11T13:56:43.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:43 smithi071 ceph-mon[33293]: pgmap v722: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 943 KiB/s wr, 115 op/s 2024-05-11T13:56:44.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:43 smithi165 ceph-mon[33235]: pgmap v722: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 943 KiB/s wr, 115 op/s 2024-05-11T13:56:44.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:43 smithi063 ceph-mon[29809]: pgmap v722: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 943 KiB/s wr, 115 op/s 2024-05-11T13:56:45.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:45 smithi071 ceph-mon[33293]: pgmap v723: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-11T13:56:46.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:45 smithi165 ceph-mon[33235]: pgmap v723: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-11T13:56:46.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:45 smithi063 ceph-mon[29809]: pgmap v723: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-11T13:56:48.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:47 smithi165 ceph-mon[33235]: pgmap v724: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 698 KiB/s wr, 80 op/s 2024-05-11T13:56:48.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:47 smithi063 ceph-mon[29809]: pgmap v724: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 698 KiB/s wr, 80 op/s 2024-05-11T13:56:48.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:47 smithi071 ceph-mon[33293]: pgmap v724: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 698 KiB/s wr, 80 op/s 2024-05-11T13:56:49.510 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:56:49.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:49 smithi063 ceph-mon[29809]: pgmap v725: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 723 KiB/s wr, 84 op/s 2024-05-11T13:56:50.010 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:56:50.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:49 smithi165 ceph-mon[33235]: pgmap v725: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 723 KiB/s wr, 84 op/s 2024-05-11T13:56:50.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:49 smithi071 ceph-mon[33293]: pgmap v725: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 723 KiB/s wr, 84 op/s 2024-05-11T13:56:51.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:50 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:56:51.169 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:50 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:56:51.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:50 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:56:52.034 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:51 smithi063 ceph-mon[29809]: pgmap v726: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 629 KiB/s wr, 71 op/s 2024-05-11T13:56:52.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:51 smithi165 ceph-mon[33235]: pgmap v726: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 629 KiB/s wr, 71 op/s 2024-05-11T13:56:52.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:51 smithi071 ceph-mon[33293]: pgmap v726: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 629 KiB/s wr, 71 op/s 2024-05-11T13:56:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 -- 172.21.15.63:0/394478566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6118103e60 msgr2=0x7f6118110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 --2- 172.21.15.63:0/394478566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6118103e60 0x7f6118110320 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f611000b0d0 tx=0x7f611000b3e0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 -- 172.21.15.63:0/394478566 shutdown_connections 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 --2- 172.21.15.63:0/394478566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6118103e60 0x7f6118110320 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 --2- 172.21.15.63:0/394478566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6118109550 0x7f6118109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 --2- 172.21.15.63:0/394478566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61181034a0 0x7f6118103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 -- 172.21.15.63:0/394478566 >> 172.21.15.63:0/394478566 conn(0x7f61180fefd0 msgr2=0x7f61181013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 -- 172.21.15.63:0/394478566 shutdown_connections 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.614+0000 7f6117200700 1 -- 172.21.15.63:0/394478566 wait complete. 2024-05-11T13:56:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.615+0000 7f6117200700 1 Processor -- start 2024-05-11T13:56:52.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6117200700 1 -- start start 2024-05-11T13:56:52.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6117200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61181034a0 0x7f61181a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6117200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6118103e60 0x7f61181a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6117200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6118109550 0x7f611819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6117200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6118113530 con 0x7f61181034a0 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6117200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f61181133b0 con 0x7f6118103e60 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6117200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61181136b0 con 0x7f6118109550 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6115e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61181034a0 0x7f61181a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6115e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61181034a0 0x7f61181a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38686/0 (socket says 172.21.15.63:38686) 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.616+0000 7f6115e00700 1 -- 172.21.15.63:0/918763837 learned_addr learned my addr 172.21.15.63:0/918763837 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:52.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6116800700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6118109550 0x7f611819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:52.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6115400700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6118103e60 0x7f61181a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:52.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6115e00700 1 -- 172.21.15.63:0/918763837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6118109550 msgr2=0x7f611819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:52.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6115e00700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6118109550 0x7f611819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6115e00700 1 -- 172.21.15.63:0/918763837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6118103e60 msgr2=0x7f61181a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:52.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6115e00700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6118103e60 0x7f61181a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6115e00700 1 -- 172.21.15.63:0/918763837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611819c130 con 0x7f61181034a0 2024-05-11T13:56:52.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.617+0000 7f6115e00700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61181034a0 0x7f61181a1810 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f610800d950 tx=0x7f610800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:52.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.618+0000 7f610ea00700 1 -- 172.21.15.63:0/918763837 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108016070 con 0x7f61181034a0 2024-05-11T13:56:52.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.618+0000 7f610ea00700 1 -- 172.21.15.63:0/918763837 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6108010460 con 0x7f61181034a0 2024-05-11T13:56:52.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.618+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611819c420 con 0x7f61181034a0 2024-05-11T13:56:52.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.618+0000 7f610ea00700 1 -- 172.21.15.63:0/918763837 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610800f620 con 0x7f61181034a0 2024-05-11T13:56:52.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.618+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61181a86e0 con 0x7f61181034a0 2024-05-11T13:56:52.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.619+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f611806a240 con 0x7f61181034a0 2024-05-11T13:56:52.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.622+0000 7f610ea00700 1 -- 172.21.15.63:0/918763837 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6108019070 con 0x7f61181034a0 2024-05-11T13:56:52.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.624+0000 7f610ea00700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60fc0b2a70 0x7f60fc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:52.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.624+0000 7f610ea00700 1 -- 172.21.15.63:0/918763837 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f61080e2500 con 0x7f61181034a0 2024-05-11T13:56:52.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.624+0000 7f6115400700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60fc0b2a70 0x7f60fc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:52.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.625+0000 7f6115400700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60fc0b2a70 0x7f60fc0b4f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6104005950 tx=0x7f61040058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:52.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.629+0000 7f610ea00700 1 -- 172.21.15.63:0/918763837 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61080a9b90 con 0x7f61181034a0 2024-05-11T13:56:52.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.925+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61181a8910 con 0x7f61181034a0 2024-05-11T13:56:52.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.927+0000 7f610ea00700 1 -- 172.21.15.63:0/918763837 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+11936 (secure 0 0 0) 0x7f61080a9490 con 0x7f61181034a0 2024-05-11T13:56:52.915 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:56:52.915 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":78,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:56:49.157057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60fc0b2a70 msgr2=0x7f60fc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60fc0b2a70 0x7f60fc0b4f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6104005950 tx=0x7f61040058e0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61181034a0 msgr2=0x7f61181a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61181034a0 0x7f61181a1810 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f610800d950 tx=0x7f610800dd10 comp rx=0 tx=0).stop 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 shutdown_connections 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60fc0b2a70 0x7f60fc0b4f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6118109550 0x7f611819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61181034a0 0x7f61181a1810 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 --2- 172.21.15.63:0/918763837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6118103e60 0x7f61181a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.931+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 >> 172.21.15.63:0/918763837 conn(0x7f61180fefd0 msgr2=0x7f6118100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.932+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 shutdown_connections 2024-05-11T13:56:52.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:52.932+0000 7f6117200700 1 -- 172.21.15.63:0/918763837 wait complete. 2024-05-11T13:56:52.920 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 78 2024-05-11T13:56:53.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:53 smithi063 ceph-mon[29809]: pgmap v727: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 629 KiB/s wr, 71 op/s 2024-05-11T13:56:53.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:53 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/918763837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:54.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:53 smithi165 ceph-mon[33235]: pgmap v727: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 629 KiB/s wr, 71 op/s 2024-05-11T13:56:54.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:53 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/918763837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:54.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:53 smithi071 ceph-mon[33293]: pgmap v727: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 629 KiB/s wr, 71 op/s 2024-05-11T13:56:54.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:53 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/918763837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:56:54.207 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --set-id c88e3984 --release --await' 2024-05-11T13:56:54.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --set-id c88e3984 --release --await 2024-05-11T13:56:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:56:56.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:55 smithi165 ceph-mon[33235]: pgmap v728: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 638 KiB/s wr, 75 op/s 2024-05-11T13:56:56.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:55 smithi063 ceph-mon[29809]: pgmap v728: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 638 KiB/s wr, 75 op/s 2024-05-11T13:56:56.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:55 smithi071 ceph-mon[33293]: pgmap v728: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 638 KiB/s wr, 75 op/s 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 -- 172.21.15.63:0/626745878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf78104fd0 msgr2=0x7faf7810d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 --2- 172.21.15.63:0/626745878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf78104fd0 0x7faf7810d4c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7faf64009990 tx=0x7faf64009ca0 comp rx=0 tx=0).stop 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 -- 172.21.15.63:0/626745878 shutdown_connections 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 --2- 172.21.15.63:0/626745878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7810dae0 0x7faf7810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 --2- 172.21.15.63:0/626745878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf781046b0 0x7faf78104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 --2- 172.21.15.63:0/626745878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf78104fd0 0x7faf7810d4c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 -- 172.21.15.63:0/626745878 >> 172.21.15.63:0/626745878 conn(0x7faf780fdf00 msgr2=0x7faf78100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 -- 172.21.15.63:0/626745878 shutdown_connections 2024-05-11T13:56:56.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.008+0000 7faf7e000700 1 -- 172.21.15.63:0/626745878 wait complete. 2024-05-11T13:56:56.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.009+0000 7faf7e000700 1 Processor -- start 2024-05-11T13:56:56.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.009+0000 7faf7e000700 1 -- start start 2024-05-11T13:56:56.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf781046b0 0x7faf7819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:56.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7e000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78104fd0 0x7faf7819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:56.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7810dae0 0x7faf781999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:56.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7e000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf781135d0 con 0x7faf7810dae0 2024-05-11T13:56:56.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7e000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf78113450 con 0x7faf781046b0 2024-05-11T13:56:56.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7e000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faf78113750 con 0x7faf78104fd0 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7810dae0 0x7faf781999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7810dae0 0x7faf781999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38710/0 (socket says 172.21.15.63:38710) 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7d600700 1 -- 172.21.15.63:0/1222752157 learned_addr learned my addr 172.21.15.63:0/1222752157 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf7cc00700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf781046b0 0x7faf7819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.010+0000 7faf77e00700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78104fd0 0x7faf7819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7d600700 1 -- 172.21.15.63:0/1222752157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78104fd0 msgr2=0x7faf7819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7d600700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78104fd0 0x7faf7819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7d600700 1 -- 172.21.15.63:0/1222752157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf781046b0 msgr2=0x7faf7819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:56.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7d600700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf781046b0 0x7faf7819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7d600700 1 -- 172.21.15.63:0/1222752157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7819a150 con 0x7faf7810dae0 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7cc00700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf781046b0 0x7faf7819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7d600700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7810dae0 0x7faf781999f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faf70007ee0 tx=0x7faf7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf75600700 1 -- 172.21.15.63:0/1222752157 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7000dbd0 con 0x7faf7810dae0 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf75600700 1 -- 172.21.15.63:0/1222752157 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf7000dd70 con 0x7faf7810dae0 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf75600700 1 -- 172.21.15.63:0/1222752157 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700058d0 con 0x7faf7810dae0 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.011+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7819a440 con 0x7faf7810dae0 2024-05-11T13:56:57.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.012+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf781a6700 con 0x7faf7810dae0 2024-05-11T13:56:57.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.016+0000 7faf75600700 1 -- 172.21.15.63:0/1222752157 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7faf7001a020 con 0x7faf7810dae0 2024-05-11T13:56:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.017+0000 7faf89f98b80 1 Processor -- start 2024-05-11T13:56:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.017+0000 7faf89f98b80 1 -- start start 2024-05-11T13:56:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.017+0000 7faf75600700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7faf600b2a70 0x7faf600b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf75600700 1 -- 172.21.15.63:0/1222752157 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7faf700e2200 con 0x7faf7810dae0 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc13e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc13efd0 0x5628bc13f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf89f98b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5628bc08cb20 con 0x5628bc13efd0 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf89f98b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5628bc02a860 con 0x5628bc144480 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf89f98b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5628bc088ae0 con 0x5628bc144da0 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf77e00700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7faf600b2a70 0x7faf600b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc13efd0 0x5628bc13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc13efd0 0x5628bc13f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38716/0 (socket says 172.21.15.63:38716) 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf7d600700 1 -- 172.21.15.63:0/2979316154 learned_addr learned my addr 172.21.15.63:0/2979316154 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.018+0000 7faf77e00700 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc13e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7cc00700 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7d600700 1 -- 172.21.15.63:0/2979316154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 msgr2=0x5628bc13e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7d600700 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc13e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7d600700 1 -- 172.21.15.63:0/2979316154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 msgr2=0x5628bc144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7d600700 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7d600700 1 -- 172.21.15.63:0/2979316154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628bbfd99d0 con 0x5628bc13efd0 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7cc00700 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf7d600700 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc13efd0 0x5628bc13f430 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faf700a5110 tx=0x7faf700a8800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdc4c00889400f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.019+0000 7faf77e00700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7faf600b2a70 0x7faf600b4f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf6400b5c0 tx=0x7faf64009510 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.020+0000 7faf6a000700 1 -- 172.21.15.63:0/2979316154 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700a9090 con 0x5628bc13efd0 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.020+0000 7faf6a000700 1 -- 172.21.15.63:0/2979316154 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf700ad440 con 0x5628bc13efd0 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.020+0000 7faf6a000700 1 -- 172.21.15.63:0/2979316154 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700a92f0 con 0x5628bc13efd0 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.020+0000 7faf89f98b80 1 -- 172.21.15.63:0/2979316154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc13efd0 msgr2=0x5628bc13f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.020+0000 7faf89f98b80 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc13efd0 0x5628bc13f430 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faf700a5110 tx=0x7faf700a8800 comp rx=0 tx=0).stop 2024-05-11T13:56:57.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 -- 172.21.15.63:0/2979316154 shutdown_connections 2024-05-11T13:56:57.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc13efd0 0x5628bc13f430 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 --2- 172.21.15.63:0/2979316154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc13e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 -- 172.21.15.63:0/2979316154 >> 172.21.15.63:0/2979316154 conn(0x5628bc068c60 msgr2=0x5628bc135100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:57.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 -- 172.21.15.63:0/2979316154 shutdown_connections 2024-05-11T13:56:57.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 -- 172.21.15.63:0/2979316154 wait complete. 2024-05-11T13:56:57.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.021+0000 7faf89f98b80 1 -- start start 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 1 Processor -- start 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144480 0x5628bc147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc148410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5628bc14b8a0 con 0x5628bc144480 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5628bbff2c20 con 0x5628bc13efd0 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5628bc14ba20 con 0x5628bc144da0 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.022+0000 7faf89f98b80 10 client.0 _ll_register_callbacks cb 0x5628bc033d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144480 0x5628bc147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144480 0x5628bc147ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38722/0 (socket says 172.21.15.63:38722) 2024-05-11T13:56:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 -- 172.21.15.63:0/4145513823 learned_addr learned my addr 172.21.15.63:0/4145513823 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf7d600700 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf7cc00700 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 -- 172.21.15.63:0/4145513823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 msgr2=0x5628bc148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 -- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 msgr2=0x5628bc14ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc14ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.023+0000 7faf77e00700 1 -- 172.21.15.63:0/4145513823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628bc052520 con 0x5628bc144480 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf77e00700 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144480 0x5628bc147ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faf64015000 tx=0x7faf64027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5525ea4483d8233d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf5ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf5ea00700 1 -- 172.21.15.63:0/4145513823 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6402d070 con 0x5628bc144480 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf5ea00700 1 -- 172.21.15.63:0/4145513823 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7faf64005510 con 0x5628bc144480 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf700ad440 con 0x5628bc144480 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf5ea00700 1 -- 172.21.15.63:0/4145513823 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7faf6403c050 con 0x5628bc144480 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf89f98b80 10 client.16275 fetch_fsmap learned FSMap version 78 2024-05-11T13:56:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.024+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5628bc156150 con 0x5628bc144480 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf5ea00700 1 -- 172.21.15.63:0/4145513823 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf700ad440 con 0x5628bc144480 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf5ea00700 1 -- 172.21.15.63:0/4145513823 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 78) ==== 8366+0+0 (secure 0 0 0) 0x7faf64031400 con 0x5628bc144480 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf89f98b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 78 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf89f98b80 10 client.16275 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf89f98b80 20 client.16275 populate_metadata read hostname 'smithi063' 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf89f98b80 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1587e0 0x5628bc15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf89f98b80 4 client.16275 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.025+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628bc1072b0 con 0x5628bc1587e0 2024-05-11T13:56:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.026+0000 7faf7cc00700 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1587e0 0x5628bc15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.027+0000 7faf7cc00700 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1587e0 0x5628bc15ac00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3df1757a7714b7c server_cookie=a5f86537959b5ffa in_seq=0 out_seq=0 2024-05-11T13:56:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.027+0000 7faf5ea00700 10 client.16275 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf5ea00700 1 -- 172.21.15.63:0/4145513823 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628bc1072b0 con 0x5628bc1587e0 2024-05-11T13:56:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf5ea00700 10 client.16275 handle_command_reply: tid=0 2024-05-11T13:56:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 client.16275 shutdown 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144480 msgr2=0x5628bc147ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144480 0x5628bc147ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faf64015000 tx=0x7faf64027000 comp rx=0 tx=0).stop 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 shutdown_connections 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144da0 0x5628bc148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1587e0 0x5628bc15ac00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 --2- 172.21.15.63:0/4145513823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144480 0x5628bc147ed0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf5ea00700 0 client.16275 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 >> 172.21.15.63:0/4145513823 conn(0x5628bc068c60 msgr2=0x5628bc139590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 shutdown_connections 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.029+0000 7faf89f98b80 1 -- 172.21.15.63:0/4145513823 wait complete. 2024-05-11T13:56:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.030+0000 7faf89f98b80 20 client.16275 trim_cache size 0 max 16384 2024-05-11T13:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.051+0000 7faf89f98b80 1 Processor -- start 2024-05-11T13:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf89f98b80 1 -- start start 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc145850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144480 0x5628bc154930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc147050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf89f98b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf781ad2b0 con 0x5628bc144da0 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf89f98b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf781aee20 con 0x5628bc13efd0 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf89f98b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faf781ad980 con 0x5628bc144480 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38736/0 (socket says 172.21.15.63:38736) 2024-05-11T13:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.052+0000 7faf7d600700 1 -- 172.21.15.63:0/3393497482 learned_addr learned my addr 172.21.15.63:0/3393497482 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf7cc00700 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf77e00700 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144480 0x5628bc154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf7d600700 1 -- 172.21.15.63:0/3393497482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144480 msgr2=0x5628bc154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf7d600700 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144480 0x5628bc154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf7d600700 1 -- 172.21.15.63:0/3393497482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 msgr2=0x5628bc145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf7d600700 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf7d600700 1 -- 172.21.15.63:0/3393497482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628bc13f790 con 0x5628bc144da0 2024-05-11T13:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf7d600700 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc147050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faf70005c30 tx=0x7faf700a4f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38042ad65320e243 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf69600700 1 -- 172.21.15.63:0/3393497482 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700b1ef0 con 0x5628bc144da0 2024-05-11T13:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf69600700 1 -- 172.21.15.63:0/3393497482 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf700a9090 con 0x5628bc144da0 2024-05-11T13:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.053+0000 7faf69600700 1 -- 172.21.15.63:0/3393497482 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700a9550 con 0x5628bc144da0 2024-05-11T13:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 -- 172.21.15.63:0/3393497482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 msgr2=0x5628bc147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc147050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faf70005c30 tx=0x7faf700a4f50 comp rx=0 tx=0).stop 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 -- 172.21.15.63:0/3393497482 shutdown_connections 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc147050 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc13efd0 0x5628bc145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3393497482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc144480 0x5628bc154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 -- 172.21.15.63:0/3393497482 >> 172.21.15.63:0/3393497482 conn(0x5628bc068c60 msgr2=0x5628bc139590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 -- 172.21.15.63:0/3393497482 shutdown_connections 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.054+0000 7faf89f98b80 1 -- 172.21.15.63:0/3393497482 wait complete. 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.055+0000 7faf89f98b80 1 -- start start 2024-05-11T13:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.055+0000 7faf89f98b80 1 Processor -- start 2024-05-11T13:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.055+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc13efd0 0x5628bc151010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.055+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc14e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.055+0000 7faf89f98b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc14e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf89f98b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf781ae720 con 0x5628bc144da0 2024-05-11T13:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf89f98b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf781ae050 con 0x5628bc144480 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf89f98b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faf781ab270 con 0x5628bc13efd0 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf89f98b80 10 client.0 _ll_register_callbacks cb 0x5628bc033d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc14e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc14e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38748/0 (socket says 172.21.15.63:38748) 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 -- 172.21.15.63:0/3407276262 learned_addr learned my addr 172.21.15.63:0/3407276262 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf77e00700 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc14e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7cc00700 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc13efd0 0x5628bc151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 -- 172.21.15.63:0/3407276262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc13efd0 msgr2=0x5628bc151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc13efd0 0x5628bc151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 -- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 msgr2=0x5628bc14e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc14e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.056+0000 7faf7d600700 1 -- 172.21.15.63:0/3407276262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628bc14ef00 con 0x5628bc144da0 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf7d600700 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc14e8f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7faf700a8fd0 tx=0x7faf700a8b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd5e7d98b7c2042 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf5ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf5ea00700 1 -- 172.21.15.63:0/3407276262 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700c8060 con 0x5628bc144da0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf5ea00700 1 -- 172.21.15.63:0/3407276262 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7faf700a5680 con 0x5628bc144da0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf700a9090 con 0x5628bc144da0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf5ea00700 1 -- 172.21.15.63:0/3407276262 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7faf700c2030 con 0x5628bc144da0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf89f98b80 10 client.16281 fetch_fsmap learned FSMap version 78 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.057+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5628bc1c4f30 con 0x5628bc144da0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf5ea00700 1 -- 172.21.15.63:0/3407276262 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf700a9090 con 0x5628bc144da0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf5ea00700 1 -- 172.21.15.63:0/3407276262 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 78) ==== 8366+0+0 (secure 0 0 0) 0x7faf700a8380 con 0x5628bc144da0 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf89f98b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 78 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf89f98b80 10 client.16281 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf89f98b80 20 client.16281 populate_metadata read hostname 'smithi063' 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1c5100 0x5628bc1c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf89f98b80 4 client.16281 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c88e3984", "release": true, "await": true}] 2024-05-11T13:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c88e3984", "release": true, "await": true}) -- 0x5628bc09a500 con 0x5628bc1c5100 2024-05-11T13:56:57.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.058+0000 7faf7cc00700 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1c5100 0x5628bc1c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:56:57.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.059+0000 7faf7cc00700 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1c5100 0x5628bc1c74f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=375426a58e1a9864 server_cookie=8fd121eb16d16dfe in_seq=0 out_seq=0 2024-05-11T13:56:57.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.059+0000 7faf5ea00700 10 client.16281 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.079+0000 7faf5ea00700 1 -- 172.21.15.63:0/3407276262 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5628bc09a500 con 0x5628bc1c5100 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.079+0000 7faf5ea00700 10 client.16281 handle_command_reply: tid=0 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.079+0000 7faf89f98b80 1 client.16281 shutdown 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 msgr2=0x5628bc14e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc14e8f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7faf700a8fd0 tx=0x7faf700a8b10 comp rx=0 tx=0).stop 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 shutdown_connections 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628bc144da0 0x5628bc14e8f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628bc13efd0 0x5628bc151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x5628bc1c5100 0x5628bc1c74f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 --2- 172.21.15.63:0/3407276262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628bc144480 0x5628bc14e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 >> 172.21.15.63:0/3407276262 conn(0x5628bc068c60 msgr2=0x5628bc139e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf5ea00700 0 client.16281 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.080+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 shutdown_connections 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.081+0000 7faf89f98b80 1 -- 172.21.15.63:0/3407276262 wait complete. 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.081+0000 7faf89f98b80 20 client.16281 trim_cache size 0 max 16384 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 78, 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 68, 2024-05-11T13:56:57.069 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": { 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "c88e3984": { 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "version": 68, 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "age_ref": 0.0, 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "timeout": 60.0, 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "expiration": 77.8, 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "members": { 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "file:/": { 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "excluded": false, 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "state": { 2024-05-11T13:56:57.070 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RELEASED", 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:56:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7faf600b2a70 msgr2=0x7faf600b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7faf600b2a70 0x7faf600b4f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf6400b5c0 tx=0x7faf64009510 comp rx=0 tx=0).stop 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7810dae0 msgr2=0x7faf781999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7810dae0 0x7faf781999f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faf70007ee0 tx=0x7faf7000d3b0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 shutdown_connections 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7faf600b2a70 0x7faf600b4f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7810dae0 0x7faf781999f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf781046b0 0x7faf7819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 --2- 172.21.15.63:0/1222752157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78104fd0 0x7faf7819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 >> 172.21.15.63:0/1222752157 conn(0x7faf780fdf00 msgr2=0x7faf780fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 shutdown_connections 2024-05-11T13:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:56:57.082+0000 7faf7e000700 1 -- 172.21.15.63:0/1222752157 wait complete. 2024-05-11T13:56:58.027 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:57 smithi063 ceph-mon[29809]: pgmap v729: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 9 op/s 2024-05-11T13:56:58.031 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c88e3984', seconds elapsed: 39.1 2024-05-11T13:56:58.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:57 smithi165 ceph-mon[33235]: pgmap v729: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 9 op/s 2024-05-11T13:56:58.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:57 smithi071 ceph-mon[33293]: pgmap v729: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 9 op/s 2024-05-11T13:57:00.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:56:59 smithi165 ceph-mon[33235]: pgmap v730: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 106 KiB/s wr, 22 op/s 2024-05-11T13:57:00.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:56:59 smithi063 ceph-mon[29809]: pgmap v730: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 106 KiB/s wr, 22 op/s 2024-05-11T13:57:00.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:56:59 smithi071 ceph-mon[33293]: pgmap v730: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 106 KiB/s wr, 22 op/s 2024-05-11T13:57:02.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:01 smithi165 ceph-mon[33235]: pgmap v731: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 81 KiB/s wr, 18 op/s 2024-05-11T13:57:02.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:01 smithi063 ceph-mon[29809]: pgmap v731: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 81 KiB/s wr, 18 op/s 2024-05-11T13:57:02.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:01 smithi071 ceph-mon[33293]: pgmap v731: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 81 KiB/s wr, 18 op/s 2024-05-11T13:57:04.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:03 smithi165 ceph-mon[33235]: pgmap v732: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 81 KiB/s wr, 17 op/s 2024-05-11T13:57:04.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:03 smithi063 ceph-mon[29809]: pgmap v732: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 81 KiB/s wr, 17 op/s 2024-05-11T13:57:04.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:03 smithi071 ceph-mon[33293]: pgmap v732: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 81 KiB/s wr, 17 op/s 2024-05-11T13:57:06.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:05 smithi165 ceph-mon[33235]: pgmap v733: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 428 KiB/s wr, 76 op/s 2024-05-11T13:57:06.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:05 smithi063 ceph-mon[29809]: pgmap v733: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 428 KiB/s wr, 76 op/s 2024-05-11T13:57:06.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:05 smithi071 ceph-mon[33293]: pgmap v733: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 428 KiB/s wr, 76 op/s 2024-05-11T13:57:08.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:07 smithi165 ceph-mon[33235]: pgmap v734: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 419 KiB/s wr, 72 op/s 2024-05-11T13:57:08.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:07 smithi063 ceph-mon[29809]: pgmap v734: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 419 KiB/s wr, 72 op/s 2024-05-11T13:57:08.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:07 smithi071 ceph-mon[33293]: pgmap v734: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 419 KiB/s wr, 72 op/s 2024-05-11T13:57:10.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:09 smithi165 ceph-mon[33235]: pgmap v735: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 601 KiB/s wr, 98 op/s 2024-05-11T13:57:10.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:09 smithi063 ceph-mon[29809]: pgmap v735: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 601 KiB/s wr, 98 op/s 2024-05-11T13:57:10.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:09 smithi071 ceph-mon[33293]: pgmap v735: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 601 KiB/s wr, 98 op/s 2024-05-11T13:57:11.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:10 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:11.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:10 smithi063 ceph-mon[29809]: osdmap e126: 12 total, 12 up, 12 in 2024-05-11T13:57:11.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:10 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:57:11.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:10 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:11.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:10 smithi071 ceph-mon[33293]: osdmap e126: 12 total, 12 up, 12 in 2024-05-11T13:57:11.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:10 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:57:11.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:10 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:11.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:10 smithi165 ceph-mon[33235]: osdmap e126: 12 total, 12 up, 12 in 2024-05-11T13:57:11.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:10 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:57:12.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:11 smithi063 ceph-mon[29809]: pgmap v737: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 792 KiB/s wr, 122 op/s 2024-05-11T13:57:12.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:11 smithi071 ceph-mon[33293]: pgmap v737: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 792 KiB/s wr, 122 op/s 2024-05-11T13:57:12.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:11 smithi165 ceph-mon[33235]: pgmap v737: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 792 KiB/s wr, 122 op/s 2024-05-11T13:57:13.093 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:12 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:57:13.093 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:12 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:57:13.093 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:12 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:57:13.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:12 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:57:13.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:12 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:57:13.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:12 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:57:13.205 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:12 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:57:13.205 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:12 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:57:13.205 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:12 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:57:14.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:13 smithi063 ceph-mon[29809]: pgmap v738: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 792 KiB/s wr, 122 op/s 2024-05-11T13:57:14.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:13 smithi071 ceph-mon[33293]: pgmap v738: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 792 KiB/s wr, 122 op/s 2024-05-11T13:57:14.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:13 smithi165 ceph-mon[33235]: pgmap v738: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 792 KiB/s wr, 122 op/s 2024-05-11T13:57:16.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:16 smithi063 ceph-mon[29809]: pgmap v739: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 658 KiB/s wr, 66 op/s 2024-05-11T13:57:16.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:16 smithi063 ceph-mon[29809]: osdmap e127: 12 total, 12 up, 12 in 2024-05-11T13:57:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:16 smithi071 ceph-mon[33293]: pgmap v739: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 658 KiB/s wr, 66 op/s 2024-05-11T13:57:16.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:16 smithi071 ceph-mon[33293]: osdmap e127: 12 total, 12 up, 12 in 2024-05-11T13:57:16.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:16 smithi165 ceph-mon[33235]: pgmap v739: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 658 KiB/s wr, 66 op/s 2024-05-11T13:57:16.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:16 smithi165 ceph-mon[33235]: osdmap e127: 12 total, 12 up, 12 in 2024-05-11T13:57:18.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:18 smithi063 ceph-mon[29809]: pgmap v741: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 551 KiB/s wr, 44 op/s 2024-05-11T13:57:18.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:18 smithi071 ceph-mon[33293]: pgmap v741: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 551 KiB/s wr, 44 op/s 2024-05-11T13:57:18.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:18 smithi165 ceph-mon[33235]: pgmap v741: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 551 KiB/s wr, 44 op/s 2024-05-11T13:57:20.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:20 smithi071 ceph-mon[33293]: pgmap v742: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 615 KiB/s wr, 32 op/s 2024-05-11T13:57:20.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:20 smithi165 ceph-mon[33235]: pgmap v742: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 615 KiB/s wr, 32 op/s 2024-05-11T13:57:20.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:20 smithi063 ceph-mon[29809]: pgmap v742: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 615 KiB/s wr, 32 op/s 2024-05-11T13:57:22.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:22 smithi063 ceph-mon[29809]: pgmap v743: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 543 KiB/s wr, 29 op/s 2024-05-11T13:57:22.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:22 smithi071 ceph-mon[33293]: pgmap v743: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 543 KiB/s wr, 29 op/s 2024-05-11T13:57:22.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:22 smithi165 ceph-mon[33235]: pgmap v743: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 543 KiB/s wr, 29 op/s 2024-05-11T13:57:24.270 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-11T13:57:24.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:24 smithi063 ceph-mon[29809]: pgmap v744: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 543 KiB/s wr, 29 op/s 2024-05-11T13:57:24.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:24 smithi165 ceph-mon[33235]: pgmap v744: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 543 KiB/s wr, 29 op/s 2024-05-11T13:57:24.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:24 smithi071 ceph-mon[33293]: pgmap v744: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 543 KiB/s wr, 29 op/s 2024-05-11T13:57:25.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:25 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:25.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:25 smithi165 ceph-mon[33235]: pgmap v745: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-05-11T13:57:25.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:25 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:25.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:25 smithi063 ceph-mon[29809]: pgmap v745: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-05-11T13:57:25.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:25 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:25.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:25 smithi071 ceph-mon[33293]: pgmap v745: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-05-11T13:57:26.390 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/: 2024-05-11T13:57:26.390 INFO:teuthology.orchestra.run.smithi063.stdout:total 2 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1218344441 May 11 13:34 . 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1218344441 May 11 13:34 .. 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218344441 May 11 13:57 data 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data: 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:total 46 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218344441 May 11 13:57 . 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1218344441 May 11 13:34 .. 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 546485813 May 11 13:40 base 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 11 13:35 current_logfiles 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 11 13:35 global 2024-05-11T13:57:26.391 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1516 May 11 13:35 log 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_commit_ts 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_dynshmem 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 11 13:34 pg_hba.conf 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 11 13:34 pg_ident.conf 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:57 pg_logical 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 pg_multixact 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_notify 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_replslot 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_serial 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_snapshots 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 11 13:57 pg_stat 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:57 pg_stat_tmp 2024-05-11T13:57:26.392 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 11 13:57 pg_subtrans 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_tblspc 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 pg_twophase 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 671088640 May 11 13:57 pg_wal 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 pg_xact 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 11 13:34 postgresql.auto.conf 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 11 13:34 postgresql.conf 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 11 13:35 postmaster.opts 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base: 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:total 3 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 546485813 May 11 13:40 . 2024-05-11T13:57:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218344441 May 11 13:57 .. 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 1 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 13493 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 530559535 May 11 13:48 13494 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:41 pgsql_tmp 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:total 7778 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 . 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 546485813 May 11 13:40 .. 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 112 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 113 2024-05-11T13:57:26.394 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 1247 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1247_fsm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1247_vm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 11 13:34 1249 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1249_fsm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1249_vm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 11 13:34 1255 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1255_fsm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1255_vm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 11 13:34 1259 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1259_fsm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1259_vm 2024-05-11T13:57:26.395 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 11 13:34 13344 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13344_fsm 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13344_vm 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13346 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13348 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13349_fsm 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349_vm 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13351 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13353 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13354_fsm 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354_vm 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13356 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13358 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359 2024-05-11T13:57:26.396 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13359_fsm 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359_vm 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13361 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13363 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1417 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1418 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 174 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 175 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2187 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2224 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2328 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2336 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2337 2024-05-11T13:57:26.397 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2579 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2600 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2600_fsm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2600_vm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2601_fsm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601_vm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2602 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2602_fsm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2602_vm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2603 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2603_fsm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2603_vm 2024-05-11T13:57:26.398 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2604 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2605 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2605_fsm 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2605_vm 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2606_fsm 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606_vm 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2607 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2607_fsm 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2607_vm 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 11 13:34 2608 2024-05-11T13:57:26.399 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2608_fsm 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2608_vm 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 11 13:34 2609 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2609_fsm 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2609_vm 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2610 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2610_fsm 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2610_vm 2024-05-11T13:57:26.400 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2611 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2612_fsm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612_vm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2613 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2615_fsm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615_vm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2616 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2616_fsm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2616_vm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 11 13:34 2617 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2617_fsm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2617_vm 2024-05-11T13:57:26.401 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 11 13:34 2618 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2618_fsm 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2618_vm 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 11 13:34 2619 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2619_fsm 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2619_vm 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2620 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2650 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2651 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2652 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2653 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2654 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2655 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2656 2024-05-11T13:57:26.402 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2657 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 11 13:34 2658 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 2659 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2660 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2661 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2662 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2663 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2664 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2665 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2666 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2667 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2668 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2669 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2670 2024-05-11T13:57:26.403 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 11 13:34 2673 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 11 13:34 2674 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 11 13:34 2675 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2678 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2679 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2680 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2681 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2682 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2683 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2684 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2685 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2686 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2687 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2688 2024-05-11T13:57:26.404 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2689 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 11 13:34 2690 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 11 13:34 2691 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2692 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2693 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2696 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2699 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2701 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2702 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2703 2024-05-11T13:57:26.405 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2704 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2753 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2753_fsm 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2753_vm 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2754 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2755 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2756 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2757 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2830 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2831 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2832 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2833 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2834 2024-05-11T13:57:26.406 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2835 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2836 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2837 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 11 13:34 2838 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2838_fsm 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2838_vm 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2839 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840_fsm 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2840_vm 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2841 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2995 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2996 2024-05-11T13:57:26.407 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3079_fsm 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079_vm 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3080 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3081 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3085 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3118 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3119 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 3164 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3256 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3257 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3258 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3350 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3351 2024-05-11T13:57:26.408 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3379 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3380 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3381 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3394 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3394_fsm 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3394_vm 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3395 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3429 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3430 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3431 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3433 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3439 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3440 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3455 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 11 13:34 3456 2024-05-11T13:57:26.409 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3456_fsm 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3456_vm 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3466 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3467 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3468 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3501 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3502 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3503 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3534 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3541_fsm 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541_vm 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3542 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3574 2024-05-11T13:57:26.410 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3575 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3576 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3596 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3597 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3598 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3599 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3600_fsm 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600_vm 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3601_fsm 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601_vm 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3602_fsm 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602_vm 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603 2024-05-11T13:57:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603_fsm 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3603_vm 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3604 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3605 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3606 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3607 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3608 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 3609 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3712 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3764_fsm 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764_vm 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3766 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3767 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3997 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4143 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4144 2024-05-11T13:57:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4145 2024-05-11T13:57:26.413 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4146 2024-05-11T13:57:26.413 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4147 2024-05-11T13:57:26.413 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4148 2024-05-11T13:57:26.413 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4149 2024-05-11T13:57:26.413 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4150 2024-05-11T13:57:26.413 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4151 2024-05-11T13:57:27.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:27 smithi071 ceph-mon[33293]: pgmap v746: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 2.5 MiB/s wr, 286 op/s 2024-05-11T13:57:28.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:27 smithi165 ceph-mon[33235]: pgmap v746: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 2.5 MiB/s wr, 286 op/s 2024-05-11T13:57:28.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:27 smithi063 ceph-mon[29809]: pgmap v746: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 2.5 MiB/s wr, 286 op/s 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4152 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4153 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4154 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4155 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4156 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4157 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4158 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4159 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4160 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4161 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4162 2024-05-11T13:57:28.272 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4163 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4164 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4165 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4166 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4167 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4168 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4169 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4170 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4171 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4172 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4173 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4174 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 5002 2024-05-11T13:57:28.273 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 548 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 549 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6102 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6104 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6106 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6110 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6111 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6112 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6113 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6117 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 826 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 827 2024-05-11T13:57:28.274 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 828 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:total 7778 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 11 13:34 . 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 546485813 May 11 13:40 .. 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 112 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 113 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 1247 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1247_fsm 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1247_vm 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 11 13:34 1249 2024-05-11T13:57:28.275 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1249_fsm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1249_vm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 11 13:34 1255 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1255_fsm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1255_vm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 11 13:34 1259 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1259_fsm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1259_vm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 11 13:34 13344 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13344_fsm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13344_vm 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13346 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13348 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349 2024-05-11T13:57:28.276 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13349_fsm 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349_vm 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13351 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13353 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13354_fsm 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354_vm 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13356 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13358 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13359_fsm 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359_vm 2024-05-11T13:57:28.277 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13361 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13363 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1417 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1418 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 174 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 175 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2187 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2224 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2328 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2336 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2337 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2579 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2600 2024-05-11T13:57:28.278 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2600_fsm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2600_vm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2601_fsm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601_vm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2602 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2602_fsm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2602_vm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2603 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2603_fsm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2603_vm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2604 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2605 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2605_fsm 2024-05-11T13:57:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2605_vm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2606_fsm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2606_vm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2607 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2607_fsm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2607_vm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 11 13:34 2608 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2608_fsm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2608_vm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 11 13:34 2609 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2609_fsm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2609_vm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2610 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2610_fsm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2610_vm 2024-05-11T13:57:28.280 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2611 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2612_fsm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612_vm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2613 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2615_fsm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615_vm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2616 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2616_fsm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2616_vm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 11 13:34 2617 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2617_fsm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2617_vm 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 11 13:34 2618 2024-05-11T13:57:28.281 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2618_fsm 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2618_vm 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 11 13:34 2619 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2619_fsm 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2619_vm 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2620 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2650 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2651 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2652 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2653 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2654 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2655 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2656 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2657 2024-05-11T13:57:28.282 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 11 13:34 2658 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:34 2659 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2660 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2661 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2662 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2663 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2664 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2665 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2666 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2667 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2668 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2669 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2670 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 11 13:34 2673 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 11 13:34 2674 2024-05-11T13:57:28.283 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 11 13:34 2675 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2678 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2679 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2680 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2681 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2682 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2683 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2684 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2685 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2686 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2687 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2688 2024-05-11T13:57:28.284 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2689 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 11 13:34 2690 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 11 13:34 2691 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2692 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2693 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2696 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2699 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2701 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2702 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2703 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2704 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2753 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2753_fsm 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2753_vm 2024-05-11T13:57:28.285 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2754 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2755 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2756 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2757 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2830 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2831 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2832 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2833 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2834 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2835 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2836 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2837 2024-05-11T13:57:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 11 13:34 2838 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2838_fsm 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2838_vm 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2839 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840_fsm 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2840_vm 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2841 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2995 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2996 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3079_fsm 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079_vm 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3080 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3081 2024-05-11T13:57:28.287 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3085 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3118 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3119 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 3164 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3256 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3257 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3258 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3350 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3351 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3379 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3380 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3381 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3394 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3394_fsm 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3394_vm 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3395 2024-05-11T13:57:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3429 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3430 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3431 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3433 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3439 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3440 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3455 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 11 13:34 3456 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3456_fsm 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3456_vm 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3466 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3467 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3468 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3501 2024-05-11T13:57:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3502 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3503 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3534 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3541_fsm 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541_vm 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3542 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3574 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3575 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3576 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3596 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3597 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3598 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3599 2024-05-11T13:57:28.290 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3600_fsm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600_vm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3601_fsm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601_vm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3602_fsm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602_vm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603_fsm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3603_vm 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3604 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3605 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3606 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3607 2024-05-11T13:57:28.291 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3608 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 3609 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3712 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3764_fsm 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764_vm 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3766 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3767 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3997 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4143 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4144 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4145 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4146 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4147 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4148 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4149 2024-05-11T13:57:28.292 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4150 2024-05-11T13:57:28.293 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4151 2024-05-11T13:57:28.293 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4152 2024-05-11T13:57:28.293 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4153 2024-05-11T13:57:29.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:29 smithi071 ceph-mon[33293]: pgmap v747: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-05-11T13:57:30.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:29 smithi165 ceph-mon[33235]: pgmap v747: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-05-11T13:57:30.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:29 smithi063 ceph-mon[29809]: pgmap v747: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-05-11T13:57:30.438 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4154 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4155 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4156 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4157 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4158 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4159 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4160 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4161 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4162 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4163 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4164 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4165 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4166 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4167 2024-05-11T13:57:30.439 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4168 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4169 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4170 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4171 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4172 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4173 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4174 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 5002 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 548 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 549 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6102 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6104 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6106 2024-05-11T13:57:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6110 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6111 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6112 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6113 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6117 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 826 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 827 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 828 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:total 518127 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 530559535 May 11 13:48 . 2024-05-11T13:57:30.441 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 546485813 May 11 13:40 .. 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 112 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 113 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:40 1247 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1247_fsm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:40 1247_vm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 11 13:42 1249 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:40 1249_fsm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:40 1249_vm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 11 13:34 1255 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1255_fsm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1255_vm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 11 13:57 1259 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1259_fsm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:40 1259_vm 2024-05-11T13:57:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 11 13:34 13344 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13344_fsm 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13344_vm 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13346 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13348 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13349_fsm 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13349_vm 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13351 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13353 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13354_fsm 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13354_vm 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13356 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13358 2024-05-11T13:57:30.443 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 13359_fsm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13359_vm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 13361 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 13363 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1417 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 1418 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 437256192 May 11 13:57 16396 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 11 13:54 16396_fsm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:56 16396_vm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 679936 May 11 13:56 16397 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:56 16397_fsm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:56 16397_vm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 524288 May 11 13:56 16399 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:56 16399_fsm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:56 16399_vm 2024-05-11T13:57:30.444 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:47 16400 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:47 16402 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 11 13:57 16404 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 11739136 May 11 13:57 16406 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:57 16406_fsm 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:57 16406_vm 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 174 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 175 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2187 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2224 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2328 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2336 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2337 2024-05-11T13:57:30.445 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:42 2579 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2600 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2600_fsm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2600_vm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2601_fsm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2601_vm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2602 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2602_fsm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2602_vm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2603 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2603_fsm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2603_vm 2024-05-11T13:57:30.446 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2604 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2605 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2605_fsm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2605_vm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:42 2606 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2606_fsm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:41 2606_vm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2607 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2607_fsm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2607_vm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 11 13:42 2608 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2608_fsm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:40 2608_vm 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 11 13:34 2609 2024-05-11T13:57:30.447 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2609_fsm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2609_vm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:42 2610 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2610_fsm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:41 2610_vm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2611 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2612_fsm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2612_vm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2613 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2615_fsm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2615_vm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2616 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2616_fsm 2024-05-11T13:57:30.448 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2616_vm 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 11 13:34 2617 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2617_fsm 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2617_vm 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 11 13:34 2618 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2618_fsm 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2618_vm 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 May 11 13:57 2619 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:43 2619_fsm 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:40 2619_vm 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2620 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2650 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2651 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2652 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2653 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2654 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2655 2024-05-11T13:57:30.449 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2656 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2657 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 11 13:42 2658 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:42 2659 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2660 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2661 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:42 2662 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:42 2663 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:42 2664 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:42 2665 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:42 2666 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:42 2667 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2668 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2669 2024-05-11T13:57:30.450 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2670 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 11 13:42 2673 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 11 13:42 2674 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 11 13:34 2675 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:42 2678 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:42 2679 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2680 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2681 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2682 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2683 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2684 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2685 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2686 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2687 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 2688 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 11 13:34 2689 2024-05-11T13:57:30.451 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 11 13:34 2690 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 11 13:34 2691 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2692 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2693 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:45 2696 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2699 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2701 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2702 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:40 2703 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:40 2704 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2753 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2753_fsm 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2753_vm 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2754 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2755 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2756 2024-05-11T13:57:30.452 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 2757 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2830 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2831 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2832 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2833 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2834 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2835 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2836 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2837 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 11 13:34 2838 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2838_fsm 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2838_vm 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2839 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2840_fsm 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2840_vm 2024-05-11T13:57:30.453 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2841 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2995 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2996 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3079_fsm 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3079_vm 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3080 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3081 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:34 3085 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3118 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3119 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 3164 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3256 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3257 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3258 2024-05-11T13:57:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3350 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3351 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3379 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3380 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3381 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3394 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3394_fsm 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3394_vm 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3395 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3429 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3430 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3431 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3433 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3439 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3440 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 11 13:42 3455 2024-05-11T13:57:30.455 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 11 13:34 3456 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3456_fsm 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3456_vm 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3466 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3467 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3468 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3501 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3502 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3503 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3534 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3541_fsm 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3541_vm 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3542 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3574 2024-05-11T13:57:30.456 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3575 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3576 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3596 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3597 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3598 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3599 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3600_fsm 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3600_vm 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3601_fsm 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3601_vm 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3602_fsm 2024-05-11T13:57:30.457 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3602_vm 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3603_fsm 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3603_vm 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3604 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3605 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3606 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3607 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3608 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 11 13:34 3609 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3712 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 3764_fsm 2024-05-11T13:57:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3764_vm 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3766 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 3767 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3997 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4143 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4144 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4145 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4146 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4147 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4148 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4149 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4150 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4151 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4152 2024-05-11T13:57:30.459 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4153 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4154 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4155 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4156 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4157 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4158 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4159 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4160 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4161 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4162 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4163 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4164 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4165 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4166 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4167 2024-05-11T13:57:30.460 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4168 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4169 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4170 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4171 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4172 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4173 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4174 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 5002 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 548 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 549 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6102 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6104 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6106 2024-05-11T13:57:30.461 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6110 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6111 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6112 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6113 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6117 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 826 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 827 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 828 2024-05-11T13:57:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 11 13:35 pg_internal.init 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 11 13:34 PG_VERSION 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:41 . 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 546485813 May 11 13:40 .. 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/global: 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:total 561 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 11 13:35 . 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1213 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1213_fsm 2024-05-11T13:57:30.812 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1213_vm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1214 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1214_fsm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1214_vm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 1232 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 1233 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1260 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1260_fsm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1260_vm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1261 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1261_fsm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1261_vm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1262 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 1262_fsm 2024-05-11T13:57:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 1262_vm 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2396 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 11 13:34 2396_fsm 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2396_vm 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2397 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2671 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2672 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2676 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2677 2024-05-11T13:57:30.814 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2694 2024-05-11T13:57:30.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2695 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2697 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 2698 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2846 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2847 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2964 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2965 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 2966 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 2967 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 3592 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 3593 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4060 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4061 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4175 2024-05-11T13:57:30.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4176 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4177 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4178 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4181 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4182 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4183 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4184 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 4185 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 4186 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6000 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6001 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6002 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 6100 2024-05-11T13:57:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6114 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 6115 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:57 pg_control 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 11 13:34 pg_filenode.map 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 11 13:35 pg_internal.init 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/log: 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:total 3 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1643 May 11 13:35 . 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1643 May 11 13:57 postgresql-Sat.log 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-11T13:57:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:total 3 2024-05-11T13:57:31.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:30 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:57 . 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 mappings 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:57 replorigin_checkpoint 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 snapshots 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-11T13:57:31.169 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:57 .. 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 11 13:57 .. 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:total 2 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 . 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 members 2024-05-11T13:57:31.170 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 offsets 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:total 9 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 . 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 .. 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 0000 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:total 9 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:34 . 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 11 13:34 .. 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 11 13:40 0000 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.171 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.172 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:total 16 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 11 13:57 . 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 11 13:57 db_0.stat 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 11 13:57 db_13494.stat 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 11 13:57 global.stat 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:57 . 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-11T13:57:31.174 INFO:teuthology.orchestra.run.smithi063.stdout:total 81 2024-05-11T13:57:31.174 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:57 . 2024-05-11T13:57:31.174 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.174 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:30 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:31.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:30 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:31.474 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 11 13:57 0003 2024-05-11T13:57:31.474 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.474 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-11T13:57:31.474 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.474 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.474 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:total 655362 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 671088640 May 11 13:57 . 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:57 00000001000000000000005D 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 00000001000000000000005E 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 00000001000000000000005F 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:45 000000010000000000000060 2024-05-11T13:57:31.475 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:48 000000010000000000000061 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:49 000000010000000000000062 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 000000010000000000000063 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 000000010000000000000064 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:48 000000010000000000000065 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 000000010000000000000066 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:48 000000010000000000000067 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:46 000000010000000000000068 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:45 000000010000000000000069 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:49 00000001000000000000006A 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 00000001000000000000006B 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 00000001000000000000006C 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:48 00000001000000000000006D 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:47 00000001000000000000006E 2024-05-11T13:57:31.476 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:45 00000001000000000000006F 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 000000010000000000000070 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:53 000000010000000000000071 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:52 000000010000000000000072 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:54 000000010000000000000073 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:55 000000010000000000000074 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 000000010000000000000075 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:52 000000010000000000000076 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:52 000000010000000000000077 2024-05-11T13:57:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 000000010000000000000078 2024-05-11T13:57:31.514 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:50 000000010000000000000079 2024-05-11T13:57:31.514 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:54 00000001000000000000007A 2024-05-11T13:57:31.514 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 00000001000000000000007B 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:54 00000001000000000000007C 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 00000001000000000000007D 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 00000001000000000000007E 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 00000001000000000000007F 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:51 000000010000000000000080 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:52 000000010000000000000081 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:50 000000010000000000000082 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:50 000000010000000000000083 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 11 13:50 000000010000000000000084 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 archive_status 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:total 1 2024-05-11T13:57:31.515 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 11 13:34 . 2024-05-11T13:57:31.516 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 671088640 May 11 13:57 .. 2024-05-11T13:57:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:31.516 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-11T13:57:31.516 INFO:teuthology.orchestra.run.smithi063.stdout:total 57 2024-05-11T13:57:31.516 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:34 . 2024-05-11T13:57:31.516 INFO:teuthology.orchestra.run.smithi063.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1218352633 May 11 13:57 .. 2024-05-11T13:57:31.516 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 11 13:57 0000 2024-05-11T13:57:31.524 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-11T13:57:31.535 INFO:tasks.quiescer:joining Quiescers 2024-05-11T13:57:31.535 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-11T13:57:31.536 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:57:31.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:31 smithi063 ceph-mon[29809]: pgmap v748: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-05-11T13:57:32.031 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:57:32.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:31 smithi165 ceph-mon[33235]: pgmap v748: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-05-11T13:57:32.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:31 smithi071 ceph-mon[33293]: pgmap v748: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-05-11T13:57:34.028 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:33 smithi063 ceph-mon[29809]: pgmap v749: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-05-11T13:57:34.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:33 smithi165 ceph-mon[33235]: pgmap v749: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-05-11T13:57:34.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:33 smithi071 ceph-mon[33293]: pgmap v749: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-05-11T13:57:34.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 -- 172.21.15.63:0/851160665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e8108900 msgr2=0x7fa8e8110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 --2- 172.21.15.63:0/851160665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e8108900 0x7fa8e8110480 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000b0d0 tx=0x7fa8e000b3e0 comp rx=0 tx=0).stop 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 -- 172.21.15.63:0/851160665 shutdown_connections 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 --2- 172.21.15.63:0/851160665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e8108900 0x7fa8e8110480 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 --2- 172.21.15.63:0/851160665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e8103800 0x7fa8e8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 --2- 172.21.15.63:0/851160665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e81041b0 0x7fa8e81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 -- 172.21.15.63:0/851160665 >> 172.21.15.63:0/851160665 conn(0x7fa8e807a6b0 msgr2=0x7fa8e807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 -- 172.21.15.63:0/851160665 shutdown_connections 2024-05-11T13:57:34.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.764+0000 7fa8eda00700 1 -- 172.21.15.63:0/851160665 wait complete. 2024-05-11T13:57:34.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.765+0000 7fa8eda00700 1 Processor -- start 2024-05-11T13:57:34.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.765+0000 7fa8eda00700 1 -- start start 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8eda00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e8103800 0x7fa8e81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8eda00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e81041b0 0x7fa8e81a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8eda00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e8108900 0x7fa8e819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8eda00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81135a0 con 0x7fa8e81041b0 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8eda00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8113420 con 0x7fa8e8108900 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8eda00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8113720 con 0x7fa8e8103800 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8e6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e81041b0 0x7fa8e81a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8e6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e81041b0 0x7fa8e81a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58700/0 (socket says 172.21.15.63:58700) 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8e6000700 1 -- 172.21.15.63:0/1538920803 learned_addr learned my addr 172.21.15.63:0/1538920803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.766+0000 7fa8e6a00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e8103800 0x7fa8e81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:34.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8e7400700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e8108900 0x7fa8e819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:34.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8e6000700 1 -- 172.21.15.63:0/1538920803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e8103800 msgr2=0x7fa8e81a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:34.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8e6000700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e8103800 0x7fa8e81a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:34.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8e6000700 1 -- 172.21.15.63:0/1538920803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e8108900 msgr2=0x7fa8e819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:34.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8e6000700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e8108900 0x7fa8e819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:34.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8e6000700 1 -- 172.21.15.63:0/1538920803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e819c220 con 0x7fa8e81041b0 2024-05-11T13:57:34.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8e6000700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e81041b0 0x7fa8e81a1df0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400d950 tx=0x7fa8d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:34.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.767+0000 7fa8df400700 1 -- 172.21.15.63:0/1538920803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4016070 con 0x7fa8e81041b0 2024-05-11T13:57:34.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.768+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e819c510 con 0x7fa8e81041b0 2024-05-11T13:57:34.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.768+0000 7fa8df400700 1 -- 172.21.15.63:0/1538920803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8d4009dd0 con 0x7fa8e81041b0 2024-05-11T13:57:34.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.768+0000 7fa8df400700 1 -- 172.21.15.63:0/1538920803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400f3c0 con 0x7fa8e81041b0 2024-05-11T13:57:34.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.768+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e81a8780 con 0x7fa8e81041b0 2024-05-11T13:57:34.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.769+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e806a240 con 0x7fa8e81041b0 2024-05-11T13:57:34.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.773+0000 7fa8df400700 1 -- 172.21.15.63:0/1538920803 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa8d4019070 con 0x7fa8e81041b0 2024-05-11T13:57:34.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.775+0000 7fa8df400700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8cc0b29f0 0x7fa8cc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:34.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.776+0000 7fa8df400700 1 -- 172.21.15.63:0/1538920803 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fa8d40e26a0 con 0x7fa8e81041b0 2024-05-11T13:57:34.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.776+0000 7fa8e6a00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8cc0b29f0 0x7fa8cc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:34.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.776+0000 7fa8e6a00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8cc0b29f0 0x7fa8cc0b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005950 tx=0x7fa8d800a300 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:34.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:34.778+0000 7fa8df400700 1 -- 172.21.15.63:0/1538920803 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8d40a9c30 con 0x7fa8e81041b0 2024-05-11T13:57:35.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.070+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8e81a89b0 con 0x7fa8e81041b0 2024-05-11T13:57:35.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.071+0000 7fa8df400700 1 -- 172.21.15.63:0/1538920803 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+11942 (secure 0 0 0) 0x7fa8d40a9530 con 0x7fa8e81041b0 2024-05-11T13:57:35.060 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:35.060 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":81,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:57:29.398796+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34451,34427,24472,34403]},"id":1}]} 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.074+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8cc0b29f0 msgr2=0x7fa8cc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.074+0000 7fa8eda00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8cc0b29f0 0x7fa8cc0b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005950 tx=0x7fa8d800a300 comp rx=0 tx=0).stop 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.074+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e81041b0 msgr2=0x7fa8e81a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.074+0000 7fa8eda00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e81041b0 0x7fa8e81a1df0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400d950 tx=0x7fa8d400dc60 comp rx=0 tx=0).stop 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 shutdown_connections 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa8cc0b29f0 0x7fa8cc0b4eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e8108900 0x7fa8e819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e8103800 0x7fa8e81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 --2- 172.21.15.63:0/1538920803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e81041b0 0x7fa8e81a1df0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 >> 172.21.15.63:0/1538920803 conn(0x7fa8e807a6b0 msgr2=0x7fa8e8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 shutdown_connections 2024-05-11T13:57:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:35.075+0000 7fa8eda00700 1 -- 172.21.15.63:0/1538920803 wait complete. 2024-05-11T13:57:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 81 2024-05-11T13:57:36.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:35 smithi165 ceph-mon[33235]: pgmap v750: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-05-11T13:57:36.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:35 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1538920803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:57:36.055 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:35 smithi063 ceph-mon[29809]: pgmap v750: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-05-11T13:57:36.056 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:35 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1538920803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:57:36.134 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34472 quiesce db --cancel --all' 2024-05-11T13:57:36.134 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.34472 quiesce db --cancel --all 2024-05-11T13:57:36.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:35 smithi071 ceph-mon[33293]: pgmap v750: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-05-11T13:57:36.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:35 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1538920803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:57:36.609 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:57:38.039 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:37 smithi063 ceph-mon[29809]: pgmap v751: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 542 KiB/s wr, 63 op/s 2024-05-11T13:57:38.040 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:37 smithi063 ceph-mon[29809]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:38.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:37 smithi165 ceph-mon[33235]: pgmap v751: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 542 KiB/s wr, 63 op/s 2024-05-11T13:57:38.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:37 smithi165 ceph-mon[33235]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:38.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:37 smithi071 ceph-mon[33293]: pgmap v751: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 542 KiB/s wr, 63 op/s 2024-05-11T13:57:38.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:37 smithi071 ceph-mon[33293]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-11T13:57:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 -- 172.21.15.63:0/3617102170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 msgr2=0x7fa98c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 --2- 172.21.15.63:0/3617102170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c1099a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009990 tx=0x7fa97c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 -- 172.21.15.63:0/3617102170 shutdown_connections 2024-05-11T13:57:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 --2- 172.21.15.63:0/3617102170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c06bbf0 0x7fa98c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 --2- 172.21.15.63:0/3617102170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c1099a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 --2- 172.21.15.63:0/3617102170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa98c06b230 0x7fa98c06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 -- 172.21.15.63:0/3617102170 >> 172.21.15.63:0/3617102170 conn(0x7fa98c079400 msgr2=0x7fa98c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 -- 172.21.15.63:0/3617102170 shutdown_connections 2024-05-11T13:57:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.109+0000 7fa991c00700 1 -- 172.21.15.63:0/3617102170 wait complete. 2024-05-11T13:57:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.110+0000 7fa991c00700 1 Processor -- start 2024-05-11T13:57:39.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.111+0000 7fa991c00700 1 -- start start 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.111+0000 7fa991c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c06b230 0x7fa98c102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.111+0000 7fa991c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa98c06bbf0 0x7fa98c102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.111+0000 7fa991c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.111+0000 7fa991c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1135f0 con 0x7fa98c1095c0 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.111+0000 7fa991c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c113470 con 0x7fa98c06b230 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.111+0000 7fa991c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c113770 con 0x7fa98c06bbf0 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c102fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58726/0 (socket says 172.21.15.63:58726) 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 -- 172.21.15.63:0/89275016 learned_addr learned my addr 172.21.15.63:0/89275016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98a000700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa98c06bbf0 0x7fa98c102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98aa00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c06b230 0x7fa98c102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 -- 172.21.15.63:0/89275016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa98c06bbf0 msgr2=0x7fa98c102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa98c06bbf0 0x7fa98c102a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 -- 172.21.15.63:0/89275016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c06b230 msgr2=0x7fa98c102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c06b230 0x7fa98c102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98b400700 1 -- 172.21.15.63:0/89275016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c1a80d0 con 0x7fa98c1095c0 2024-05-11T13:57:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.112+0000 7fa98aa00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c06b230 0x7fa98c102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:57:39.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.113+0000 7fa98b400700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c102fa0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa984007ee0 tx=0x7fa98400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.113+0000 7fa983400700 1 -- 172.21.15.63:0/89275016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98400dbd0 con 0x7fa98c1095c0 2024-05-11T13:57:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.113+0000 7fa983400700 1 -- 172.21.15.63:0/89275016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa98400dd70 con 0x7fa98c1095c0 2024-05-11T13:57:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.113+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c1a83c0 con 0x7fa98c1095c0 2024-05-11T13:57:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.113+0000 7fa983400700 1 -- 172.21.15.63:0/89275016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9840058d0 con 0x7fa98c1095c0 2024-05-11T13:57:39.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.113+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c1a88b0 con 0x7fa98c1095c0 2024-05-11T13:57:39.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.117+0000 7fa983400700 1 -- 172.21.15.63:0/89275016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa98401a020 con 0x7fa98c1095c0 2024-05-11T13:57:39.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.118+0000 7fa983400700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa9700b29f0 0x7fa9700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.118+0000 7fa983400700 1 -- 172.21.15.63:0/89275016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fa9840e2290 con 0x7fa98c1095c0 2024-05-11T13:57:39.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.118+0000 7fa98aa00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa9700b29f0 0x7fa9700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.119+0000 7fa98aa00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa9700b29f0 0x7fa9700b4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b5c0 tx=0x7fa97c01a040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:39.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.226+0000 7fa99c684b80 1 Processor -- start 2024-05-11T13:57:39.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.226+0000 7fa99c684b80 1 -- start start 2024-05-11T13:57:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.227+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b59055320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.227+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b5904f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.227+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904fed0 0x556b5905bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.227+0000 7fa99c684b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b58fa1a90 con 0x556b59054f40 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.227+0000 7fa99c684b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556b58f3f7d0 con 0x556b5904f510 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.227+0000 7fa99c684b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b58f9da50 con 0x556b5904fed0 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.227+0000 7fa98aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b59055320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b59055320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58728/0 (socket says 172.21.15.63:58728) 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98aa00700 1 -- 172.21.15.63:0/1944555566 learned_addr learned my addr 172.21.15.63:0/1944555566 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98a000700 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b5904f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98b400700 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904fed0 0x556b5905bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98aa00700 1 -- 172.21.15.63:0/1944555566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904fed0 msgr2=0x556b5905bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98aa00700 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904fed0 0x556b5905bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98aa00700 1 -- 172.21.15.63:0/1944555566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 msgr2=0x556b5904f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98aa00700 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b5904f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.228+0000 7fa98aa00700 1 -- 172.21.15.63:0/1944555566 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b58eee940 con 0x556b59054f40 2024-05-11T13:57:39.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.229+0000 7fa98aa00700 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b59055320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009990 tx=0x7fa97c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba3a1c179159418 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.229+0000 7fa977e00700 1 -- 172.21.15.63:0/1944555566 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00f840 con 0x556b59054f40 2024-05-11T13:57:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.229+0000 7fa977e00700 1 -- 172.21.15.63:0/1944555566 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa97c032070 con 0x556b59054f40 2024-05-11T13:57:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.230+0000 7fa977e00700 1 -- 172.21.15.63:0/1944555566 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c023ad0 con 0x556b59054f40 2024-05-11T13:57:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 -- 172.21.15.63:0/1944555566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 msgr2=0x556b59055320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b59055320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009990 tx=0x7fa97c01e000 comp rx=0 tx=0).stop 2024-05-11T13:57:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 -- 172.21.15.63:0/1944555566 shutdown_connections 2024-05-11T13:57:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904fed0 0x556b5905bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b59055320 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 --2- 172.21.15.63:0/1944555566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b5904f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 -- 172.21.15.63:0/1944555566 >> 172.21.15.63:0/1944555566 conn(0x556b58f7dbd0 msgr2=0x556b59049e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 -- 172.21.15.63:0/1944555566 shutdown_connections 2024-05-11T13:57:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.231+0000 7fa99c684b80 1 -- 172.21.15.63:0/1944555566 wait complete. 2024-05-11T13:57:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.232+0000 7fa99c684b80 1 -- start start 2024-05-11T13:57:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.233+0000 7fa99c684b80 1 Processor -- start 2024-05-11T13:57:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.233+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b5905b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.233+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59058c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.233+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b59054f40 0x556b590591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.233+0000 7fa99c684b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b5905c2a0 con 0x556b5904fed0 2024-05-11T13:57:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.233+0000 7fa99c684b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556b58f07b90 con 0x556b59054f40 2024-05-11T13:57:39.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.233+0000 7fa99c684b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b5905c420 con 0x556b5904f510 2024-05-11T13:57:39.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.234+0000 7fa99c684b80 10 client.0 _ll_register_callbacks cb 0x556b58f48ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:57:39.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.234+0000 7fa98a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59058c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.234+0000 7fa98a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59058c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58736/0 (socket says 172.21.15.63:58736) 2024-05-11T13:57:39.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.234+0000 7fa98a000700 1 -- 172.21.15.63:0/848472270 learned_addr learned my addr 172.21.15.63:0/848472270 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:39.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.234+0000 7fa98b400700 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b59054f40 0x556b590591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.234+0000 7fa98aa00700 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b5905b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa98a000700 1 -- 172.21.15.63:0/848472270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 msgr2=0x556b5905b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa98a000700 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b5905b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa98a000700 1 -- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b59054f40 msgr2=0x556b590591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa98a000700 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b59054f40 0x556b590591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa98a000700 1 -- 172.21.15.63:0/848472270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b58f67490 con 0x556b5904fed0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa98a000700 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59058c90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa97800bef0 tx=0x7fa978012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858dbcc6a294796a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa974c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.235+0000 7fa974c00700 1 -- 172.21.15.63:0/848472270 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9780107e0 con 0x556b5904fed0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.236+0000 7fa974c00700 1 -- 172.21.15.63:0/848472270 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fa978010980 con 0x556b5904fed0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.236+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa97c032070 con 0x556b5904fed0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.236+0000 7fa974c00700 1 -- 172.21.15.63:0/848472270 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa978010b80 con 0x556b5904fed0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.236+0000 7fa99c684b80 10 client.16299 fetch_fsmap learned FSMap version 82 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.236+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x556b59066b00 con 0x556b5904fed0 2024-05-11T13:57:39.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.236+0000 7fa974c00700 1 -- 172.21.15.63:0/848472270 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa97c032070 con 0x556b5904fed0 2024-05-11T13:57:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.237+0000 7fa974c00700 1 -- 172.21.15.63:0/848472270 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 82) ==== 8382+0+0 (secure 0 0 0) 0x7fa978019070 con 0x556b5904fed0 2024-05-11T13:57:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.237+0000 7fa99c684b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 82 2024-05-11T13:57:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.237+0000 7fa99c684b80 10 client.16299 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:57:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.237+0000 7fa99c684b80 20 client.16299 populate_metadata read hostname 'smithi063' 2024-05-11T13:57:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.237+0000 7fa99c684b80 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590691e0 0x556b5906b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.237+0000 7fa99c684b80 4 client.16299 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T13:57:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.237+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b5901c220 con 0x556b590691e0 2024-05-11T13:57:39.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.238+0000 7fa98aa00700 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590691e0 0x556b5906b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.239+0000 7fa98aa00700 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590691e0 0x556b5906b5d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4f690f5bb915c3 server_cookie=893ab9cd380ff490 in_seq=0 out_seq=0 2024-05-11T13:57:39.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.239+0000 7fa974c00700 10 client.16299 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:57:39.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.240+0000 7fa974c00700 1 -- 172.21.15.63:0/848472270 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b5901c220 con 0x556b590691e0 2024-05-11T13:57:39.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.240+0000 7fa974c00700 10 client.16299 handle_command_reply: tid=0 2024-05-11T13:57:39.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 client.16299 shutdown 2024-05-11T13:57:39.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 msgr2=0x556b59058c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59058c90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa97800bef0 tx=0x7fa978012730 comp rx=0 tx=0).stop 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 shutdown_connections 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590691e0 0x556b5906b5d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b59054f40 0x556b590591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b5905b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 --2- 172.21.15.63:0/848472270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59058c90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 >> 172.21.15.63:0/848472270 conn(0x556b58f7dbd0 msgr2=0x556b590448d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa974c00700 0 client.16299 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 shutdown_connections 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 1 -- 172.21.15.63:0/848472270 wait complete. 2024-05-11T13:57:39.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.241+0000 7fa99c684b80 20 client.16299 trim_cache size 0 max 16384 2024-05-11T13:57:39.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.264+0000 7fa99c684b80 1 Processor -- start 2024-05-11T13:57:39.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa99c684b80 1 -- start start 2024-05-11T13:57:39.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b590652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59044040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b59054f40 0x556b59044580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa99c684b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1b1840 con 0x556b5904fed0 2024-05-11T13:57:39.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa99c684b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1b3470 con 0x556b5904f510 2024-05-11T13:57:39.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa99c684b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1b1f10 con 0x556b59054f40 2024-05-11T13:57:39.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa98a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59044040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa98a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59044040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58752/0 (socket says 172.21.15.63:58752) 2024-05-11T13:57:39.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.265+0000 7fa98a000700 1 -- 172.21.15.63:0/935638407 learned_addr learned my addr 172.21.15.63:0/935638407 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98aa00700 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b590652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98b400700 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b59054f40 0x556b59044580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98a000700 1 -- 172.21.15.63:0/935638407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b59054f40 msgr2=0x556b59044580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98a000700 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b59054f40 0x556b59044580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98a000700 1 -- 172.21.15.63:0/935638407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 msgr2=0x556b590652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98a000700 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b590652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98a000700 1 -- 172.21.15.63:0/935638407 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b59046a00 con 0x556b5904fed0 2024-05-11T13:57:39.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.266+0000 7fa98a000700 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59044040 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa9780056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a09f4471ee6f23 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:39.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.267+0000 7fa977400700 1 -- 172.21.15.63:0/935638407 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97802d020 con 0x556b5904fed0 2024-05-11T13:57:39.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.267+0000 7fa977400700 1 -- 172.21.15.63:0/935638407 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa97801b440 con 0x556b5904fed0 2024-05-11T13:57:39.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.267+0000 7fa977400700 1 -- 172.21.15.63:0/935638407 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97801a780 con 0x556b5904fed0 2024-05-11T13:57:39.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 -- 172.21.15.63:0/935638407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 msgr2=0x556b59044040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59044040 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa9780056e0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 -- 172.21.15.63:0/935638407 shutdown_connections 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b59054f40 0x556b59044580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904f510 0x556b590652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 --2- 172.21.15.63:0/935638407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b5904fed0 0x556b59044040 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 -- 172.21.15.63:0/935638407 >> 172.21.15.63:0/935638407 conn(0x556b58f7dbd0 msgr2=0x556b59053250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 -- 172.21.15.63:0/935638407 shutdown_connections 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 -- 172.21.15.63:0/935638407 wait complete. 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.268+0000 7fa99c684b80 1 -- start start 2024-05-11T13:57:39.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 1 Processor -- start 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b59061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904fed0 0x556b5905e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b5905eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1b2da0 con 0x556b59054f40 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1b26a0 con 0x556b5904fed0 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1af800 con 0x556b5904f510 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.269+0000 7fa99c684b80 10 client.0 _ll_register_callbacks cb 0x556b58f48ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b5905eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b5905eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58762/0 (socket says 172.21.15.63:58762) 2024-05-11T13:57:39.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 -- 172.21.15.63:0/3056361886 learned_addr learned my addr 172.21.15.63:0/3056361886 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98aa00700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b59061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98a000700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904fed0 0x556b5905e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 -- 172.21.15.63:0/3056361886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 msgr2=0x556b59061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b59061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 -- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904fed0 msgr2=0x556b5905e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904fed0 0x556b5905e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98b400700 1 -- 172.21.15.63:0/3056361886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5905f300 con 0x556b59054f40 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.270+0000 7fa98a000700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904fed0 0x556b5905e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa98b400700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b5905eb80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa9840a5150 tx=0x7fa984004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d064e58fe26c32 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa974c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa974c00700 1 -- 172.21.15.63:0/3056361886 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9840a8c30 con 0x556b59054f40 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa974c00700 1 -- 172.21.15.63:0/3056361886 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fa9840b1af0 con 0x556b59054f40 2024-05-11T13:57:39.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa97801b440 con 0x556b59054f40 2024-05-11T13:57:39.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa974c00700 1 -- 172.21.15.63:0/3056361886 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa9840a9820 con 0x556b59054f40 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa99c684b80 10 client.16305 fetch_fsmap learned FSMap version 82 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.271+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x556b590d4a70 con 0x556b59054f40 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa974c00700 1 -- 172.21.15.63:0/3056361886 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa97801b440 con 0x556b59054f40 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa974c00700 1 -- 172.21.15.63:0/3056361886 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 82) ==== 8382+0+0 (secure 0 0 0) 0x7fa9840dfce0 con 0x556b59054f40 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa99c684b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 82 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa99c684b80 10 client.16305 resolve_mds: validated gid 34472 aka daemon mds.b 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa99c684b80 20 client.16305 populate_metadata read hostname 'smithi063' 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa99c684b80 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590d5150 0x556b590d7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa99c684b80 4 client.16305 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.272+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 --> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x556b58faf470 con 0x556b590d5150 2024-05-11T13:57:39.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.273+0000 7fa98aa00700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590d5150 0x556b590d7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:39.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.274+0000 7fa98aa00700 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590d5150 0x556b590d7570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da7764d83f29d71 server_cookie=f35c0e5773f6b78e in_seq=0 out_seq=0 2024-05-11T13:57:39.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.274+0000 7fa974c00700 10 client.16305 ms_handle_connect on v2:172.21.15.71:6838/1525017608 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.274+0000 7fa974c00700 1 -- 172.21.15.63:0/3056361886 <== mds.0 v2:172.21.15.71:6838/1525017608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x556b58faf470 con 0x556b590d5150 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.274+0000 7fa974c00700 10 client.16305 handle_command_reply: tid=0 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.274+0000 7fa99c684b80 1 client.16305 shutdown 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 msgr2=0x556b5905eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b5905eb80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa9840a5150 tx=0x7fa984004d50 comp rx=0 tx=0).stop 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 shutdown_connections 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b59054f40 0x556b5905eb80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5904f510 0x556b59061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:6838/1525017608,v1:172.21.15.71:6839/1525017608] conn(0x556b590d5150 0x556b590d7570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 --2- 172.21.15.63:0/3056361886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b5904fed0 0x556b5905e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa974c00700 0 client.16305 ms_handle_reset on v2:172.21.15.71:6838/1525017608 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 >> 172.21.15.63:0/3056361886 conn(0x556b58f7dbd0 msgr2=0x556b5904e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 shutdown_connections 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 1 -- 172.21.15.63:0/3056361886 wait complete. 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.275+0000 7fa99c684b80 20 client.16305 trim_cache size 0 max 16384 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 82, 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "leader": 34472, 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "set_version": 68, 2024-05-11T13:57:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "sets": {} 2024-05-11T13:57:39.266 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:57:39.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa9700b29f0 msgr2=0x7fa9700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa9700b29f0 0x7fa9700b4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b5c0 tx=0x7fa97c01a040 comp rx=0 tx=0).stop 2024-05-11T13:57:39.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 msgr2=0x7fa98c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:39.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c102fa0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa984007ee0 tx=0x7fa98400d3b0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 shutdown_connections 2024-05-11T13:57:39.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fa9700b29f0 0x7fa9700b4eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c1095c0 0x7fa98c102fa0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c06b230 0x7fa98c102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 --2- 172.21.15.63:0/89275016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa98c06bbf0 0x7fa98c102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:39.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.276+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 >> 172.21.15.63:0/89275016 conn(0x7fa98c079400 msgr2=0x7fa98c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:39.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.277+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 shutdown_connections 2024-05-11T13:57:39.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:39.277+0000 7fa991c00700 1 -- 172.21.15.63:0/89275016 wait complete. 2024-05-11T13:57:40.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:39 smithi165 ceph-mon[33235]: pgmap v752: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 834 KiB/s wr, 74 op/s 2024-05-11T13:57:40.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:39 smithi063 ceph-mon[29809]: pgmap v752: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 834 KiB/s wr, 74 op/s 2024-05-11T13:57:40.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:39 smithi071 ceph-mon[33293]: pgmap v752: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 834 KiB/s wr, 74 op/s 2024-05-11T13:57:40.269 INFO:tasks.quiescer:done joining Quiescers 2024-05-11T13:57:40.269 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-11T13:57:40.280 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-11T13:57:40.280 INFO:teuthology.task.exec:Executing custom commands... 2024-05-11T13:57:40.280 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:57:40.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-11T13:57:40.395 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-11T13:57:40.505 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-11T13:57:40.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 -- 172.21.15.63:0/2635960049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd00109fa0 msgr2=0x7efd00111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 --2- 172.21.15.63:0/2635960049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd00109fa0 0x7efd00111b70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009ca0 tx=0x7efcf802a8b0 comp rx=0 tx=0).stop 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 -- 172.21.15.63:0/2635960049 shutdown_connections 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 --2- 172.21.15.63:0/2635960049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd00109fa0 0x7efd00111b70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 --2- 172.21.15.63:0/2635960049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd00105880 0x7efd001098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 --2- 172.21.15.63:0/2635960049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00104ed0 0x7efd001052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 -- 172.21.15.63:0/2635960049 >> 172.21.15.63:0/2635960049 conn(0x7efd00100730 msgr2=0x7efd00102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 -- 172.21.15.63:0/2635960049 shutdown_connections 2024-05-11T13:57:40.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.915+0000 7efd06800640 1 -- 172.21.15.63:0/2635960049 wait complete. 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.916+0000 7efd06800640 1 Processor -- start 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.916+0000 7efd06800640 1 -- start start 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efd06800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00104ed0 0x7efd0019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efd06800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd00109fa0 0x7efd0019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efd06800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0019fb10 0x7efd0019ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efd06800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd00114e00 con 0x7efd0019fb10 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efd06800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd00114c80 con 0x7efd00104ed0 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efd06800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd00114f80 con 0x7efd00109fa0 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efcffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0019fb10 0x7efd0019ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efcffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0019fb10 0x7efd0019ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44262/0 (socket says 172.21.15.63:44262) 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efcffe00640 1 -- 172.21.15.63:0/3232404314 learned_addr learned my addr 172.21.15.63:0/3232404314 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efcff400640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00104ed0 0x7efd0019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.917+0000 7efcfea00640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd00109fa0 0x7efd0019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.918+0000 7efcffe00640 1 -- 172.21.15.63:0/3232404314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd00109fa0 msgr2=0x7efd0019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.918+0000 7efcffe00640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd00109fa0 0x7efd0019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.918+0000 7efcffe00640 1 -- 172.21.15.63:0/3232404314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00104ed0 msgr2=0x7efd0019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.918+0000 7efcffe00640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00104ed0 0x7efd0019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.918+0000 7efcffe00640 1 -- 172.21.15.63:0/3232404314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd001a0600 con 0x7efd0019fb10 2024-05-11T13:57:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.918+0000 7efcffe00640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0019fb10 0x7efd0019ffc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efcf8030750 tx=0x7efcf8030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.918+0000 7efcf7e00640 1 -- 172.21.15.63:0/3232404314 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8004040 con 0x7efd0019fb10 2024-05-11T13:57:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.919+0000 7efcf7e00640 1 -- 172.21.15.63:0/3232404314 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcf8040070 con 0x7efd0019fb10 2024-05-11T13:57:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.919+0000 7efcf7e00640 1 -- 172.21.15.63:0/3232404314 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8045ae0 con 0x7efd0019fb10 2024-05-11T13:57:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.919+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd001a0890 con 0x7efd0019fb10 2024-05-11T13:57:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.919+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd001a80e0 con 0x7efd0019fb10 2024-05-11T13:57:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.920+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd000507f0 con 0x7efd0019fb10 2024-05-11T13:57:40.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.923+0000 7efcf7e00640 1 -- 172.21.15.63:0/3232404314 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7efcf80041e0 con 0x7efd0019fb10 2024-05-11T13:57:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.925+0000 7efcf7e00640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efce40b2a90 0x7efce40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:40.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.926+0000 7efcf7e00640 1 -- 172.21.15.63:0/3232404314 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7efcf8103ff0 con 0x7efd0019fb10 2024-05-11T13:57:40.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.926+0000 7efcff400640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efce40b2a90 0x7efce40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:40.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.927+0000 7efcff400640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efce40b2a90 0x7efce40b4f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005fd0 tx=0x7efcf000b590 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:40.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:40.932+0000 7efcf7e00640 1 -- 172.21.15.63:0/3232404314 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efcf80cb6b0 con 0x7efd0019fb10 2024-05-11T13:57:41.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.178+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efd00105880 con 0x7efce40b2a90 2024-05-11T13:57:41.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.197+0000 7efcf7e00640 1 -- 172.21.15.63:0/3232404314 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7efd00105880 con 0x7efce40b2a90 2024-05-11T13:57:41.185 INFO:teuthology.orchestra.run.smithi063.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T13:33:52", "first": "2024-05-11T13:34:00", "last": "2024-05-11T13:57:00", "last_pruned": "2024-05-11T13:57:00", "created_count": 24, "pruned_count": 18, "active": true} 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.199+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efce40b2a90 msgr2=0x7efce40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.199+0000 7efd06800640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efce40b2a90 0x7efce40b4f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005fd0 tx=0x7efcf000b590 comp rx=0 tx=0).stop 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.199+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0019fb10 msgr2=0x7efd0019ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.199+0000 7efd06800640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0019fb10 0x7efd0019ffc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efcf8030750 tx=0x7efcf8030d50 comp rx=0 tx=0).stop 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 shutdown_connections 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7efce40b2a90 0x7efce40b4f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0019fb10 0x7efd0019ffc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd00109fa0 0x7efd0019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 --2- 172.21.15.63:0/3232404314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00104ed0 0x7efd0019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 >> 172.21.15.63:0/3232404314 conn(0x7efd00100730 msgr2=0x7efd00101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:41.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 shutdown_connections 2024-05-11T13:57:41.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.200+0000 7efd06800640 1 -- 172.21.15.63:0/3232404314 wait complete. 2024-05-11T13:57:41.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-11T13:57:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 -- 172.21.15.63:0/1091373338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc109920 msgr2=0x7f53bc115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 --2- 172.21.15.63:0/1091373338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc109920 0x7f53bc115d60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009cd0 tx=0x7f53b402a8f0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 -- 172.21.15.63:0/1091373338 shutdown_connections 2024-05-11T13:57:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 --2- 172.21.15.63:0/1091373338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc109920 0x7f53bc115d60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 --2- 172.21.15.63:0/1091373338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53bc108fa0 0x7f53bc1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 --2- 172.21.15.63:0/1091373338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc10ef30 0x7f53bc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 -- 172.21.15.63:0/1091373338 >> 172.21.15.63:0/1091373338 conn(0x7f53bc078840 msgr2=0x7f53bc107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 -- 172.21.15.63:0/1091373338 shutdown_connections 2024-05-11T13:57:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.428+0000 7f53bbe00640 1 -- 172.21.15.63:0/1091373338 wait complete. 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.429+0000 7f53bbe00640 1 Processor -- start 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.429+0000 7f53bbe00640 1 -- start start 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53bbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc108fa0 0x7f53bc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53bbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53bc109920 0x7f53bc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc10ef30 0x7f53bc1a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53bbe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc119010 con 0x7f53bc108fa0 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53bbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc118e90 con 0x7f53bc10ef30 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53bbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc119190 con 0x7f53bc109920 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53baa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc108fa0 0x7f53bc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53baa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc108fa0 0x7f53bc19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44274/0 (socket says 172.21.15.63:44274) 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.430+0000 7f53baa00640 1 -- 172.21.15.63:0/345588039 learned_addr learned my addr 172.21.15.63:0/345588039 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53ba000640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53bc109920 0x7f53bc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53bb400640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc10ef30 0x7f53bc1a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53baa00640 1 -- 172.21.15.63:0/345588039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53bc109920 msgr2=0x7f53bc19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53baa00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53bc109920 0x7f53bc19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53baa00640 1 -- 172.21.15.63:0/345588039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc10ef30 msgr2=0x7f53bc1a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53baa00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc10ef30 0x7f53bc1a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53baa00640 1 -- 172.21.15.63:0/345588039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc1a45c0 con 0x7f53bc108fa0 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53bb400640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc10ef30 0x7f53bc1a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53ba000640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53bc109920 0x7f53bc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53baa00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc108fa0 0x7f53bc19f5d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00e9f0 tx=0x7f53ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.431+0000 7f53b3400640 1 -- 172.21.15.63:0/345588039 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00cd80 con 0x7f53bc108fa0 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.432+0000 7f53b3400640 1 -- 172.21.15.63:0/345588039 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53ac0108c0 con 0x7f53bc108fa0 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.432+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc1a48b0 con 0x7f53bc108fa0 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.432+0000 7f53b3400640 1 -- 172.21.15.63:0/345588039 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0187c0 con 0x7f53bc108fa0 2024-05-11T13:57:41.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.432+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1ac140 con 0x7f53bc108fa0 2024-05-11T13:57:41.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.433+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc051040 con 0x7f53bc108fa0 2024-05-11T13:57:41.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.435+0000 7f53b3400640 1 -- 172.21.15.63:0/345588039 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f53ac010430 con 0x7f53bc108fa0 2024-05-11T13:57:41.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.437+0000 7f53b3400640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f53a00b2950 0x7f53a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.437+0000 7f53b3400640 1 -- 172.21.15.63:0/345588039 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f53ac0e2ef0 con 0x7f53bc108fa0 2024-05-11T13:57:41.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.437+0000 7f53ba000640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f53a00b2950 0x7f53a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.438+0000 7f53ba000640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f53a00b2950 0x7f53a00b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f53bc10e750 tx=0x7f53a8005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:41.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.441+0000 7f53b3400640 1 -- 172.21.15.63:0/345588039 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53ac0aa480 con 0x7f53bc108fa0 2024-05-11T13:57:41.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.676+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 --> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f53bc062ea0 con 0x7f53a00b2950 2024-05-11T13:57:41.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.693+0000 7f53b3400640 1 -- 172.21.15.63:0/345588039 <== mgr.15279 v2:172.21.15.63:6800/3203389144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f53bc062ea0 con 0x7f53a00b2950 2024-05-11T13:57:41.682 INFO:teuthology.orchestra.run.smithi063.stdout:/ 1m 6m3h 2024-05-11T13:57:41.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f53a00b2950 msgr2=0x7f53a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f53a00b2950 0x7f53a00b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f53bc10e750 tx=0x7f53a8005f90 comp rx=0 tx=0).stop 2024-05-11T13:57:41.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc108fa0 msgr2=0x7f53bc19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc108fa0 0x7f53bc19f5d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00e9f0 tx=0x7f53ac00eeb0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 shutdown_connections 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f53a00b2950 0x7f53a00b4e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc10ef30 0x7f53bc1a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53bc109920 0x7f53bc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 --2- 172.21.15.63:0/345588039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53bc108fa0 0x7f53bc19f5d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.696+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 >> 172.21.15.63:0/345588039 conn(0x7f53bc078840 msgr2=0x7f53bc079aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.697+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 shutdown_connections 2024-05-11T13:57:41.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.697+0000 7f53bbe00640 1 -- 172.21.15.63:0/345588039 wait complete. 2024-05-11T13:57:41.705 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-11T13:57:41.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3192794192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 msgr2=0x7fd29c10ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3192794192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c10ec00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd28802a3f0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3192794192 shutdown_connections 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3192794192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd29c10f1d0 0x7fd29c1115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3192794192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c10ec00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3192794192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd29c101620 0x7fd29c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3192794192 >> 172.21.15.63:0/3192794192 conn(0x7fd29c0fd230 msgr2=0x7fd29c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.966+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3192794192 shutdown_connections 2024-05-11T13:57:41.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.967+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3192794192 wait complete. 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.967+0000 7fd2a1a00640 1 Processor -- start 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd2a1a00640 1 -- start start 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd2a1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd29c101620 0x7fd29c19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd2a1a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd2a1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd29c10f1d0 0x7fd29c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd2a1a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c114bf0 con 0x7fd29c101fd0 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd2a1a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c114a70 con 0x7fd29c101620 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd2a1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c114d70 con 0x7fd29c10f1d0 2024-05-11T13:57:41.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd29b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd29b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c19b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44284/0 (socket says 172.21.15.63:44284) 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.968+0000 7fd29b400640 1 -- 172.21.15.63:0/3992931297 learned_addr learned my addr 172.21.15.63:0/3992931297 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd29be00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd29c101620 0x7fd29c19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd2a1000640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd29c10f1d0 0x7fd29c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd29b400640 1 -- 172.21.15.63:0/3992931297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd29c10f1d0 msgr2=0x7fd29c19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd29b400640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd29c10f1d0 0x7fd29c19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd29b400640 1 -- 172.21.15.63:0/3992931297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd29c101620 msgr2=0x7fd29c19b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd29b400640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd29c101620 0x7fd29c19b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:41.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd29b400640 1 -- 172.21.15.63:0/3992931297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c1a0100 con 0x7fd29c101fd0 2024-05-11T13:57:41.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.969+0000 7fd29b400640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c19b650 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd28802a910 tx=0x7fd288003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:41.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.970+0000 7fd298c00640 1 -- 172.21.15.63:0/3992931297 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28803f070 con 0x7fd29c101fd0 2024-05-11T13:57:41.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.970+0000 7fd298c00640 1 -- 172.21.15.63:0/3992931297 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd28803bdc0 con 0x7fd29c101fd0 2024-05-11T13:57:41.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.970+0000 7fd298c00640 1 -- 172.21.15.63:0/3992931297 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288035030 con 0x7fd29c101fd0 2024-05-11T13:57:41.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.970+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29c1a0390 con 0x7fd29c101fd0 2024-05-11T13:57:41.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.970+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29c1a7c80 con 0x7fd29c101fd0 2024-05-11T13:57:41.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.971+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29c069bf0 con 0x7fd29c101fd0 2024-05-11T13:57:41.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.974+0000 7fd298c00640 1 -- 172.21.15.63:0/3992931297 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd288042070 con 0x7fd29c101fd0 2024-05-11T13:57:41.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.975+0000 7fd298c00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd2800b29e0 0x7fd2800b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:41.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.976+0000 7fd298c00640 1 -- 172.21.15.63:0/3992931297 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fd288103910 con 0x7fd29c101fd0 2024-05-11T13:57:41.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.976+0000 7fd29be00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd2800b29e0 0x7fd2800b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:41.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.977+0000 7fd29be00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd2800b29e0 0x7fd2800b4ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005e10 tx=0x7fd28c005da0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:41.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:41.977+0000 7fd298c00640 1 -- 172.21.15.63:0/3992931297 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2880caf30 con 0x7fd29c101fd0 2024-05-11T13:57:42.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:41 smithi165 ceph-mon[33235]: pgmap v753: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 19 op/s 2024-05-11T13:57:42.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:41 smithi165 ceph-mon[33235]: from='client.16311 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:57:42.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:41 smithi063 ceph-mon[29809]: pgmap v753: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 19 op/s 2024-05-11T13:57:42.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:41 smithi063 ceph-mon[29809]: from='client.16311 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:57:42.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.191+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd29c10a300 con 0x7fd29c101fd0 2024-05-11T13:57:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.194+0000 7fd298c00640 1 -- 172.21.15.63:0/3992931297 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+25322 (secure 0 0 0) 0x7fd2880ca830 con 0x7fd29c101fd0 2024-05-11T13:57:42.183 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:42.183 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":127,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:57:15.419454+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":43,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":43,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/1742389159":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/712822368":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/2407940590":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/1742389159":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/66647058":"2024-05-12T13:33:51.933837+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/531186954":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":30,"length":1},{"begin":43,"length":1}]},{"pool":4,"snaps":[{"begin":30,"length":1},{"begin":43,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:57:42.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd2800b29e0 msgr2=0x7fd2800b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd2800b29e0 0x7fd2800b4ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005e10 tx=0x7fd28c005da0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 msgr2=0x7fd29c19b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c19b650 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd28802a910 tx=0x7fd288003d60 comp rx=0 tx=0).stop 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 shutdown_connections 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fd2800b29e0 0x7fd2800b4ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd29c10f1d0 0x7fd29c19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29c101fd0 0x7fd29c19b650 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 --2- 172.21.15.63:0/3992931297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd29c101620 0x7fd29c19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.197+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 >> 172.21.15.63:0/3992931297 conn(0x7fd29c0fd230 msgr2=0x7fd29c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.198+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 shutdown_connections 2024-05-11T13:57:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.198+0000 7fd2a1a00640 1 -- 172.21.15.63:0/3992931297 wait complete. 2024-05-11T13:57:42.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:41 smithi071 ceph-mon[33293]: pgmap v753: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 19 op/s 2024-05-11T13:57:42.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:41 smithi071 ceph-mon[33293]: from='client.16311 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:57:42.206 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T13:21:55.534075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-11T13:28:29.069877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-11T13:28:33.385508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 43, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-11T13:28:47.644228+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 43, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-11T13:57:42.207 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 -- 172.21.15.63:0/2877852263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 msgr2=0x7f998810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 --2- 172.21.15.63:0/2877852263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998810f8e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9980009930 tx=0x7f998002a420 comp rx=0 tx=0).stop 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 -- 172.21.15.63:0/2877852263 shutdown_connections 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 --2- 172.21.15.63:0/2877852263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998810f8e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 --2- 172.21.15.63:0/2877852263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9988102a10 0x7f9988102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 --2- 172.21.15.63:0/2877852263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9988108a20 0x7f9988108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 -- 172.21.15.63:0/2877852263 >> 172.21.15.63:0/2877852263 conn(0x7f99880fe4e0 msgr2=0x7f9988100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.433+0000 7f998dc00640 1 -- 172.21.15.63:0/2877852263 shutdown_connections 2024-05-11T13:57:42.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.434+0000 7f998dc00640 1 -- 172.21.15.63:0/2877852263 wait complete. 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 Processor -- start 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 -- start start 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9988102a10 0x7f998819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9988108a20 0x7f998819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9988112bb0 con 0x7f99881033d0 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9988112a30 con 0x7f9988102a10 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f998dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9988112d30 con 0x7f9988108a20 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f9986000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f9986000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998819b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44300/0 (socket says 172.21.15.63:44300) 2024-05-11T13:57:42.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.435+0000 7f9986000640 1 -- 172.21.15.63:0/1890190557 learned_addr learned my addr 172.21.15.63:0/1890190557 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9987400640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9988108a20 0x7f998819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986a00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9988102a10 0x7f998819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986000640 1 -- 172.21.15.63:0/1890190557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9988108a20 msgr2=0x7f998819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986000640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9988108a20 0x7f998819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986000640 1 -- 172.21.15.63:0/1890190557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9988102a10 msgr2=0x7f998819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986000640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9988102a10 0x7f998819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986000640 1 -- 172.21.15.63:0/1890190557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99881a0360 con 0x7f99881033d0 2024-05-11T13:57:42.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986a00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9988102a10 0x7f998819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:57:42.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.436+0000 7f9986000640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998819b8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f997000ebc0 tx=0x7f997000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:42.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.437+0000 7f997f400640 1 -- 172.21.15.63:0/1890190557 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970010630 con 0x7f99881033d0 2024-05-11T13:57:42.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.437+0000 7f997f400640 1 -- 172.21.15.63:0/1890190557 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9970010c60 con 0x7f99881033d0 2024-05-11T13:57:42.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.437+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99881a0650 con 0x7f99881033d0 2024-05-11T13:57:42.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.437+0000 7f997f400640 1 -- 172.21.15.63:0/1890190557 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970018b00 con 0x7f99881033d0 2024-05-11T13:57:42.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.437+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881a7ee0 con 0x7f99881033d0 2024-05-11T13:57:42.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.438+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9988068ff0 con 0x7f99881033d0 2024-05-11T13:57:42.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.440+0000 7f997f400640 1 -- 172.21.15.63:0/1890190557 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9970018ca0 con 0x7f99881033d0 2024-05-11T13:57:42.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.442+0000 7f997f400640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f99680b2960 0x7f99680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.442+0000 7f997f400640 1 -- 172.21.15.63:0/1890190557 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f99700e3570 con 0x7f99881033d0 2024-05-11T13:57:42.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.442+0000 7f9986a00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f99680b2960 0x7f99680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.443+0000 7f9986a00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f99680b2960 0x7f99680b4e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9978005fd0 tx=0x7f9978005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:42.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.445+0000 7f997f400640 1 -- 172.21.15.63:0/1890190557 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99700aab00 con 0x7f99881033d0 2024-05-11T13:57:42.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.661+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99881a81d0 con 0x7f99881033d0 2024-05-11T13:57:42.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.662+0000 7f997f400640 1 -- 172.21.15.63:0/1890190557 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) ==== 93+0+10 (secure 0 0 0) 0x7f99700aa400 con 0x7f99881033d0 2024-05-11T13:57:42.650 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-11T13:57:42.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f99680b2960 msgr2=0x7f99680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f99680b2960 0x7f99680b4e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9978005fd0 tx=0x7f9978005850 comp rx=0 tx=0).stop 2024-05-11T13:57:42.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 msgr2=0x7f998819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998819b8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f997000ebc0 tx=0x7f997000c6a0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 shutdown_connections 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f99680b2960 0x7f99680b4e20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9988108a20 0x7f998819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99881033d0 0x7f998819b8b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 --2- 172.21.15.63:0/1890190557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9988102a10 0x7f998819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 >> 172.21.15.63:0/1890190557 conn(0x7f99880fe4e0 msgr2=0x7f99880feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 shutdown_connections 2024-05-11T13:57:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.665+0000 7f998dc00640 1 -- 172.21.15.63:0/1890190557 wait complete. 2024-05-11T13:57:42.672 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-11T13:57:42.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 -- 172.21.15.63:0/2409971738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 msgr2=0x7fe94810f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 --2- 172.21.15.63:0/2409971738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe94810f400 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009900 tx=0x7fe93c02a480 comp rx=0 tx=0).stop 2024-05-11T13:57:42.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 -- 172.21.15.63:0/2409971738 shutdown_connections 2024-05-11T13:57:42.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 --2- 172.21.15.63:0/2409971738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481099a0 0x7fe948115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 --2- 172.21.15.63:0/2409971738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe948109000 0x7fe948109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 --2- 172.21.15.63:0/2409971738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe94810f400 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 -- 172.21.15.63:0/2409971738 >> 172.21.15.63:0/2409971738 conn(0x7fe948078880 msgr2=0x7fe948107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:42.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 -- 172.21.15.63:0/2409971738 shutdown_connections 2024-05-11T13:57:42.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.898+0000 7fe94f000640 1 -- 172.21.15.63:0/2409971738 wait complete. 2024-05-11T13:57:42.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.899+0000 7fe94f000640 1 Processor -- start 2024-05-11T13:57:42.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.900+0000 7fe94f000640 1 -- start start 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.900+0000 7fe94f000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe948109000 0x7fe94819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.900+0000 7fe94f000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9481099a0 0x7fe94819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.900+0000 7fe94f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe9481a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.900+0000 7fe94f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe948119060 con 0x7fe94810f020 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.900+0000 7fe94f000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe948118ee0 con 0x7fe948109000 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.900+0000 7fe94f000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe9481191e0 con 0x7fe9481099a0 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe9481a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe9481a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44322/0 (socket says 172.21.15.63:44322) 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 -- 172.21.15.63:0/131433651 learned_addr learned my addr 172.21.15.63:0/131433651 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:42.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94dc00640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe948109000 0x7fe94819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94d200640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9481099a0 0x7fe94819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 -- 172.21.15.63:0/131433651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9481099a0 msgr2=0x7fe94819fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9481099a0 0x7fe94819fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 -- 172.21.15.63:0/131433651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe948109000 msgr2=0x7fe94819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:42.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe948109000 0x7fe94819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:42.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 -- 172.21.15.63:0/131433651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9481a45f0 con 0x7fe94810f020 2024-05-11T13:57:42.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94dc00640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe948109000 0x7fe94819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:57:42.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.901+0000 7fe94e600640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe9481a3ec0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe94000da40 tx=0x7fe94000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:42.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.902+0000 7fe946000640 1 -- 172.21.15.63:0/131433651 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940017070 con 0x7fe94810f020 2024-05-11T13:57:42.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.902+0000 7fe946000640 1 -- 172.21.15.63:0/131433651 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe940003e00 con 0x7fe94810f020 2024-05-11T13:57:42.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.902+0000 7fe946000640 1 -- 172.21.15.63:0/131433651 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940010460 con 0x7fe94810f020 2024-05-11T13:57:42.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.902+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9481a48e0 con 0x7fe94810f020 2024-05-11T13:57:42.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.902+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481ac170 con 0x7fe94810f020 2024-05-11T13:57:42.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.903+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe948069bf0 con 0x7fe94810f020 2024-05-11T13:57:42.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.906+0000 7fe946000640 1 -- 172.21.15.63:0/131433651 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe940004020 con 0x7fe94810f020 2024-05-11T13:57:42.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.908+0000 7fe946000640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe9300b29e0 0x7fe9300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:42.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.908+0000 7fe946000640 1 -- 172.21.15.63:0/131433651 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fe9400e2770 con 0x7fe94810f020 2024-05-11T13:57:42.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.908+0000 7fe94dc00640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe9300b29e0 0x7fe9300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:42.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.909+0000 7fe94dc00640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe9300b29e0 0x7fe9300b4ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe93c000c00 tx=0x7fe93c005a20 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:42.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:42.912+0000 7fe946000640 1 -- 172.21.15.63:0/131433651 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9400a9d00 con 0x7fe94810f020 2024-05-11T13:57:42.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:42 smithi063 ceph-mon[29809]: from='client.16317 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:57:42.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:42 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3992931297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:57:42.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:42 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1890190557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T13:57:43.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:42 smithi165 ceph-mon[33235]: from='client.16317 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:57:43.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3992931297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:57:43.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:42 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1890190557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T13:57:43.092 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:42 smithi071 ceph-mon[33293]: from='client.16317 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T13:57:43.093 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:42 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3992931297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:57:43.093 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:42 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1890190557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T13:57:43.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.126+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe948111440 con 0x7fe94810f020 2024-05-11T13:57:43.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.127+0000 7fe946000640 1 -- 172.21.15.63:0/131433651 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) ==== 104+0+11 (secure 0 0 0) 0x7fe9400a9600 con 0x7fe94810f020 2024-05-11T13:57:43.115 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-11T13:57:43.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.129+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe9300b29e0 msgr2=0x7fe9300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.129+0000 7fe94f000640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe9300b29e0 0x7fe9300b4ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe93c000c00 tx=0x7fe93c005a20 comp rx=0 tx=0).stop 2024-05-11T13:57:43.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.129+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 msgr2=0x7fe9481a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.129+0000 7fe94f000640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe9481a3ec0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe94000da40 tx=0x7fe94000df00 comp rx=0 tx=0).stop 2024-05-11T13:57:43.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 shutdown_connections 2024-05-11T13:57:43.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe9300b29e0 0x7fe9300b4ea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe94810f020 0x7fe9481a3ec0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9481099a0 0x7fe94819fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 --2- 172.21.15.63:0/131433651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe948109000 0x7fe94819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 >> 172.21.15.63:0/131433651 conn(0x7fe948078880 msgr2=0x7fe948079b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:43.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 shutdown_connections 2024-05-11T13:57:43.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.130+0000 7fe94f000640 1 -- 172.21.15.63:0/131433651 wait complete. 2024-05-11T13:57:43.138 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-11T13:57:43.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 -- 172.21.15.63:0/3791011801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 msgr2=0x7fe82010f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 --2- 172.21.15.63:0/3791011801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82010f780 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe818009ca0 tx=0x7fe81802a8c0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 -- 172.21.15.63:0/3791011801 shutdown_connections 2024-05-11T13:57:43.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 --2- 172.21.15.63:0/3791011801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82010f780 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 --2- 172.21.15.63:0/3791011801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe82006afc0 0x7fe82006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 --2- 172.21.15.63:0/3791011801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe820108930 0x7fe820108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 -- 172.21.15.63:0/3791011801 >> 172.21.15.63:0/3791011801 conn(0x7fe820078710 msgr2=0x7fe820078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:43.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 -- 172.21.15.63:0/3791011801 shutdown_connections 2024-05-11T13:57:43.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.369+0000 7fe81fe00640 1 -- 172.21.15.63:0/3791011801 wait complete. 2024-05-11T13:57:43.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.370+0000 7fe81fe00640 1 Processor -- start 2024-05-11T13:57:43.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.370+0000 7fe81fe00640 1 -- start start 2024-05-11T13:57:43.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe82006afc0 0x7fe82019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe820108930 0x7fe82019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe820112ad0 con 0x7fe82006b960 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe820112950 con 0x7fe82006afc0 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe820112c50 con 0x7fe820108930 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44334/0 (socket says 172.21.15.63:44334) 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81e000640 1 -- 172.21.15.63:0/633369401 learned_addr learned my addr 172.21.15.63:0/633369401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81ea00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe82006afc0 0x7fe82019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81f400640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe820108930 0x7fe82019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.371+0000 7fe81e000640 1 -- 172.21.15.63:0/633369401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe820108930 msgr2=0x7fe82019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe81e000640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe820108930 0x7fe82019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe81e000640 1 -- 172.21.15.63:0/633369401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe82006afc0 msgr2=0x7fe82019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe81e000640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe82006afc0 0x7fe82019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe81e000640 1 -- 172.21.15.63:0/633369401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8201a0200 con 0x7fe82006b960 2024-05-11T13:57:43.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe81e000640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82019b750 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00d770 tx=0x7fe80c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:43.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe817400640 1 -- 172.21.15.63:0/633369401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c017070 con 0x7fe82006b960 2024-05-11T13:57:43.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe817400640 1 -- 172.21.15.63:0/633369401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe80c0054b0 con 0x7fe82006b960 2024-05-11T13:57:43.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8201a04f0 con 0x7fe82006b960 2024-05-11T13:57:43.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe817400640 1 -- 172.21.15.63:0/633369401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c004c60 con 0x7fe82006b960 2024-05-11T13:57:43.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.372+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8201a7d80 con 0x7fe82006b960 2024-05-11T13:57:43.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.374+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe820069bf0 con 0x7fe82006b960 2024-05-11T13:57:43.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.376+0000 7fe817400640 1 -- 172.21.15.63:0/633369401 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe80c004e00 con 0x7fe82006b960 2024-05-11T13:57:43.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.377+0000 7fe817400640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe8040b2960 0x7fe8040b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.378+0000 7fe817400640 1 -- 172.21.15.63:0/633369401 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fe80c0e24c0 con 0x7fe82006b960 2024-05-11T13:57:43.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.378+0000 7fe81ea00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe8040b2960 0x7fe8040b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.379+0000 7fe81ea00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe8040b2960 0x7fe8040b4e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe81000bb60 tx=0x7fe81000b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:43.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.382+0000 7fe817400640 1 -- 172.21.15.63:0/633369401 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe80c0a9a50 con 0x7fe82006b960 2024-05-11T13:57:43.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.598+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8201a7f50 con 0x7fe82006b960 2024-05-11T13:57:43.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.599+0000 7fe817400640 1 -- 172.21.15.63:0/633369401 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) ==== 100+0+11 (secure 0 0 0) 0x7fe80c0a9350 con 0x7fe82006b960 2024-05-11T13:57:43.588 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 64 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe8040b2960 msgr2=0x7fe8040b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe8040b2960 0x7fe8040b4e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe81000bb60 tx=0x7fe81000b390 comp rx=0 tx=0).stop 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 msgr2=0x7fe82019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82019b750 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00d770 tx=0x7fe80c00dc30 comp rx=0 tx=0).stop 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 shutdown_connections 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fe8040b2960 0x7fe8040b4e20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe820108930 0x7fe82019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82006b960 0x7fe82019b750 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 --2- 172.21.15.63:0/633369401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe82006afc0 0x7fe82019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.602+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 >> 172.21.15.63:0/633369401 conn(0x7fe820078710 msgr2=0x7fe820100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.603+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 shutdown_connections 2024-05-11T13:57:43.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.603+0000 7fe81fe00640 1 -- 172.21.15.63:0/633369401 wait complete. 2024-05-11T13:57:43.611 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-11T13:57:43.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 -- 172.21.15.63:0/3506389211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c104b30 msgr2=0x7f712c104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 --2- 172.21.15.63:0/3506389211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c104b30 0x7f712c104fb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7118009900 tx=0x7f711802a410 comp rx=0 tx=0).stop 2024-05-11T13:57:43.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 -- 172.21.15.63:0/3506389211 shutdown_connections 2024-05-11T13:57:43.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 --2- 172.21.15.63:0/3506389211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1054f0 0x7f712c1119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 --2- 172.21.15.63:0/3506389211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c104b30 0x7f712c104fb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 --2- 172.21.15.63:0/3506389211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c10abf0 0x7f712c10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 -- 172.21.15.63:0/3506389211 >> 172.21.15.63:0/3506389211 conn(0x7f712c1005a0 msgr2=0x7f712c1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:43.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 -- 172.21.15.63:0/3506389211 shutdown_connections 2024-05-11T13:57:43.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.843+0000 7f7131200640 1 -- 172.21.15.63:0/3506389211 wait complete. 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 Processor -- start 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 -- start start 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c104b30 0x7f712c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1054f0 0x7f712c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c10abf0 0x7f712c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f712c114ce0 con 0x7f712c10abf0 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f712c114b60 con 0x7f712c1054f0 2024-05-11T13:57:43.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.844+0000 7f7131200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f712c114e60 con 0x7f712c104b30 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c10abf0 0x7f712c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c10abf0 0x7f712c19fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44342/0 (socket says 172.21.15.63:44342) 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 -- 172.21.15.63:0/2514212096 learned_addr learned my addr 172.21.15.63:0/2514212096 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712b400640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c104b30 0x7f712c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712aa00640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1054f0 0x7f712c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 -- 172.21.15.63:0/2514212096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c104b30 msgr2=0x7f712c19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c104b30 0x7f712c19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 -- 172.21.15.63:0/2514212096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1054f0 msgr2=0x7f712c19b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1054f0 0x7f712c19b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712be00640 1 -- 172.21.15.63:0/2514212096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c1a0200 con 0x7f712c10abf0 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.845+0000 7f712b400640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c104b30 0x7f712c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.846+0000 7f712be00640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c10abf0 0x7f712c19fad0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f712400b970 tx=0x7f712400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.846+0000 7f7123e00640 1 -- 172.21.15.63:0/2514212096 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124019070 con 0x7f712c10abf0 2024-05-11T13:57:43.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.846+0000 7f7123e00640 1 -- 172.21.15.63:0/2514212096 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7124014440 con 0x7f712c10abf0 2024-05-11T13:57:43.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.846+0000 7f7123e00640 1 -- 172.21.15.63:0/2514212096 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240075d0 con 0x7f712c10abf0 2024-05-11T13:57:43.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.846+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c1a04f0 con 0x7f712c10abf0 2024-05-11T13:57:43.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.846+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1a7d80 con 0x7f712c10abf0 2024-05-11T13:57:43.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.847+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c069bf0 con 0x7f712c10abf0 2024-05-11T13:57:43.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.849+0000 7f7123e00640 1 -- 172.21.15.63:0/2514212096 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f712401c070 con 0x7f712c10abf0 2024-05-11T13:57:43.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.852+0000 7f7123e00640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b29e0 0x7f71100b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:43.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.852+0000 7f7123e00640 1 -- 172.21.15.63:0/2514212096 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f71240e1f30 con 0x7f712c10abf0 2024-05-11T13:57:43.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.852+0000 7f712b400640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b29e0 0x7f71100b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:43.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.853+0000 7f712b400640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b29e0 0x7f71100b4ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f711c0058c0 tx=0x7f711c005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:43.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:43.857+0000 7f7123e00640 1 -- 172.21.15.63:0/2514212096 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71240a95d0 con 0x7f712c10abf0 2024-05-11T13:57:43.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:43 smithi063 ceph-mon[29809]: pgmap v754: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 18 op/s 2024-05-11T13:57:43.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:43 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/131433651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T13:57:43.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:43 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/633369401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T13:57:44.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:43 smithi165 ceph-mon[33235]: pgmap v754: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 18 op/s 2024-05-11T13:57:44.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:43 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/131433651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T13:57:44.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:43 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/633369401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T13:57:44.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.070+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f712c10cf60 con 0x7f712c10abf0 2024-05-11T13:57:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.071+0000 7f7123e00640 1 -- 172.21.15.63:0/2514212096 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v127) ==== 103+0+11 (secure 0 0 0) 0x7f71240a8ed0 con 0x7f712c10abf0 2024-05-11T13:57:44.059 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-11T13:57:44.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b29e0 msgr2=0x7f71100b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:44.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b29e0 0x7f71100b4ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f711c0058c0 tx=0x7f711c005850 comp rx=0 tx=0).stop 2024-05-11T13:57:44.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c10abf0 msgr2=0x7f712c19fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:44.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c10abf0 0x7f712c19fad0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f712400b970 tx=0x7f712400be30 comp rx=0 tx=0).stop 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 shutdown_connections 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f71100b29e0 0x7f71100b4ea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f712c10abf0 0x7f712c19fad0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f712c1054f0 0x7f712c19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 --2- 172.21.15.63:0/2514212096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c104b30 0x7f712c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 >> 172.21.15.63:0/2514212096 conn(0x7f712c1005a0 msgr2=0x7f712c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 shutdown_connections 2024-05-11T13:57:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.074+0000 7f7131200640 1 -- 172.21.15.63:0/2514212096 wait complete. 2024-05-11T13:57:44.083 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-11T13:57:44.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:43 smithi071 ceph-mon[33293]: pgmap v754: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 18 op/s 2024-05-11T13:57:44.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:43 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/131433651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T13:57:44.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:43 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/633369401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T13:57:44.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.309+0000 7f832b200640 1 -- 172.21.15.63:0/4249845760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 msgr2=0x7f832c077370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:44.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.309+0000 7f832b200640 1 --2- 172.21.15.63:0/4249845760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c077370 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f831c009900 tx=0x7f831c02a410 comp rx=0 tx=0).stop 2024-05-11T13:57:44.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.310+0000 7f832b200640 1 -- 172.21.15.63:0/4249845760 shutdown_connections 2024-05-11T13:57:44.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.310+0000 7f832b200640 1 --2- 172.21.15.63:0/4249845760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c113700 0x7f832c115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.310+0000 7f832b200640 1 --2- 172.21.15.63:0/4249845760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f832c0778b0 0x7f832c1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.310+0000 7f832b200640 1 --2- 172.21.15.63:0/4249845760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c077370 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.310+0000 7f832b200640 1 -- 172.21.15.63:0/4249845760 >> 172.21.15.63:0/4249845760 conn(0x7f832c1004f0 msgr2=0x7f832c102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:44.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.310+0000 7f832b200640 1 -- 172.21.15.63:0/4249845760 shutdown_connections 2024-05-11T13:57:44.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.310+0000 7f832b200640 1 -- 172.21.15.63:0/4249845760 wait complete. 2024-05-11T13:57:44.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 Processor -- start 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 -- start start 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0778b0 0x7f832c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f832c113700 0x7f832c1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f832c118fd0 con 0x7f832c076f90 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f832c118e50 con 0x7f832c0778b0 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.311+0000 7f832b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f832c119150 con 0x7f832c113700 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44360/0 (socket says 172.21.15.63:44360) 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 -- 172.21.15.63:0/1125881467 learned_addr learned my addr 172.21.15.63:0/1125881467 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f832a800640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f832c113700 0x7f832c1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:44.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329400640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0778b0 0x7f832c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:44.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 -- 172.21.15.63:0/1125881467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f832c113700 msgr2=0x7f832c1a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:44.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f832c113700 0x7f832c1a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 -- 172.21.15.63:0/1125881467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0778b0 msgr2=0x7f832c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:44.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0778b0 0x7f832c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:44.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.312+0000 7f8329e00640 1 -- 172.21.15.63:0/1125881467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c1a46d0 con 0x7f832c076f90 2024-05-11T13:57:44.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.313+0000 7f8329e00640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c19f6e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f831c006010 tx=0x7f831c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:44.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.313+0000 7f8322a00640 1 -- 172.21.15.63:0/1125881467 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c03f030 con 0x7f832c076f90 2024-05-11T13:57:44.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.313+0000 7f8322a00640 1 -- 172.21.15.63:0/1125881467 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f831c003c90 con 0x7f832c076f90 2024-05-11T13:57:44.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.313+0000 7f8322a00640 1 -- 172.21.15.63:0/1125881467 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c035070 con 0x7f832c076f90 2024-05-11T13:57:44.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.313+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c1a4960 con 0x7f832c076f90 2024-05-11T13:57:44.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.313+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c1ac250 con 0x7f832c076f90 2024-05-11T13:57:44.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.314+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832c069bf0 con 0x7f832c076f90 2024-05-11T13:57:44.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.317+0000 7f8322a00640 1 -- 172.21.15.63:0/1125881467 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f831c042030 con 0x7f832c076f90 2024-05-11T13:57:44.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.318+0000 7f8322a00640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83100b2960 0x7f83100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:57:44.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.318+0000 7f8322a00640 1 -- 172.21.15.63:0/1125881467 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f831c103440 con 0x7f832c076f90 2024-05-11T13:57:44.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.319+0000 7f8329400640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83100b2960 0x7f83100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:57:44.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.319+0000 7f8329400640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83100b2960 0x7f83100b4e20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8318005fd0 tx=0x7f8318005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:57:44.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.320+0000 7f8322a00640 1 -- 172.21.15.63:0/1125881467 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f831c0ca9d0 con 0x7f832c076f90 2024-05-11T13:57:44.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.581+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f832c10e890 con 0x7f832c076f90 2024-05-11T13:57:44.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.602+0000 7f8322a00640 1 -- 172.21.15.63:0/1125881467 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+816173 (secure 0 0 0) 0x7f831c0ca2d0 con 0x7f832c076f90 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 25, 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "active_gid": 15279, 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "active_name": "x", 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "active_addrs": { 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6800", 2024-05-11T13:57:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3203389144 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6801", 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3203389144 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "active_addr": "172.21.15.63:6801/3203389144", 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "active_change": "2024-05-11T13:33:51.934388+0000", 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "active_mgr_features": 4540701547738038271, 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "available": true, 2024-05-11T13:57:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "standbys": [ 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "gid": 15270, 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "y", 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_features": 4540701547738038271, 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "available_modules": [ 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "alerts", 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How frequently to reexamine health status", 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_destination": { 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_destination", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email address to send alerts to", 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_from_name": { 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_from_name", 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "Ceph", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email From: name", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_host": { 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_host", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP server", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_password": { 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_password", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Password to authenticate with", 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_port": { 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_port", 2024-05-11T13:57:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "465", 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP port", 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_sender": { 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_sender", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP envelope sender", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_ssl": { 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_ssl", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_user": { 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_user", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "User to authenticate as", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "balancer", 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "active": { 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "active", 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically balance PGs across cluster", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_time": { 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_time", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0000", 2024-05-11T13:57:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "beginning time of day to automatically balance", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_weekday": { 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_weekday", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-11T13:57:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_max_iterations": { 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_max_iterations", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "25", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "250", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_metrics": { 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_metrics", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "pgs,objects,bytes", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_step": { 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_step", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.5", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0.001", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "0.999", 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "aggressiveness of optimization", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "end_time": { 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_time", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2359", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ending time of day to automatically balance", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "end_weekday": { 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_weekday", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-11T13:57:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "min_score": { 2024-05-11T13:57:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_score", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-11T13:57:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "upmap", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-compat", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "none", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "read", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap-read" 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Balancer mode", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_ids": { 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_ids", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_deviation": { 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_deviation", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_optimizations": { 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_optimizations", 2024-05-11T13:57:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_down_multiplier": { 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_down_multiplier", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_refresh_rate": { 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_refresh_rate", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "20", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_starting_port": { 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_starting_port", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4721", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_ptrace": { 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_ptrace", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_interval": { 2024-05-11T13:57:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_interval", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_memory_target_ratio": { 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_memory_target_ratio", 2024-05-11T13:57:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.7", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm_log_destination": { 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm_log_destination", 2024-05-11T13:57:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "file", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "file,syslog", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "syslog" 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "cgroups_split": { 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cgroups_split", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "config_checks_enabled": { 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_checks_enabled", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "config_dashboard": { 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_dashboard", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_alertmanager": { 2024-05-11T13:57:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_alertmanager", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_base": { 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_base", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Container image name, without the tag", 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_elasticsearch": { 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_elasticsearch", 2024-05-11T13:57:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "elasticsearch container image", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_grafana": { 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_grafana", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_haproxy": { 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_haproxy", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "HAproxy container image", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_agent": { 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_agent", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger agent container image", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_collector": { 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_collector", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-11T13:57:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger collector container image", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_query": { 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_query", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-11T13:57:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger query container image", 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_keepalived": { 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_keepalived", 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Keepalived container image", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_loki": { 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_loki", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Loki container image", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_node_exporter": { 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_node_exporter", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_nvmeof": { 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_nvmeof", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Nvme-of container image", 2024-05-11T13:57:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_prometheus": { 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_prometheus", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_promtail": { 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_promtail", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Promtail container image", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_samba": { 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_samba", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Samba/SMB container image", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_snmp_gateway": { 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_snmp_gateway", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SNMP Gateway container image", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "container_init": { 2024-05-11T13:57:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_init", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Run podman/docker with `--init`", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "daemon_cache_timeout": { 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "daemon_cache_timeout", 2024-05-11T13:57:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "default_cephadm_command_timeout": { 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_cephadm_command_timeout", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "900", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "default_registry": { 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_registry", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "device_cache_timeout": { 2024-05-11T13:57:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_cache_timeout", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1800", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache device inventory", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "device_enhanced_scan": { 2024-05-11T13:57:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_enhanced_scan", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "facts_cache_timeout": { 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "facts_cache_timeout", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache host facts data", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "grafana_dashboards_path": { 2024-05-11T13:57:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "grafana_dashboards_path", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "host_check_interval": { 2024-05-11T13:57:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "host_check_interval", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to perform a host check", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "hw_monitoring": { 2024-05-11T13:57:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hw_monitoring", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "inventory_list_all": { 2024-05-11T13:57:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "inventory_list_all", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "log_refresh_metadata": { 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_refresh_metadata", 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-11T13:57:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "*", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "max_count_per_host": { 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_count_per_host", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of daemons per service per host", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd_draining_count": { 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_osd_draining_count", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "migration_current": { 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "migration_current", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "internal - do not modify", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "root", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm-package", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "root" 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "mode for remote execution of cephadm", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "oob_default_addr": { 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "oob_default_addr", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "169.254.1.1", 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "prometheus_alerts_path": { 2024-05-11T13:57:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus_alerts_path", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_insecure": { 2024-05-11T13:57:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_insecure", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_password": { 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_password", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_url": { 2024-05-11T13:57:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_url", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_username": { 2024-05-11T13:57:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_username", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "secure_monitoring_stack": { 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "secure_monitoring_stack", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "service_discovery_port": { 2024-05-11T13:57:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "service_discovery_port", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8765", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "cephadm service discovery port", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "ssh_config_file": { 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssh_config_file", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "use_agent": { 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_agent", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "use_repo_digest": { 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_repo_digest", 2024-05-11T13:57:44.699 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_failed_host_check": { 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_failed_host_check", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.700 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if the host check fails", 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_daemons": { 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_daemons", 2024-05-11T13:57:44.701 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_hosts": { 2024-05-11T13:57:44.702 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_hosts", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crash", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.704 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.705 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.706 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.707 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.708 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "retain_interval": { 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retain_interval", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "31536000", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.709 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.710 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_recent_interval": { 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_recent_interval", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1209600", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.711 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dashboard", 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.712 INFO:teuthology.orchestra.run.smithi063.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.713 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_HOST": { 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.714 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.715 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_ENABLED": { 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_ENABLED", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.716 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.717 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "ENABLE_BROWSABLE_API": { 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.718 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.719 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.720 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.721 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.722 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.723 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.724 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.725 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_PASSWORD": { 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.728 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_URL": { 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_URL", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.729 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_USERNAME": { 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.730 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.731 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.732 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.733 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.734 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "[]", 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.735 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "MULTICLUSTER_CONFIG": { 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "{}", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.736 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_HOST": { 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.737 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.738 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.739 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.740 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.741 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.742 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.743 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.744 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.745 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_ENABLED": { 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.746 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.747 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.748 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.749 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8", 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-11T13:57:44.750 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "45", 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ACCESS_KEY": { 2024-05-11T13:57:44.751 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-11T13:57:44.752 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SECRET_KEY": { 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.753 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SSL_VERIFY": { 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-11T13:57:44.754 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "UNSAFE_TLS_v1_2": { 2024-05-11T13:57:44.755 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-11T13:57:44.756 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.757 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.758 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "cross_origin_url": { 2024-05-11T13:57:44.759 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cross_origin_url", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "crt_file": { 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crt_file", 2024-05-11T13:57:44.760 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "debug": { 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "debug", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.761 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable/disable debug options", 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "jwt_token_ttl": { 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "jwt_token_ttl", 2024-05-11T13:57:44.762 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "28800", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-11T13:57:44.763 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.764 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.765 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.766 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.767 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.768 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "motd": { 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "motd", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.769 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "The message of the day", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect_resolve_ip_addr": { 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "redirect_resolve_ip_addr", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.770 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.771 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.772 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8080", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.773 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.774 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl_server_port": { 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl_server_port", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8443", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.775 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "redirect", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.776 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect" 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.777 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "url_prefix": { 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url_prefix", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.778 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.779 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_monitoring": { 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_monitoring", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "monitor device health metrics", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.780 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.781 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.782 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.783 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.784 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mark_out_threshold": { 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mark_out_threshold", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.785 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2419200", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": { 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_name", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.786 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "device_health_metrics", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "retention_period": { 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retention_period", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.787 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15552000", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain device health metrics", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_frequency": { 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_frequency", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.788 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "self_heal": { 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "self_heal", 2024-05-11T13:57:44.789 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:44.790 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and check device health", 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.791 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_threshold": { 2024-05-11T13:57:44.792 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_threshold", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7257600", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "diskprediction_local", 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.794 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.795 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.796 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.797 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.798 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "predict_interval": { 2024-05-11T13:57:44.799 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predict_interval", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: "predictor_model": { 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predictor_model", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "prophetstor", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:44.801 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.802 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.803 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "influx", 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": false, 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "influxdb python module not found", 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "batch_size": { 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "batch_size", 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5000", 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.804 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "database": { 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "database", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "ceph", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.805 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": { 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hostname", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.806 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server hostname", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "30", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "5", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.807 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.808 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.809 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.811 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.812 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.813 INFO:teuthology.orchestra.run.smithi063.stdout: "password": { 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "password", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "password of InfluxDB server user", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "port": { 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "port", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8086", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server port", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "false", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "threads": { 2024-05-11T13:57:44.817 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threads", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "32", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "username": { 2024-05-11T13:57:44.818 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "username", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "username of InfluxDB server user", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "verify_ssl": { 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "verify_ssl", 2024-05-11T13:57:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "true", 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "insights", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.821 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.822 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.823 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.824 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.825 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "iostat", 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.827 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.828 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.829 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.830 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.831 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.831 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.831 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.831 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.831 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.831 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.831 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.834 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.835 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "k8sevents", 2024-05-11T13:57:44.836 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_event_retention_days": { 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ceph_event_retention_days", 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7", 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.837 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "config_check_secs": { 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_check_secs", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "10", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.839 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.840 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.843 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.844 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "localpool", 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "failure_domain": { 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "failure_domain", 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "host", 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.846 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "failure domain for any created local pool", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.848 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.849 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.850 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.851 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": { 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_size", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default min_size for any created local pool", 2024-05-11T13:57:44.852 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "num_rep": { 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "num_rep", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default replica count for any created local pool", 2024-05-11T13:57:44.853 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": { 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_num", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "128", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default pg_num for any created local pool", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.854 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "prefix": { 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prefix", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name prefix for any created local pool", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.855 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.856 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "subtree": { 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "subtree", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "rack", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.857 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mds_autoscaler", 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.858 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.859 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.860 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.861 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.862 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.863 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirroring", 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.864 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.865 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.866 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.867 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.868 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "nfs", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.871 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.872 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.873 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.874 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.875 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-11T13:57:44.876 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "fail_fs": { 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "fail_fs", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.877 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.878 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.879 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.881 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.882 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator": { 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rook", 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: "test_orchestrator" 2024-05-11T13:57:44.883 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Orchestrator backend", 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.884 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_perf_query", 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.885 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.886 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.887 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.888 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.889 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.890 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.891 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_support", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.892 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.893 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.895 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.896 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.897 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_autoscaler", 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.898 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.899 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.900 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.901 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.902 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.903 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.904 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "threshold": { 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threshold", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1.0", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.905 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "scaling threshold", 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "progress", 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_pg_recovery_event": { 2024-05-11T13:57:44.906 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_pg_recovery_event", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-11T13:57:44.907 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.908 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.909 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.910 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.911 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.912 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "max_completed_events": { 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_completed_events", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "50", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "number of past completed events to remember", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long the module is going to sleep", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.915 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.916 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus", 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "cache": { 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cache", 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.917 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "exclude_perf_counters": { 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "exclude_perf_counters", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.918 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.919 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.920 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.922 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.923 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools": { 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.924 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.925 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "300", 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_interval": { 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_interval", 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15.0", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.927 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.928 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "9283", 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.929 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.930 INFO:teuthology.orchestra.run.smithi063.stdout: "stale_cache_strategy": { 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stale_cache_strategy", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "log", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "default", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "default", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "error" 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.932 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.933 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.935 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.936 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.937 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.938 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_snap_create": { 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_snap_create", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.939 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "mirror_snapshot_schedule": { 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirror_snapshot_schedule", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.940 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.941 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "trash_purge_schedule": { 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "trash_purge_schedule", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.942 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "restful", 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_auth": { 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_auth", 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.943 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.944 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.945 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.946 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.947 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.948 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.949 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:44.950 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:44.951 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.952 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rgw", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.954 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.955 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.956 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.957 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.958 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.959 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rook", 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.960 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.961 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.962 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.963 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.964 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.965 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "storage_class": { 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "storage_class", 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "local", 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.966 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "selftest", 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.967 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.968 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.969 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.970 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "roption1": { 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption1", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "roption2": { 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption2", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "xyz", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.973 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption1": { 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption1", 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.974 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption2": { 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption2", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.975 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption3": { 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption3", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.976 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption4": { 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption4", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.977 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption5": { 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption5", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.978 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption6": { 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption6", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.979 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption7": { 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption7", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "42", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.981 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "testkey": { 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testkey", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.982 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "testlkey": { 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testlkey", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.983 INFO:teuthology.orchestra.run.smithi063.stdout: "testnewline": { 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testnewline", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smb", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.985 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.986 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.987 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.988 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.989 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.990 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "update_orchestration": { 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "update_orchestration", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.991 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:44.992 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_m_granularity": { 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_m_granularity", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow minute scheduled snapshots", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.993 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "dump_on_update": { 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dump_on_update", 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.994 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "dump database to debug log on update", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.995 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.996 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:44.998 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:44.999 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.000 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stats", 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.001 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.002 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.003 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.004 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.005 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.006 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "status", 2024-05-11T13:57:45.007 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.008 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.009 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.010 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.012 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.013 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telegraf", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "address": { 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "address", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.014 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.015 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.016 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.017 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.018 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.019 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.020 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telemetry", 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_basic": { 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_basic", 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.022 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_crash": { 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_crash", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.023 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_device": { 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_device", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.024 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_ident": { 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_ident", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.025 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_perf": { 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_perf", 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.026 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "contact": { 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "contact", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "description": { 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "description", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.028 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "device_url": { 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_url", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.030 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "24", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "8", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.031 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "last_opt_revision": { 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "last_opt_revision", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.032 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard": { 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard_description": { 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard_description", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.034 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.035 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.037 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.038 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "organization": { 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "organization", 2024-05-11T13:57:45.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "proxy": { 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "proxy", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.040 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.041 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "url": { 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.042 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "test_orchestrator", 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.043 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.044 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.045 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.046 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.047 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.049 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.050 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.051 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.052 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_clones": { 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_clones", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "periodic_async_work": { 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "periodic_async_work", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Periodically check for async work", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_delay": { 2024-05-11T13:57:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_delay", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_no_wait": { 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_no_wait", 2024-05-11T13:57:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix", 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "discovery_interval": { 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "discovery_interval", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "100", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "identifier": { 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "identifier", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_host": { 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_host", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_port": { 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_port", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10051", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_sender": { 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_sender", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "gid": 25009, 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "z", 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_features": 4540701547738038271, 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "available_modules": [ 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "alerts", 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How frequently to reexamine health status", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_destination": { 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_destination", 2024-05-11T13:57:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email address to send alerts to", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_from_name": { 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_from_name", 2024-05-11T13:57:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "Ceph", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email From: name", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_host": { 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_host", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP server", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_password": { 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_password", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Password to authenticate with", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_port": { 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_port", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "465", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP port", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_sender": { 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_sender", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP envelope sender", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_ssl": { 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_ssl", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_user": { 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_user", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "User to authenticate as", 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "balancer", 2024-05-11T13:57:45.087 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "active": { 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "active", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically balance PGs across cluster", 2024-05-11T13:57:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_time": { 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_time", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0000", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "beginning time of day to automatically balance", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T13:57:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_weekday": { 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_weekday", 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-11T13:57:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_max_iterations": { 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_max_iterations", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "25", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "250", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_metrics": { 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_metrics", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "pgs,objects,bytes", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_step": { 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_step", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.5", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0.001", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "0.999", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "aggressiveness of optimization", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "end_time": { 2024-05-11T13:57:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_time", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2359", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ending time of day to automatically balance", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "end_weekday": { 2024-05-11T13:57:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_weekday", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "min_score": { 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_score", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "upmap", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-compat", 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "none", 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "read", 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap", 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap-read" 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Balancer mode", 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_ids": { 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_ids", 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_deviation": { 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_deviation", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-11T13:57:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_optimizations": { 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_optimizations", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-11T13:57:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm", 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_down_multiplier": { 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_down_multiplier", 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_refresh_rate": { 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_refresh_rate", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "20", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_starting_port": { 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_starting_port", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4721", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_ptrace": { 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_ptrace", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_interval": { 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_interval", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_memory_target_ratio": { 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_memory_target_ratio", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.7", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm_log_destination": { 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm_log_destination", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "file", 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "file,syslog", 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "syslog" 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "cgroups_split": { 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cgroups_split", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "config_checks_enabled": { 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_checks_enabled", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "config_dashboard": { 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_dashboard", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_alertmanager": { 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_alertmanager", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_base": { 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_base", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Container image name, without the tag", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_elasticsearch": { 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_elasticsearch", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "elasticsearch container image", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_grafana": { 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_grafana", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_haproxy": { 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_haproxy", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "HAproxy container image", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_agent": { 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_agent", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger agent container image", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_collector": { 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_collector", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger collector container image", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_query": { 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_query", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger query container image", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_keepalived": { 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_keepalived", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Keepalived container image", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_loki": { 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_loki", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Loki container image", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_node_exporter": { 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_node_exporter", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_nvmeof": { 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_nvmeof", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Nvme-of container image", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_prometheus": { 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_prometheus", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_promtail": { 2024-05-11T13:57:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_promtail", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Promtail container image", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_samba": { 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_samba", 2024-05-11T13:57:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Samba/SMB container image", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_snmp_gateway": { 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_snmp_gateway", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SNMP Gateway container image", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "container_init": { 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_init", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Run podman/docker with `--init`", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "daemon_cache_timeout": { 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "daemon_cache_timeout", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "default_cephadm_command_timeout": { 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_cephadm_command_timeout", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "900", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "default_registry": { 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_registry", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-11T13:57:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "device_cache_timeout": { 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_cache_timeout", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1800", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache device inventory", 2024-05-11T13:57:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "device_enhanced_scan": { 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_enhanced_scan", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "facts_cache_timeout": { 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "facts_cache_timeout", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache host facts data", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "grafana_dashboards_path": { 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "grafana_dashboards_path", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "host_check_interval": { 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "host_check_interval", 2024-05-11T13:57:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to perform a host check", 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "hw_monitoring": { 2024-05-11T13:57:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hw_monitoring", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "inventory_list_all": { 2024-05-11T13:57:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "inventory_list_all", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "log_refresh_metadata": { 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_refresh_metadata", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "*", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "max_count_per_host": { 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_count_per_host", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of daemons per service per host", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd_draining_count": { 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_osd_draining_count", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "migration_current": { 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "migration_current", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.154 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "internal - do not modify", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "root", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm-package", 2024-05-11T13:57:45.155 INFO:teuthology.orchestra.run.smithi063.stdout: "root" 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "mode for remote execution of cephadm", 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "oob_default_addr": { 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "oob_default_addr", 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "169.254.1.1", 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.156 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "prometheus_alerts_path": { 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus_alerts_path", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_insecure": { 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_insecure", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_password": { 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_password", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-11T13:57:45.159 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_url": { 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_url", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-11T13:57:45.160 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_username": { 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_username", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.161 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "secure_monitoring_stack": { 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "secure_monitoring_stack", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "service_discovery_port": { 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "service_discovery_port", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8765", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "cephadm service discovery port", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.163 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "ssh_config_file": { 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssh_config_file", 2024-05-11T13:57:45.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "use_agent": { 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_agent", 2024-05-11T13:57:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "use_repo_digest": { 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_repo_digest", 2024-05-11T13:57:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_failed_host_check": { 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_failed_host_check", 2024-05-11T13:57:45.167 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if the host check fails", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_daemons": { 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_daemons", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.168 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_hosts": { 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_hosts", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.169 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crash", 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.170 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.171 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.172 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.173 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.174 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "retain_interval": { 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retain_interval", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.175 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "31536000", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.176 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_recent_interval": { 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_recent_interval", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1209600", 2024-05-11T13:57:45.177 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dashboard", 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.178 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_HOST": { 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.180 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_ENABLED": { 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_ENABLED", 2024-05-11T13:57:45.181 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-11T13:57:45.182 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "ENABLE_BROWSABLE_API": { 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-11T13:57:45.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.184 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.185 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.186 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.187 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.188 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.189 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.190 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_PASSWORD": { 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.192 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_URL": { 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_URL", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_USERNAME": { 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-11T13:57:45.195 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.196 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.198 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "[]", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.199 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "MULTICLUSTER_CONFIG": { 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "{}", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.200 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_HOST": { 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-11T13:57:45.203 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.204 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.205 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.206 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.207 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.208 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_ENABLED": { 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.209 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-11T13:57:45.211 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.212 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "45", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ACCESS_KEY": { 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.213 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.214 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SECRET_KEY": { 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.215 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SSL_VERIFY": { 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.216 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "UNSAFE_TLS_v1_2": { 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.217 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.218 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.219 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.220 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "cross_origin_url": { 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cross_origin_url", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "crt_file": { 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crt_file", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "debug": { 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "debug", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable/disable debug options", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.223 INFO:teuthology.orchestra.run.smithi063.stdout: "jwt_token_ttl": { 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "jwt_token_ttl", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "28800", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.224 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.225 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.226 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.227 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.228 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.229 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "motd": { 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "motd", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.230 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "The message of the day", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect_resolve_ip_addr": { 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "redirect_resolve_ip_addr", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.231 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.232 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8080", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.235 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl_server_port": { 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl_server_port", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8443", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-11T13:57:45.236 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "redirect", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect" 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.237 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "url_prefix": { 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url_prefix", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.238 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-11T13:57:45.239 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_monitoring": { 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_monitoring", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "monitor device health metrics", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.241 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.243 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.244 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "mark_out_threshold": { 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mark_out_threshold", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2419200", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": { 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_name", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "device_health_metrics", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "retention_period": { 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retention_period", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15552000", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain device health metrics", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_frequency": { 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_frequency", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.248 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "self_heal": { 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "self_heal", 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-11T13:57:45.249 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and check device health", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.250 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.251 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_threshold": { 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_threshold", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7257600", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.252 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "diskprediction_local", 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.253 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.254 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.255 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.256 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.257 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "predict_interval": { 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predict_interval", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.258 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "predictor_model": { 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predictor_model", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "prophetstor", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.259 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.260 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.261 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "influx", 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": false, 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "influxdb python module not found", 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.262 INFO:teuthology.orchestra.run.smithi063.stdout: "batch_size": { 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "batch_size", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5000", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "database": { 2024-05-11T13:57:45.263 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "database", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "ceph", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": { 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hostname", 2024-05-11T13:57:45.264 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server hostname", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.265 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "30", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "5", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.266 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.268 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.269 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.270 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "password": { 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "password", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.271 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "password of InfluxDB server user", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "port": { 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "port", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.272 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8086", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server port", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.273 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "false", 2024-05-11T13:57:45.274 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "threads": { 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threads", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "32", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "username": { 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "username", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.276 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "username of InfluxDB server user", 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "verify_ssl": { 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "verify_ssl", 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "true", 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.277 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "insights", 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.278 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.279 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.280 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.281 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "iostat", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.285 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.286 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.287 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.288 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.289 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.290 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "k8sevents", 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_event_retention_days": { 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ceph_event_retention_days", 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.291 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "config_check_secs": { 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_check_secs", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.292 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "10", 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.293 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.295 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.298 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "localpool", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "failure_domain": { 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "failure_domain", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "host", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "failure domain for any created local pool", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-11T13:57:45.300 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.301 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.302 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.303 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.304 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": { 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_size", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.305 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default min_size for any created local pool", 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "num_rep": { 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "num_rep", 2024-05-11T13:57:45.306 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3", 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default replica count for any created local pool", 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": { 2024-05-11T13:57:45.307 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_num", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "128", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default pg_num for any created local pool", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "prefix": { 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prefix", 2024-05-11T13:57:45.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name prefix for any created local pool", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.309 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "subtree": { 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "subtree", 2024-05-11T13:57:45.310 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "rack", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mds_autoscaler", 2024-05-11T13:57:45.311 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.312 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.313 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.314 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.315 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.316 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirroring", 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.317 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.318 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.319 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.320 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.321 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "nfs", 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.324 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.325 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.326 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.327 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.328 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.329 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "fail_fs": { 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "fail_fs", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.331 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.332 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.334 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator": { 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "rook", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "test_orchestrator" 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Orchestrator backend", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.337 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_perf_query", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.338 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.339 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.340 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.341 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.342 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.343 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_support", 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.344 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.345 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.348 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.349 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_autoscaler", 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.350 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.351 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.352 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.353 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.355 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "threshold": { 2024-05-11T13:57:45.356 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threshold", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1.0", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "scaling threshold", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "progress", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_pg_recovery_event": { 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_pg_recovery_event", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.358 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.360 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.361 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.362 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.363 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.364 INFO:teuthology.orchestra.run.smithi063.stdout: "max_completed_events": { 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_completed_events", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "50", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "number of past completed events to remember", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.365 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long the module is going to sleep", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.366 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.367 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "cache": { 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cache", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.368 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "exclude_perf_counters": { 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "exclude_perf_counters", 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.369 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.370 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.372 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.373 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.374 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools": { 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "300", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_interval": { 2024-05-11T13:57:45.376 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_interval", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15.0", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:45.377 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.378 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "9283", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.379 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "stale_cache_strategy": { 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stale_cache_strategy", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "log", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.380 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "default", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "default", 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: "error" 2024-05-11T13:57:45.381 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.382 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.383 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.384 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.386 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.387 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_snap_create": { 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_snap_create", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.388 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "mirror_snapshot_schedule": { 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirror_snapshot_schedule", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.389 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.390 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "trash_purge_schedule": { 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "trash_purge_schedule", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.391 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "restful", 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.392 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_auth": { 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_auth", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.393 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.394 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.395 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.396 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.397 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.400 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.401 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rgw", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.403 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.404 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.405 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.406 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.407 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.408 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rook", 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.409 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.411 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.412 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.413 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.414 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "storage_class": { 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "storage_class", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "local", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.415 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "selftest", 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.416 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.417 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.419 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.420 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "roption1": { 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption1", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.421 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "roption2": { 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption2", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "xyz", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.422 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption1": { 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption1", 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.423 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption2": { 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption2", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.424 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption3": { 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption3", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.425 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption4": { 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption4", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.426 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption5": { 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption5", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.427 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption6": { 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption6", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.428 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption7": { 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption7", 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.429 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "42", 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.430 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "testkey": { 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testkey", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.431 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "testlkey": { 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testlkey", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.432 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "testnewline": { 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testnewline", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.433 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smb", 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.434 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.435 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.436 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.437 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.438 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.439 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "update_orchestration": { 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "update_orchestration", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_m_granularity": { 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_m_granularity", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.443 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow minute scheduled snapshots", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "dump_on_update": { 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dump_on_update", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "dump database to debug log on update", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.445 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.446 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.448 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stats", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.451 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.452 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.453 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.454 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.455 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.456 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.457 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "status", 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.458 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.459 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.460 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.461 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.462 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.463 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telegraf", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "address": { 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "address", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.465 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15", 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.466 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.467 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.469 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.470 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.471 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.472 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telemetry", 2024-05-11T13:57:45.473 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_basic": { 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_basic", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_crash": { 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_crash", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_device": { 2024-05-11T13:57:45.475 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_device", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_ident": { 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_ident", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.476 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_perf": { 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_perf", 2024-05-11T13:57:45.477 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "contact": { 2024-05-11T13:57:45.478 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "contact", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "description": { 2024-05-11T13:57:45.479 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "description", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "device_url": { 2024-05-11T13:57:45.480 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_url", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-11T13:57:45.481 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.482 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "24", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "8", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "last_opt_revision": { 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "last_opt_revision", 2024-05-11T13:57:45.483 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard": { 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.484 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard_description": { 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard_description", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.485 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.486 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.487 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.488 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.489 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.490 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "organization": { 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "organization", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.491 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "proxy": { 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "proxy", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.493 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "url": { 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.494 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "test_orchestrator", 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.495 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.496 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.497 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.498 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.499 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.500 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.501 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.504 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.505 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_clones": { 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_clones", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.506 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "periodic_async_work": { 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "periodic_async_work", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.507 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Periodically check for async work", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_delay": { 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_delay", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.508 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_no_wait": { 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_no_wait", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.509 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.510 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix", 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "discovery_interval": { 2024-05-11T13:57:45.511 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "discovery_interval", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "100", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "identifier": { 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "identifier", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.512 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.513 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.514 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.515 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.517 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.518 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.519 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_host": { 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_host", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_port": { 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_port", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.521 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10051", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_sender": { 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_sender", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.522 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "modules": [ 2024-05-11T13:57:45.523 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "dashboard", 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "iostat", 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "nfs", 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "restful", 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_schedule" 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "available_modules": [ 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "alerts", 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.524 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How frequently to reexamine health status", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.525 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.526 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.528 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.529 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_destination": { 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_destination", 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.530 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email address to send alerts to", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_from_name": { 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_from_name", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "Ceph", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.531 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email From: name", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_host": { 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_host", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.532 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP server", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_password": { 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_password", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.533 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Password to authenticate with", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_port": { 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_port", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "465", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP port", 2024-05-11T13:57:45.534 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_sender": { 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_sender", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP envelope sender", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.535 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_ssl": { 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_ssl", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.536 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_user": { 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_user", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "User to authenticate as", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "balancer", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "active": { 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "active", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.539 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically balance PGs across cluster", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_time": { 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_time", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0000", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "beginning time of day to automatically balance", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_weekday": { 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_weekday", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_max_iterations": { 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_max_iterations", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "25", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "250", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_metrics": { 2024-05-11T13:57:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_metrics", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "pgs,objects,bytes", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_step": { 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_step", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.5", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0.001", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "0.999", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "aggressiveness of optimization", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "end_time": { 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_time", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.544 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2359", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ending time of day to automatically balance", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "end_weekday": { 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_weekday", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-11T13:57:45.545 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.546 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.547 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.548 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.549 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "min_score": { 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_score", 2024-05-11T13:57:45.550 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.551 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "upmap", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-compat", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "none", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "read", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap-read" 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Balancer mode", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_ids": { 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_ids", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.553 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_deviation": { 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_deviation", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_optimizations": { 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_optimizations", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.556 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm", 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.557 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_down_multiplier": { 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_down_multiplier", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_refresh_rate": { 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_refresh_rate", 2024-05-11T13:57:45.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "20", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_starting_port": { 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_starting_port", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.559 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4721", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_ptrace": { 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_ptrace", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.560 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_interval": { 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_interval", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_memory_target_ratio": { 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_memory_target_ratio", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.7", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.562 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm_log_destination": { 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm_log_destination", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "file", 2024-05-11T13:57:45.563 INFO:teuthology.orchestra.run.smithi063.stdout: "file,syslog", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "syslog" 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "cgroups_split": { 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cgroups_split", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.564 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "config_checks_enabled": { 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_checks_enabled", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.565 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "config_dashboard": { 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_dashboard", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_alertmanager": { 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_alertmanager", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_base": { 2024-05-11T13:57:45.567 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_base", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Container image name, without the tag", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_elasticsearch": { 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_elasticsearch", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.568 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "elasticsearch container image", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_grafana": { 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_grafana", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-11T13:57:45.569 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_haproxy": { 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_haproxy", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.570 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "HAproxy container image", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_agent": { 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_agent", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger agent container image", 2024-05-11T13:57:45.571 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_collector": { 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_collector", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger collector container image", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.572 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_query": { 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_query", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger query container image", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.573 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_keepalived": { 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_keepalived", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Keepalived container image", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_loki": { 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_loki", 2024-05-11T13:57:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Loki container image", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_node_exporter": { 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_node_exporter", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_nvmeof": { 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_nvmeof", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-11T13:57:45.576 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Nvme-of container image", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_prometheus": { 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_prometheus", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.577 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_promtail": { 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_promtail", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Promtail container image", 2024-05-11T13:57:45.578 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_samba": { 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_samba", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Samba/SMB container image", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.579 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_snmp_gateway": { 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_snmp_gateway", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SNMP Gateway container image", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "container_init": { 2024-05-11T13:57:45.580 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_init", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Run podman/docker with `--init`", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "daemon_cache_timeout": { 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "daemon_cache_timeout", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.581 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "default_cephadm_command_timeout": { 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_cephadm_command_timeout", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.582 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "900", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "default_registry": { 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_registry", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.583 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "device_cache_timeout": { 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_cache_timeout", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1800", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.584 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache device inventory", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "device_enhanced_scan": { 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_enhanced_scan", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-11T13:57:45.585 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "facts_cache_timeout": { 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "facts_cache_timeout", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache host facts data", 2024-05-11T13:57:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "grafana_dashboards_path": { 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "grafana_dashboards_path", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "host_check_interval": { 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "host_check_interval", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to perform a host check", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.588 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "hw_monitoring": { 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hw_monitoring", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "inventory_list_all": { 2024-05-11T13:57:45.589 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "inventory_list_all", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.590 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.591 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "log_refresh_metadata": { 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_refresh_metadata", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.593 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.594 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.595 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.596 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "*", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "max_count_per_host": { 2024-05-11T13:57:45.597 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_count_per_host", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of daemons per service per host", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.598 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd_draining_count": { 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_osd_draining_count", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "migration_current": { 2024-05-11T13:57:45.599 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "migration_current", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "internal - do not modify", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.600 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "root", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm-package", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "root" 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "mode for remote execution of cephadm", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "oob_default_addr": { 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "oob_default_addr", 2024-05-11T13:57:45.601 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "169.254.1.1", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "prometheus_alerts_path": { 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus_alerts_path", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.602 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_insecure": { 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_insecure", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.603 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_password": { 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_password", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.604 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_url": { 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_url", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.605 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_username": { 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_username", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.606 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "secure_monitoring_stack": { 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "secure_monitoring_stack", 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.607 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "service_discovery_port": { 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "service_discovery_port", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8765", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.608 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "cephadm service discovery port", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.609 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "ssh_config_file": { 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssh_config_file", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.610 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "use_agent": { 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_agent", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.611 INFO:teuthology.orchestra.run.smithi063.stdout: "use_repo_digest": { 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_repo_digest", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_failed_host_check": { 2024-05-11T13:57:45.612 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_failed_host_check", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if the host check fails", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_daemons": { 2024-05-11T13:57:45.613 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_daemons", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_hosts": { 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_hosts", 2024-05-11T13:57:45.614 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crash", 2024-05-11T13:57:45.615 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.616 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.617 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.618 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.619 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.620 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "retain_interval": { 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retain_interval", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "31536000", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.621 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.622 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_recent_interval": { 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_recent_interval", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1209600", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.623 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dashboard", 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.624 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_HOST": { 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.625 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.626 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_ENABLED": { 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_ENABLED", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.627 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.628 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "ENABLE_BROWSABLE_API": { 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.629 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.630 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.631 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.634 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.637 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_PASSWORD": { 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.638 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.639 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_URL": { 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_URL", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.640 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_USERNAME": { 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.642 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-11T13:57:45.643 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.644 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "[]", 2024-05-11T13:57:45.645 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "MULTICLUSTER_CONFIG": { 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.646 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "{}", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_HOST": { 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.647 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.648 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.649 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.650 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.651 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.652 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.653 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-11T13:57:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_ENABLED": { 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-11T13:57:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.658 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "45", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.659 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ACCESS_KEY": { 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.660 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.661 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SECRET_KEY": { 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.662 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SSL_VERIFY": { 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.663 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "UNSAFE_TLS_v1_2": { 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.664 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.665 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-11T13:57:45.666 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.667 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "cross_origin_url": { 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cross_origin_url", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.669 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "crt_file": { 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crt_file", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.670 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "debug": { 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "debug", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable/disable debug options", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.671 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "jwt_token_ttl": { 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "jwt_token_ttl", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "28800", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.673 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.674 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.675 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.676 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.677 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.678 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "motd": { 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "motd", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.679 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "The message of the day", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect_resolve_ip_addr": { 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "redirect_resolve_ip_addr", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.680 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.681 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8080", 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.682 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.683 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.684 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl_server_port": { 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl_server_port", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8443", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.685 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "redirect", 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.686 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect" 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.687 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "url_prefix": { 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url_prefix", 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.688 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-11T13:57:45.689 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_monitoring": { 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_monitoring", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "monitor device health metrics", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.690 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.691 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.692 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.693 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.694 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "mark_out_threshold": { 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mark_out_threshold", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2419200", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.696 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": { 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_name", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "device_health_metrics", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.697 INFO:teuthology.orchestra.run.smithi063.stdout: "retention_period": { 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retention_period", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15552000", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain device health metrics", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_frequency": { 2024-05-11T13:57:45.698 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_frequency", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "self_heal": { 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "self_heal", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.699 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.700 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and check device health", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.701 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_threshold": { 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_threshold", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7257600", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.702 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "diskprediction_local", 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.703 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.704 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.705 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.706 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.707 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "predict_interval": { 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predict_interval", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.708 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "predictor_model": { 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predictor_model", 2024-05-11T13:57:45.709 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "prophetstor", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.711 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "influx", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": false, 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "influxdb python module not found", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "batch_size": { 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "batch_size", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5000", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.713 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "database": { 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "database", 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "ceph", 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.714 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": { 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hostname", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server hostname", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.715 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "30", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "5", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.716 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.717 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.718 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.719 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.720 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "password": { 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "password", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "password of InfluxDB server user", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "port": { 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "port", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8086", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server port", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.723 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.724 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "false", 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T13:57:45.725 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "threads": { 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threads", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "32", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-11T13:57:45.726 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "username": { 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "username", 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.727 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "username of InfluxDB server user", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "verify_ssl": { 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "verify_ssl", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "true", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.728 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "insights", 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.729 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.730 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.731 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.732 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.733 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.734 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "iostat", 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.735 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.736 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.737 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.738 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.739 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.740 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "k8sevents", 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.741 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_event_retention_days": { 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ceph_event_retention_days", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.742 INFO:teuthology.orchestra.run.smithi063.stdout: "config_check_secs": { 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_check_secs", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "10", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.743 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.744 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.745 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.746 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.747 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.748 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "localpool", 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "failure_domain": { 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "failure_domain", 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "host", 2024-05-11T13:57:45.750 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "failure domain for any created local pool", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.751 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.754 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.755 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": { 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_size", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.756 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default min_size for any created local pool", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "num_rep": { 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "num_rep", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3", 2024-05-11T13:57:45.757 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default replica count for any created local pool", 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": { 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_num", 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.758 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "128", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default pg_num for any created local pool", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "prefix": { 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prefix", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.759 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name prefix for any created local pool", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.760 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "subtree": { 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "subtree", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.761 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "rack", 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.762 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mds_autoscaler", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.763 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.764 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.765 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirroring", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.769 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.770 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.771 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.772 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.773 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "nfs", 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.775 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.776 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.778 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.779 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.780 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "fail_fs": { 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "fail_fs", 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.781 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.782 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.783 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.784 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.785 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.786 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator": { 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.787 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "rook", 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "test_orchestrator" 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Orchestrator backend", 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.788 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_perf_query", 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.789 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.790 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.791 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.792 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.793 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.794 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.795 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_support", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.796 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.797 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.798 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.799 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.800 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.801 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_autoscaler", 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.802 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.805 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.806 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.808 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "threshold": { 2024-05-11T13:57:45.809 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threshold", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1.0", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "scaling threshold", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.810 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "progress", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_pg_recovery_event": { 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_pg_recovery_event", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.811 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.812 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.813 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.814 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.816 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.817 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "max_completed_events": { 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_completed_events", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "50", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.818 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "number of past completed events to remember", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.819 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long the module is going to sleep", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.820 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus", 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "cache": { 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cache", 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.821 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "exclude_perf_counters": { 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "exclude_perf_counters", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.822 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.824 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.825 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.826 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.827 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.828 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools": { 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools", 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.829 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.830 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "300", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_interval": { 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_interval", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.831 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15.0", 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:45.832 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:45.833 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "9283", 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.835 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "stale_cache_strategy": { 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stale_cache_strategy", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "log", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.836 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "default", 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "default", 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: "error" 2024-05-11T13:57:45.837 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-11T13:57:45.838 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.839 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.840 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.841 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.842 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.843 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.844 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_snap_create": { 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_snap_create", 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.845 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "mirror_snapshot_schedule": { 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirror_snapshot_schedule", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.846 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.847 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "trash_purge_schedule": { 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "trash_purge_schedule", 2024-05-11T13:57:45.848 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "restful", 2024-05-11T13:57:45.849 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_auth": { 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_auth", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.850 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.851 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.852 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.853 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.854 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.855 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-11T13:57:45.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.857 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.858 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.859 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rgw", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.860 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.861 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.862 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.864 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rook", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.866 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.867 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.868 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.869 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.870 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.871 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "storage_class": { 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "storage_class", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "local", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.872 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "selftest", 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.873 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.874 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.875 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.876 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.877 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "roption1": { 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption1", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.878 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "roption2": { 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption2", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "xyz", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.879 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption1": { 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption1", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.880 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption2": { 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption2", 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.881 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption3": { 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption3", 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.882 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption4": { 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption4", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.883 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption5": { 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption5", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.884 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption6": { 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption6", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.885 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption7": { 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption7", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "42", 2024-05-11T13:57:45.886 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.887 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "testkey": { 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testkey", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.888 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "testlkey": { 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testlkey", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.889 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "testnewline": { 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testnewline", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.890 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smb", 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.891 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.893 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.894 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.895 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.896 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "update_orchestration": { 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "update_orchestration", 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.897 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_m_granularity": { 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_m_granularity", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.899 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow minute scheduled snapshots", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "dump_on_update": { 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dump_on_update", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.900 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "dump database to debug log on update", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.901 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.902 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.904 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.905 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.906 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stats", 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.907 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.908 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.909 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.910 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.911 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.912 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.913 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "status", 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.914 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.915 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.916 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.917 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.918 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.919 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telegraf", 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "address": { 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "address", 2024-05-11T13:57:45.920 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.921 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15", 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.922 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.923 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.924 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.926 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.929 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.930 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telemetry", 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_basic": { 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_basic", 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.931 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_crash": { 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_crash", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_device": { 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_device", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.933 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_ident": { 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_ident", 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.934 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_perf": { 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_perf", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.935 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "contact": { 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "contact", 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.936 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "description": { 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "description", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.937 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "device_url": { 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_url", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.938 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.939 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "24", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "8", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.940 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "last_opt_revision": { 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "last_opt_revision", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.941 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard": { 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.942 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard_description": { 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard_description", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.944 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.945 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.946 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.947 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "organization": { 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "organization", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.948 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "proxy": { 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "proxy", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.949 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.950 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "url": { 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.951 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "test_orchestrator", 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.952 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.953 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.954 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.955 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.956 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.957 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.958 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.959 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.960 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.961 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.962 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_clones": { 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_clones", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.963 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "periodic_async_work": { 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "periodic_async_work", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Periodically check for async work", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.964 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_delay": { 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_delay", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_no_wait": { 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_no_wait", 2024-05-11T13:57:45.965 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.966 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix", 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-11T13:57:45.967 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "discovery_interval": { 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "discovery_interval", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "100", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "identifier": { 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "identifier", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-11T13:57:45.969 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.970 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.971 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.972 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.973 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.974 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.975 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_host": { 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_host", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.978 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_port": { 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_port", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10051", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_sender": { 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_sender", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-11T13:57:45.980 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "services": { 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "dashboard": "https://172.21.15.63:8443/" 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "always_on_modules": { 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "octopus": [ 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-11T13:57:45.981 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "pacific": [ 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-11T13:57:45.982 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "quincy": [ 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-11T13:57:45.983 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "reef": [ 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "squid": [ 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-11T13:57:45.984 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "last_failure_osd_epoch": 91, 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "active_clients": [ 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-11T13:57:45.985 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3731819671 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "libcephsqlite", 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3772056260 2024-05-11T13:57:45.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 139218293 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 197218610 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-11T13:57:45.988 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2123984367 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2796213987 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:45.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-11T13:57:45.990 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:57:45.991 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:44 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2514212096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-11T13:57:45.991 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:44 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1125881467' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-11T13:57:45.991 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:44 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2514212096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-11T13:57:45.991 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:44 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1125881467' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-11T13:57:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83100b2960 msgr2=0x7f83100b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83100b2960 0x7f83100b4e20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8318005fd0 tx=0x7f8318005850 comp rx=0 tx=0).stop 2024-05-11T13:57:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 msgr2=0x7f832c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c19f6e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f831c006010 tx=0x7f831c0042f0 comp rx=0 tx=0).stop 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 shutdown_connections 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83100b2960 0x7f83100b4e20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f832c113700 0x7f832c1a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0778b0 0x7f832c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 --2- 172.21.15.63:0/1125881467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832c076f90 0x7f832c19f6e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 >> 172.21.15.63:0/1125881467 conn(0x7f832c1004f0 msgr2=0x7f832c114f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 shutdown_connections 2024-05-11T13:57:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:57:44.612+0000 7f832b200640 1 -- 172.21.15.63:0/1125881467 wait complete. 2024-05-11T13:57:46.000 DEBUG:tasks.check_counter:Getting stats from a 2024-05-11T13:57:46.000 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-11T13:57:46.000 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:44 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2514212096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-11T13:57:46.001 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:44 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1125881467' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-11T13:57:46.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:45 smithi165 ceph-mon[33235]: pgmap v755: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 344 KiB/s wr, 26 op/s 2024-05-11T13:57:46.082 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:45 smithi063 ceph-mon[29809]: pgmap v755: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 344 KiB/s wr, 26 op/s 2024-05-11T13:57:46.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:45 smithi071 ceph-mon[33293]: pgmap v755: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 344 KiB/s wr, 26 op/s 2024-05-11T13:57:46.481 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:57:48.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:47 smithi165 ceph-mon[33235]: pgmap v756: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 302 KiB/s wr, 18 op/s 2024-05-11T13:57:48.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:47 smithi063 ceph-mon[29809]: pgmap v756: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 302 KiB/s wr, 18 op/s 2024-05-11T13:57:48.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:47 smithi071 ceph-mon[33293]: pgmap v756: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 302 KiB/s wr, 18 op/s 2024-05-11T13:57:48.866 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:57:48.866 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:57:48.866 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 3519, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 3519, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 3174923, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 1118219, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 6, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 2, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.689386621, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.236127998, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.179457554, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.198396481, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3518, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.343032343, 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000097507 2024-05-11T13:57:48.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 448 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1440, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1797, 2024-05-11T13:57:48.868 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 354720, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 1234488, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 7, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 5, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.363779255, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.145912864, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.069701213, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.107982576, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1794, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.216844643, 2024-05-11T13:57:48.869 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000120872 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 777136 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 579, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 514, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 2057868, 2024-05-11T13:57:48.870 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 298846, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 2, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.189087572, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.077795254, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.110690902, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.020686216, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 513, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.057833091, 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000112735 2024-05-11T13:57:48.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2057600, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 298528 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 1, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-MDSRank": { 2024-05-11T13:57:48.872 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 8152, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.437853710, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000053711 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-PurgeQueue": { 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.873 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "request": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "reply": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "reply_latency": { 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_reply": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "forward": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_complete": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_keys": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_commit": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_split": 0, 2024-05-11T13:57:48.874 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_merge": 0, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes": 560, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_top": 9, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_bottom": 551, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pin_tail": 0, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pinned": 548, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_expired": 0, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_with_caps": 0, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": 0, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "subtrees": 11, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse": 0, 2024-05-11T13:57:48.875 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_hit": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_forward": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_discover": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_remote_ino": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_lock": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "load_cent": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "q": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "exported": 0, 2024-05-11T13:57:48.876 INFO:teuthology.orchestra.run.smithi063.stdout: "exported_inodes": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "imported": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "imported_inodes": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_dir_fetch": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_peer_discover": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rfiles": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rbytes": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rsnaps": 0, 2024-05-11T13:57:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_set_tag": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_file_inodes": 0, 2024-05-11T13:57:48.878 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_inode_file_caps": 0, 2024-05-11T13:57:48.879 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_revoke": 0, 2024-05-11T13:57:48.879 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_grant": 0, 2024-05-11T13:57:48.879 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:57:48.879 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:57:48.879 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:57:48.879 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps": 0, 2024-05-11T13:57:48.879 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps_dirty": 0, 2024-05-11T13:57:48.880 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_cap_release": 0, 2024-05-11T13:57:48.880 INFO:teuthology.orchestra.run.smithi063.stdout: "process_request_cap_release": 0 2024-05-11T13:57:48.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.880 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache": { 2024-05-11T13:57:48.880 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update": 0, 2024-05-11T13:57:48.880 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update_receipt": 0, 2024-05-11T13:57:48.881 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_try_discover": 0, 2024-05-11T13:57:48.881 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_send_discover": 0, 2024-05-11T13:57:48.881 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_handle_discover": 0, 2024-05-11T13:57:48.881 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays": 0, 2024-05-11T13:57:48.881 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_delayed": 0, 2024-05-11T13:57:48.881 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:57:48.881 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_created": 0, 2024-05-11T13:57:48.882 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_enqueued": 0, 2024-05-11T13:57:48.882 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_reintegrated": 0, 2024-05-11T13:57:48.882 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_migrated": 0, 2024-05-11T13:57:48.882 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_processing": 0, 2024-05-11T13:57:48.882 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:57:48.882 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:57:48.882 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_started": 0, 2024-05-11T13:57:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_completed": 0, 2024-05-11T13:57:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_path": 0, 2024-05-11T13:57:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_inode": 0, 2024-05-11T13:57:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:57:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_exportdir": 0, 2024-05-11T13:57:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_flush": 0, 2024-05-11T13:57:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragmentdir": 0, 2024-05-11T13:57:48.884 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragstats": 0, 2024-05-11T13:57:48.884 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_inodestats": 0 2024-05-11T13:57:48.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.884 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log": { 2024-05-11T13:57:48.884 INFO:teuthology.orchestra.run.smithi063.stdout: "evlrg": 0, 2024-05-11T13:57:48.884 INFO:teuthology.orchestra.run.smithi063.stdout: "evadd": 0, 2024-05-11T13:57:48.884 INFO:teuthology.orchestra.run.smithi063.stdout: "evex": 0, 2024-05-11T13:57:48.885 INFO:teuthology.orchestra.run.smithi063.stdout: "evtrm": 0, 2024-05-11T13:57:48.885 INFO:teuthology.orchestra.run.smithi063.stdout: "ev": 1227, 2024-05-11T13:57:48.885 INFO:teuthology.orchestra.run.smithi063.stdout: "evexg": 0, 2024-05-11T13:57:48.885 INFO:teuthology.orchestra.run.smithi063.stdout: "evexd": 0, 2024-05-11T13:57:48.885 INFO:teuthology.orchestra.run.smithi063.stdout: "segadd": 0, 2024-05-11T13:57:48.885 INFO:teuthology.orchestra.run.smithi063.stdout: "segex": 0, 2024-05-11T13:57:48.885 INFO:teuthology.orchestra.run.smithi063.stdout: "segtrm": 0, 2024-05-11T13:57:48.886 INFO:teuthology.orchestra.run.smithi063.stdout: "seg": 3, 2024-05-11T13:57:48.886 INFO:teuthology.orchestra.run.smithi063.stdout: "segmjr": 2, 2024-05-11T13:57:48.886 INFO:teuthology.orchestra.run.smithi063.stdout: "segexg": 0, 2024-05-11T13:57:48.886 INFO:teuthology.orchestra.run.smithi063.stdout: "segexd": 0, 2024-05-11T13:57:48.886 INFO:teuthology.orchestra.run.smithi063.stdout: "expos": 4194304, 2024-05-11T13:57:48.886 INFO:teuthology.orchestra.run.smithi063.stdout: "wrpos": 6325174, 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rdpos": 6324600, 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: "jlat": { 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: "replayed": 1227 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_mem": { 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "ino": 536, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "ino+": 1229, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "ino-": 693, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "dir": 19, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "dir+": 30, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "dir-": 11, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "dn": 560, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "dn+": 1260, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "dn-": 700, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "cap": 0, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "cap+": 0, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "cap-": 0, 2024-05-11T13:57:48.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rss": 53104, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "heap": 215416 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_server": { 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_client_request": 0, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_server_request": 0, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_request": 0, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_session": 0, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_peer_request": 0, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "req_create_latency": { 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getattr_latency": { 2024-05-11T13:57:48.889 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getfilelock_latency": { 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "req_link_latency": { 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookup_latency": { 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookuphash_latency": { 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupino_latency": { 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupname_latency": { 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupparent_latency": { 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupsnap_latency": { 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.892 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lssnap_latency": { 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.893 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mkdir_latency": { 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mknod_latency": { 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mksnap_latency": { 2024-05-11T13:57:48.894 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "req_open_latency": { 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "req_readdir_latency": { 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.895 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rename_latency": { 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "req_renamesnap_latency": { 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "req_snapdiff_latency": { 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmdir_latency": { 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmsnap_latency": { 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmxattr_latency": { 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setattr_latency": { 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setdirlayout_latency": { 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.898 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setfilelock_latency": { 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setlayout_latency": { 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.899 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setxattr_latency": { 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "req_symlink_latency": { 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "req_unlink_latency": { 2024-05-11T13:57:48.900 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getvxattr_latency": { 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sessions": { 2024-05-11T13:57:48.901 INFO:teuthology.orchestra.run.smithi063.stdout: "session_count": 1, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "session_add": 2, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "session_remove": 1, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_open": 2, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_stale": 0, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "total_load": 0, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "average_load": 0, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "avg_session_uptime": 4644, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "mdthresh_evicted": 0 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:57:48.902 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:57:48.903 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:57:48.904 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 190945, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 54, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:57:48.905 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 9608, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 184, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 1577176, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 9574, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-11T13:57:48.906 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter": { 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "op_active": 0, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "op_laggy": 0, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send": 4954, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send_bytes": 21, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "op_resend": 3, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "op_reply": 4954, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "op_latency": { 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 4954, 2024-05-11T13:57:48.907 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 7.631926536, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.001540558 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "op_inflight": 0, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "oplen_avg": { 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 4954, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 4955 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "op": 4954, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "op_r": 4954, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "op_w": 0, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "op_rmw": 0, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "op_pg": 0, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_stat": 3099, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_create": 0, 2024-05-11T13:57:48.908 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_read": 1853, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_write": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writefull": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writesame": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_append": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_zero": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_truncate": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_delete": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_mapext": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_sparse_read": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_clonerange": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_getxattr": 1, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_setxattr": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_rmxattr": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:57:48.909 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_call": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_watch": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_notify": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_other": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_active": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_send": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_resend": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_ping": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_active": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_send": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_resend": 0, 2024-05-11T13:57:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_active": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_send": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_resend": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_active": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_send": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_resend": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "command_active": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "command_send": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "command_resend": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "map_epoch": 127, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "map_full": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "map_inc": 39, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_sessions": 5, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_open": 6, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_close": 1, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_laggy": 0, 2024-05-11T13:57:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_wr": 0, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_rd": 2, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_del": 0 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "oft": { 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_objs": 0, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_updates": 0, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_removes": 0 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "osd-slow-ops": { 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_ops_count": 0 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "purge_queue": { 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops": 0, 2024-05-11T13:57:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops_high_water": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_high_water": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed_ops": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_item_in_journal": 0 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 5534, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 5057057, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 5534, 2024-05-11T13:57:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 5534, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 5057057, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_bytes": { 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:48.914 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 4954, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 2130875, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 4954, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 2130875, 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:48.915 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_ops": { 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 1024, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 4954, 2024-05-11T13:57:48.916 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 4954, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 4954, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 4954, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:48.917 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:48.918 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle-0x56199e980780": { 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-05-11T13:57:48.919 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:48.920 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:48.920 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:48.920 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:48.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:48.920 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:57:49.864 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-11T13:57:49.864 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-11T13:57:49.864 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-11T13:57:49.864 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-11T13:57:49.865 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-11T13:57:49.865 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:57:49.865 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:57:49.865 DEBUG:tasks.check_counter:Getting stats from d 2024-05-11T13:57:49.865 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-11T13:57:50.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:49 smithi165 ceph-mon[33235]: pgmap v757: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 25 op/s 2024-05-11T13:57:50.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:49 smithi063 ceph-mon[29809]: pgmap v757: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 25 op/s 2024-05-11T13:57:50.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:49 smithi071 ceph-mon[33293]: pgmap v757: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 25 op/s 2024-05-11T13:57:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:57:52.172 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:51 smithi063 ceph-mon[29809]: pgmap v758: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-05-11T13:57:52.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:51 smithi071 ceph-mon[33293]: pgmap v758: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-05-11T13:57:52.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:51 smithi165 ceph-mon[33235]: pgmap v758: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-05-11T13:57:52.762 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:57:52.762 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:57:52.762 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 0, 2024-05-11T13:57:52.762 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 2, 2024-05-11T13:57:52.762 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 0, 2024-05-11T13:57:52.762 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 0, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 0, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.003981309, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000000000, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.000716264, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:52.763 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 0 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 6, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 367, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 3264, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 221952, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 5, 2024-05-11T13:57:52.764 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 3, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.069657271, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.053904635, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.006686930, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 365, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.052975750, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000145139 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-11T13:57:52.765 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 221952 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 488, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 478, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1637744, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 296288, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 2, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.181843592, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.077019224, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.095289182, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.021570016, 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:52.766 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 477, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.053058933, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000111234 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 1637744, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 296288 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 0, 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-11T13:57:52.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:57:52.768 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:57:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 148714, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 30, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:57:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 928, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 2, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-11T13:57:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 490, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 1568756, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 490, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 490, 2024-05-11T13:57:52.772 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 1568756, 2024-05-11T13:57:52.773 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:52.773 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:52.773 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:52.773 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:52.773 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:52.773 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:52.773 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:57:53.738 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-11T13:57:53.738 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-11T13:57:53.738 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-11T13:57:53.738 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-11T13:57:53.739 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-11T13:57:53.739 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-11T13:57:53.739 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-11T13:57:53.739 DEBUG:tasks.check_counter:Getting stats from g 2024-05-11T13:57:53.739 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-11T13:57:53.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:53 smithi063 ceph-mon[29809]: pgmap v759: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-05-11T13:57:54.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:53 smithi071 ceph-mon[33293]: pgmap v759: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-05-11T13:57:54.212 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:57:54.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:53 smithi165 ceph-mon[33235]: pgmap v759: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-05-11T13:57:56.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:55 smithi063 ceph-mon[29809]: pgmap v760: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 71 KiB/s wr, 18 op/s 2024-05-11T13:57:56.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:55 smithi071 ceph-mon[33293]: pgmap v760: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 71 KiB/s wr, 18 op/s 2024-05-11T13:57:56.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:55 smithi165 ceph-mon[33235]: pgmap v760: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 71 KiB/s wr, 18 op/s 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1675, 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1675, 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 564345, 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 531827, 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 7, 2024-05-11T13:57:57.005 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 2, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.412426642, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.181611204, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.090658630, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.100763415, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1674, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.190303620, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000113681 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.006 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 448 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 3995, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 3930, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 4678481, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 1385999, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 3, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.713244717, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.250160393, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.250150917, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.194921788, 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:57.007 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3929, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.387890958, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000098725 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2058752, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 300608 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 5, 2024-05-11T13:57:57.008 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 364, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1668, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 780069, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 8, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 5, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.067955266, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.054003661, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.005293058, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000138699, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 361, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.070623921, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000195634 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:57:57.009 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 779104 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 1, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-MDSRank": { 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-05-11T13:57:57.010 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 8471, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.419147830, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000049480 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-PurgeQueue": { 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "request": 0, 2024-05-11T13:57:57.011 INFO:teuthology.orchestra.run.smithi063.stdout: "reply": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "reply_latency": { 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_reply": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "forward": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_complete": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_keys": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_commit": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_split": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_merge": 0, 2024-05-11T13:57:57.012 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes": 518, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_top": 26, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_bottom": 492, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pin_tail": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pinned": 481, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_expired": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_with_caps": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "subtrees": 12, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_hit": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_forward": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_discover": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_remote_ino": 0, 2024-05-11T13:57:57.013 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_lock": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "load_cent": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "q": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "exported": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "exported_inodes": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "imported": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "imported_inodes": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_dir_fetch": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_peer_discover": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rfiles": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rbytes": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rsnaps": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:57:57.014 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_set_tag": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_file_inodes": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_inode_file_caps": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_revoke": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_grant": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps_dirty": 0, 2024-05-11T13:57:57.015 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_cap_release": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "process_request_cap_release": 0 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache": { 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update_receipt": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_try_discover": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_send_discover": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_handle_discover": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_delayed": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_created": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_enqueued": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_reintegrated": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_migrated": 0, 2024-05-11T13:57:57.016 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_processing": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_started": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_completed": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_path": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_inode": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_exportdir": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_flush": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragmentdir": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragstats": 0, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_inodestats": 0 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.017 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log": { 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "evlrg": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "evadd": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "evex": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "evtrm": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "ev": 1283, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "evexg": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "evexd": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "segadd": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "segex": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "segtrm": 0, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "seg": 3, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "segmjr": 2, 2024-05-11T13:57:57.018 INFO:teuthology.orchestra.run.smithi063.stdout: "segexg": 0, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "segexd": 0, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "expos": 4194304, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "wrpos": 5947651, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "rdpos": 5947077, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "jlat": { 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "replayed": 1283 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_mem": { 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "ino": 520, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "ino+": 683, 2024-05-11T13:57:57.019 INFO:teuthology.orchestra.run.smithi063.stdout: "ino-": 163, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dir": 19, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dir+": 21, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dir-": 2, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dn": 518, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dn+": 681, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dn-": 163, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "cap": 0, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "cap+": 0, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "cap-": 0, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "rss": 52496, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "heap": 215416 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_server": { 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_client_request": 0, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_server_request": 0, 2024-05-11T13:57:57.020 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_request": 0, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_session": 0, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_peer_request": 0, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "req_create_latency": { 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getattr_latency": { 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getfilelock_latency": { 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.021 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "req_link_latency": { 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookup_latency": { 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookuphash_latency": { 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupino_latency": { 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupname_latency": { 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupparent_latency": { 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.023 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupsnap_latency": { 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lssnap_latency": { 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.024 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mkdir_latency": { 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mknod_latency": { 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mksnap_latency": { 2024-05-11T13:57:57.025 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "req_open_latency": { 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "req_readdir_latency": { 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rename_latency": { 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "req_renamesnap_latency": { 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "req_snapdiff_latency": { 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmdir_latency": { 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmsnap_latency": { 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmxattr_latency": { 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setattr_latency": { 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setdirlayout_latency": { 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setfilelock_latency": { 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.029 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setlayout_latency": { 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setxattr_latency": { 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "req_symlink_latency": { 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.030 INFO:teuthology.orchestra.run.smithi063.stdout: "req_unlink_latency": { 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getvxattr_latency": { 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sessions": { 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "session_count": 2, 2024-05-11T13:57:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: "session_add": 4, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "session_remove": 2, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_open": 3, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_stale": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "total_load": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "average_load": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "avg_session_uptime": 3851, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "mdthresh_evicted": 0 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:57:57.032 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:57:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:57:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 199065, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 59, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 9608, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 184, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-11T13:57:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 1489718, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 4392, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter": { 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "op_active": 0, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "op_laggy": 0, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send": 5087, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send_bytes": 21, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "op_resend": 3, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "op_reply": 5087, 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "op_latency": { 2024-05-11T13:57:57.036 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 5087, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 9.800905910, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.001926657 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "op_inflight": 0, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "oplen_avg": { 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 5087, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 5088 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "op": 5087, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "op_r": 5087, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "op_w": 0, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "op_rmw": 0, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "op_pg": 0, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_stat": 3239, 2024-05-11T13:57:57.037 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_create": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_read": 1846, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_write": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writefull": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writesame": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_append": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_zero": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_truncate": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_delete": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_mapext": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_sparse_read": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_clonerange": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_getxattr": 1, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_setxattr": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_rmxattr": 0, 2024-05-11T13:57:57.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_call": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_watch": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_notify": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_other": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_active": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_send": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_resend": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_ping": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_active": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_send": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_resend": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_active": 0, 2024-05-11T13:57:57.039 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_send": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_resend": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_active": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_send": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_resend": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "command_active": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "command_send": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "command_resend": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "map_epoch": 127, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "map_full": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "map_inc": 39, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_sessions": 6, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_open": 8, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_close": 2, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_laggy": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_wr": 0, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_rd": 2, 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_del": 0 2024-05-11T13:57:57.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "oft": { 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_objs": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_updates": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_removes": 0 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "osd-slow-ops": { 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_ops_count": 0 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "purge_queue": { 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops_high_water": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_high_water": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed_ops": 0, 2024-05-11T13:57:57.041 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed": 0, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_item_in_journal": 0 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 5671, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 4701502, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 5671, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 5671, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 4701502, 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:57.042 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_bytes": { 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 5087, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 1753352, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 5087, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 1753352, 2024-05-11T13:57:57.043 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_ops": { 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 1024, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 5087, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 5087, 2024-05-11T13:57:57.044 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 5087, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 5087, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:57:57.045 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle-0x561075646780": { 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-05-11T13:57:57.046 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:57:57.047 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:57:57.861 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-11T13:57:57.861 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-11T13:57:57.861 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-11T13:57:57.861 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-11T13:57:57.861 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-11T13:57:57.861 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:57:57.861 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:57:57.861 DEBUG:tasks.check_counter:Getting stats from j 2024-05-11T13:57:57.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-11T13:57:58.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:57 smithi063 ceph-mon[29809]: pgmap v761: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2024-05-11T13:57:58.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:57 smithi071 ceph-mon[33293]: pgmap v761: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2024-05-11T13:57:58.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:57 smithi165 ceph-mon[33235]: pgmap v761: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2024-05-11T13:57:58.352 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:58:00.131 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:57:59 smithi063 ceph-mon[29809]: pgmap v762: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-05-11T13:58:00.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:57:59 smithi071 ceph-mon[33293]: pgmap v762: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-05-11T13:58:00.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:57:59 smithi165 ceph-mon[33235]: pgmap v762: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-05-11T13:58:00.809 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-11T13:58:00.809 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:00.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 4, 2024-05-11T13:58:00.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 4, 2024-05-11T13:58:00.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 2944, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 448, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 0, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.006372906, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000080106, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.001334413, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.001877286, 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000625762 2024-05-11T13:58:00.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 448 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 493, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 483, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1639168, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 299008, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 2, 2024-05-11T13:58:00.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.180290476, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.075405217, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.098332911, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.022111410, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 482, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.047113396, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000097745 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:58:00.812 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 1639168, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 299008 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 3, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 369, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 480, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 301104, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 6, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 3, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.069859990, 2024-05-11T13:58:00.813 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.051882638, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.007415903, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 366, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.055097446, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000150539 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 301104 2024-05-11T13:58:00.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:00.815 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:00.816 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:00.817 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 148714, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 30, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:00.818 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 928, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 2, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 496, 2024-05-11T13:58:00.819 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 1569497, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 496, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 496, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 1569497, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-11T13:58:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-11T13:58:01.713 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-11T13:58:01.713 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-11T13:58:01.713 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-11T13:58:01.714 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-11T13:58:01.714 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-11T13:58:01.714 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-11T13:58:01.714 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-11T13:58:01.714 DEBUG:tasks.check_counter:Getting stats from b 2024-05-11T13:58:01.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-11T13:58:01.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:01 smithi063 ceph-mon[29809]: pgmap v763: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-11T13:58:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:01 smithi071 ceph-mon[33293]: pgmap v763: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-11T13:58:02.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:01 smithi165 ceph-mon[33235]: pgmap v763: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-11T13:58:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:58:04.100 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:03 smithi071 ceph-mon[33293]: pgmap v764: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-11T13:58:04.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:03 smithi063 ceph-mon[29809]: pgmap v764: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-11T13:58:04.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:03 smithi165 ceph-mon[33235]: pgmap v764: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 66416, 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 123946, 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 17415408, 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 61295194, 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 18, 2024-05-11T13:58:06.020 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 14.005898225, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 8.205848570, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 6.492993642, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.143330327, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 123944, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 9.761770072, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000078759 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 174687, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.063655103, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000364 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.021 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2206544, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 934048 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 115050, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 133465, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 36588280, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 92858825, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 13, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 14.560829903, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 5.922398783, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 9.035448632, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 1.031180183, 2024-05-11T13:58:06.022 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 133464, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 12.134856485, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000090922 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 236703, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.076028354, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000321 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 783824, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 465888 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 16396, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 20084, 2024-05-11T13:58:06.023 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 5000597, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 40286489, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 84, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 3.859829581, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 1.950365517, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 1.775091649, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.443608609, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 20082, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.967918964, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000147790 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.024 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 28532, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.017717041, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000620 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 1376 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7439, 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 7.215898859, 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000970009 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 825, 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.084010517, 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000101830 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-11T13:58:06.026 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 47366, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 46220, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 46220, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 200.098455060, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004329261 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 1146, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 46, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 4, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 341, 2024-05-11T13:58:06.027 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 57, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 266, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 18, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 240, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 1233, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 64, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 64, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 41, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 91856, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 89063, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 1128, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 18, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:06.028 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 36, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 342, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 29, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 1256, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 28, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 793, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 1001, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 1218352994, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:06.029 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 41, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 38119, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 30736, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 57637, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 6475, 2024-05-11T13:58:06.030 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 57, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 1619 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 93, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 102, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 59, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 68, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 641, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 277, 2024-05-11T13:58:06.031 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 222, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 55, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 17, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 4180, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 32, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 4, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:06.032 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 10133, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 10133, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 27, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 27, 2024-05-11T13:58:06.033 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 104566799, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3631, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 22.357999382, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006157532 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 215, 2024-05-11T13:58:06.034 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 1577, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 1362, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 113, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 178, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 65, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 341, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 1858, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 1517, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 64, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 927, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 863, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 73192, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 231800 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-11T13:58:06.035 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 77430, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 1536, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 47366, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 171, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 4533, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 533, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 96.453867154, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.180964103 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 40989, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 54.420418394, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001327683 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:06.036 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 55, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.263474994, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004790454 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2966, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 25.897971389, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.008731615 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:06.037 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.038 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-11T13:58:06.039 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 25, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.012773699, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000510947 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 32, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 19.021799467, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.594431233 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.040 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 25, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.436414105, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.017456564 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1194, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.993956896, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000832459 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 32, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.070405551, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002200173 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-11T13:58:06.041 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 277, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.917729312, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006923210 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-11T13:58:06.042 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 19, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.285329125, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.015017322 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.043 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.003662610, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001831305 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-11T13:58:06.044 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.055350652, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006918831 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 63, 2024-05-11T13:58:06.045 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.265301712, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004211138 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 6, 2024-05-11T13:58:06.046 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 4, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 5767, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 2883, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 1449, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:06.047 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:06.048 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 575975, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 175, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9608, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 184, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 1224006, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 24440, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 5351, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 100898654, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 37, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 5351, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5351, 2024-05-11T13:58:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 52.735077999, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.009855181 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5351, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 5739 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 5351, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 104, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 5247, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 11, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 104, 2024-05-11T13:58:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 3735, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 176, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 2, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 981, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 22, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-11T13:58:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-11T13:58:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 127, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 48, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 559, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 149 2024-05-11T13:58:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 5271 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 10, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-11T13:58:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 5, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 444, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 222, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 197858, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 41063531, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 197858, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 197858, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 41063531, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 5351, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 100923516, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5351, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 100923516, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 5351, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 5351, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5351, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 5351, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 222, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 24862, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 222, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 104, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 24862, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55b205a46500": { 2024-05-11T13:58:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 10133, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 100372495, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 10133, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 3631, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 100372495, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:06.063 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-11T13:58:06.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:05 smithi071 ceph-mon[33293]: pgmap v765: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-05-11T13:58:06.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:05 smithi063 ceph-mon[29809]: pgmap v765: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-05-11T13:58:06.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:05 smithi165 ceph-mon[33235]: pgmap v765: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-05-11T13:58:07.079 INFO:tasks.check_counter:Daemon mds.b mds.exported=29 2024-05-11T13:58:07.080 INFO:tasks.check_counter:Daemon mds.b mds.imported=28 2024-05-11T13:58:07.080 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=93 2024-05-11T13:58:07.080 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=102 2024-05-11T13:58:07.080 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-11T13:58:07.080 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=25 2024-05-11T13:58:07.080 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2024-05-11T13:58:07.080 DEBUG:tasks.check_counter:Getting stats from e 2024-05-11T13:58:07.080 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-11T13:58:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:58:07.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:07 smithi071 ceph-mon[33293]: pgmap v766: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-05-11T13:58:08.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:07 smithi063 ceph-mon[29809]: pgmap v766: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-05-11T13:58:08.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:07 smithi165 ceph-mon[33235]: pgmap v766: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-05-11T13:58:09.844 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-11T13:58:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 124036, 2024-05-11T13:58:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 67540, 2024-05-11T13:58:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 35388619, 2024-05-11T13:58:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 33467766, 2024-05-11T13:58:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 11, 2024-05-11T13:58:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-05-11T13:58:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 13.798204115, 2024-05-11T13:58:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 7.771005661, 2024-05-11T13:58:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 8.419079709, 2024-05-11T13:58:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.100083444, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 67543, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.467131514, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000066137 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 188281, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.037155339, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000197 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-11T13:58:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 579328 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 781936, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 742605, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 232429209, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 234955488, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 16, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 81.955134848, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 42.119909459, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 52.497805251, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.065431254, 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 742606, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 39.063192049, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000052602 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1396318, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.351545730, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000251 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2087136, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 314320 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 258577, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 131031, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 70650424, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 37096353, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 14, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 9, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 26.148777771, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 14.777073215, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 16.610766253, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.031146346, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 131022, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 6.833860400, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000052158 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 387965, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.065945497, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000169 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 143264 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3436, 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.686065194, 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001363814 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 153, 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.019765938, 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000129189 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 3963, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 3698, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3698, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 8.162778034, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002207349 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 265, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 5, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 61, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 6, 2024-05-11T13:58:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 895, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 210, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 633, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 52, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 595, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 2717, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 291, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 291, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 49, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 6092, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 4899, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 208, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 24, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 6, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 148, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 67, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 2634, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 45, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 2314, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 1005, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 1252128184, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 30, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 229112, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 229522, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 233460, 2024-05-11T13:58:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 1523, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 11, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 874 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 144, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 296, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 191, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 260, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 236, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 2, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 56, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 35, 2024-05-11T13:58:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 21, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 17, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 7063, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 69, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 6, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-11T13:58:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 5255, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 5255, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 14, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 14, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-11T13:58:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 42649815, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1624, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 12.584939156, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007749346 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 863, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 3481, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 2618, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 126, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 219, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 93, 2024-05-11T13:58:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 895, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 3725, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 2830, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 291, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 1708, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 1417, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 73784, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 223608 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 4321, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 1088, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 3963, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 139, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 2461, 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-11T13:58:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 675, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.457501841, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003640743 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 138, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.339907044, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002463094 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 21, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.171956111, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.008188386 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1541, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.154266588, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001397966 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-11T13:58:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5, 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.020553116, 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004110623 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1076, 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.698032368, 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001578096 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-11T13:58:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 162, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.463932427, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002863780 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 51, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.662933543, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.012998696 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-11T13:58:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 29, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.193694996, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006679137 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 4, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 3, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 978, 2024-05-11T13:58:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 978, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 1461, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 324169, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 112, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-11T13:58:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9608, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 184, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 3031367, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 15367, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 2083, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 39034683, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 33, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 2083, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2083, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 12.797779514, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006143917 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-11T13:58:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2083, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2336 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 2083, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 23, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 2060, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 18, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 1642, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 96, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-11T13:58:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 2, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 2, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 97, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-11T13:58:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-11T13:58:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 127, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 39, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 354, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 10, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 85 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-11T13:58:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 7802 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 12, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 5, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 74, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 35, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 1164545, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 233623923, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 1164545, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1164545, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 233623923, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2083, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 39038746, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2083, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 39038746, 2024-05-11T13:58:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2083, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2083, 2024-05-11T13:58:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2083, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2083, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 35, 2024-05-11T13:58:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 4143, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 35, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 18, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 4143, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55b122556500": { 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 5255, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 38455511, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 5255, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1624, 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 38455511, 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:09.889 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-11T13:58:10.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:09 smithi063 ceph-mon[29809]: pgmap v767: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 KiB/s wr, 29 op/s 2024-05-11T13:58:10.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:09 smithi071 ceph-mon[33293]: pgmap v767: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 KiB/s wr, 29 op/s 2024-05-11T13:58:10.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:09 smithi165 ceph-mon[33235]: pgmap v767: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 KiB/s wr, 29 op/s 2024-05-11T13:58:11.161 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:10 smithi071 ceph-mon[33293]: osdmap e128: 12 total, 12 up, 12 in 2024-05-11T13:58:11.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:10 smithi063 ceph-mon[29809]: osdmap e128: 12 total, 12 up, 12 in 2024-05-11T13:58:11.227 INFO:tasks.check_counter:Daemon mds.e mds.exported=67 2024-05-11T13:58:11.227 INFO:tasks.check_counter:Daemon mds.e mds.imported=45 2024-05-11T13:58:11.227 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=144 2024-05-11T13:58:11.227 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=296 2024-05-11T13:58:11.228 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-11T13:58:11.228 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:58:11.228 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:58:11.228 DEBUG:tasks.check_counter:Getting stats from h 2024-05-11T13:58:11.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-11T13:58:11.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:10 smithi165 ceph-mon[33235]: osdmap e128: 12 total, 12 up, 12 in 2024-05-11T13:58:11.735 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:58:12.171 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:11 smithi063 ceph-mon[29809]: pgmap v769: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:58:12.190 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:11 smithi071 ceph-mon[33293]: pgmap v769: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:58:12.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:11 smithi165 ceph-mon[33235]: pgmap v769: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:58:13.177 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:12 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:58:13.204 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:12 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:58:13.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:12 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:58:14.191 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:13 smithi071 ceph-mon[33293]: pgmap v770: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:58:14.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:13 smithi165 ceph-mon[33235]: pgmap v770: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:58:14.414 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-11T13:58:14.414 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:14.414 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 275286, 2024-05-11T13:58:14.414 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 544690, 2024-05-11T13:58:14.414 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 57375674, 2024-05-11T13:58:14.414 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 153810567, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 17, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 54.100722560, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 30.546573879, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 26.692109964, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.317630118, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 544685, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 31.281381974, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000057430 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:14.415 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 758508, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.220446856, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000290 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 259328 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 335717, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 470261, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 103214078, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 213875200, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 11, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 40.270702964, 2024-05-11T13:58:14.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 18.080396718, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 25.023820162, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.358906956, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 470263, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 29.598928548, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000062941 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 778676, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.232107245, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000298 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-11T13:58:14.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 142096 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 936, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1060, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2182011, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 971329, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 11, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 9, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.282362367, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.111925037, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.153613817, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.038221030, 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:14.418 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1058, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.156847364, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000148248 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000571, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000285 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2094064, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 691952 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-11T13:58:14.419 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2227, 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.829168610, 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000821359 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:14.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 89, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.008158979, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000091673 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 170207, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 169168, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 169168, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 389.502152788, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002302457 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.421 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 1039, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 28, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 2, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 630, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 39, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 513, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 78, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 574, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 808, 2024-05-11T13:58:14.422 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 243, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 243, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 51, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 340249, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 337893, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 1039, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 41, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 84, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 25, 2024-05-11T13:58:14.423 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 767, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 29, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 1083, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 1005, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 1252128184, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:14.424 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 3, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 152249, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 136738, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 155055, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 1834, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 23, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 233 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-11T13:58:14.425 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 265, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 155, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 88, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 130, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 241, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 1, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 67, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 10, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 56, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:14.426 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 17, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 1445, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 28, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 2, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 3278, 2024-05-11T13:58:14.427 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 3278, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 5, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 5, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 9634236, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-11T13:58:14.428 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 948, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 5.492771338, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005794062 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 570, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 1306, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 736, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 126, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 172, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 46, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 630, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 1462, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 832, 2024-05-11T13:58:14.429 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 243, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 553, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 310, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 63484, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 223608 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 307553, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 79, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 170207, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 43, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 1269, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 59, 2024-05-11T13:58:14.430 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.396016292, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006712140 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 167123, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 214.052094653, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001280805 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 56, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.447371410, 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007988775 2024-05-11T13:58:14.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1482, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 172.107988316, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.116132245 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.432 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.434 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:13 smithi063 ceph-mon[29809]: pgmap v770: 129 pgs: 129 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:58:14.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.434 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-11T13:58:14.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.434 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 298, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.573294577, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001923807 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 24, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.054372787, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002265532 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 64, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.424011957, 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.022250186 2024-05-11T13:58:14.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.036236776, 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.018118388 2024-05-11T13:58:14.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 60, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.410766020, 2024-05-11T13:58:14.440 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006846100 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 2, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 1, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-05-11T13:58:14.441 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 5791, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 5791, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 1463, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:14.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:14.443 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 385335, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 130, 2024-05-11T13:58:14.444 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9848, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 190, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 2024864, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 12352, 2024-05-11T13:58:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 1390, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 5674599, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 33, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 1390, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1390, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 8.426850439, 2024-05-11T13:58:14.446 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006062482 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1390, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1689 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 1390, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 7, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 1383, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 7, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 955, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 90, 2024-05-11T13:58:14.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 1, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 63, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:14.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-11T13:58:14.449 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 128, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 40, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 437, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 106 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.450 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 1516 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 23, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 4, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 49, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 10, 2024-05-11T13:58:14.451 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 611935, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 107662599, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 611935, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 611935, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 107662599, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.452 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 1390, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 5675697, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1390, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 5675697, 2024-05-11T13:58:14.453 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:14.454 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 1390, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 1390, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1390, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1390, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 10, 2024-05-11T13:58:14.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1098, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 10, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 7, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1098, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55983ea00500": { 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:14.456 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 3278, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 5439932, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 3278, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 948, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 5439932, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:14.457 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-11T13:58:15.557 INFO:tasks.check_counter:Daemon mds.h mds.exported=25 2024-05-11T13:58:15.557 INFO:tasks.check_counter:Daemon mds.h mds.imported=29 2024-05-11T13:58:15.557 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=265 2024-05-11T13:58:15.557 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=155 2024-05-11T13:58:15.557 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-11T13:58:15.557 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:58:15.557 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:58:15.557 DEBUG:tasks.check_counter:Getting stats from k 2024-05-11T13:58:15.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-11T13:58:15.991 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T13:58:16.679 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:16 smithi165 ceph-mon[33235]: pgmap v771: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-11T13:58:16.679 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:16 smithi165 ceph-mon[33235]: osdmap e129: 12 total, 12 up, 12 in 2024-05-11T13:58:16.719 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:16 smithi071 ceph-mon[33293]: pgmap v771: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-11T13:58:16.719 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:16 smithi071 ceph-mon[33293]: osdmap e129: 12 total, 12 up, 12 in 2024-05-11T13:58:16.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:16 smithi063 ceph-mon[29809]: pgmap v771: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-11T13:58:16.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:16 smithi063 ceph-mon[29809]: osdmap e129: 12 total, 12 up, 12 in 2024-05-11T13:58:18.498 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:18 smithi071 ceph-mon[33293]: pgmap v773: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-11T13:58:18.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:18 smithi165 ceph-mon[33235]: pgmap v773: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-11T13:58:18.921 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:18 smithi063 ceph-mon[29809]: pgmap v773: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-11T13:58:19.638 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-11T13:58:19.638 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:19.638 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 138944, 2024-05-11T13:58:19.639 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 265140, 2024-05-11T13:58:19.639 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 32591770, 2024-05-11T13:58:19.639 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 75046640, 2024-05-11T13:58:19.639 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 10, 2024-05-11T13:58:19.639 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-05-11T13:58:19.639 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 28.334073565, 2024-05-11T13:58:19.640 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 16.504022060, 2024-05-11T13:58:19.640 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 13.135989568, 2024-05-11T13:58:19.640 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.385498543, 2024-05-11T13:58:19.640 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:19.640 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 265141, 2024-05-11T13:58:19.641 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 20.109553241, 2024-05-11T13:58:19.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000075844 2024-05-11T13:58:19.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.641 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:19.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 372945, 2024-05-11T13:58:19.641 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.135492445, 2024-05-11T13:58:19.642 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000363 2024-05-11T13:58:19.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.642 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2084448, 2024-05-11T13:58:19.642 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 316384 2024-05-11T13:58:19.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.643 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:19.643 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 33, 2024-05-11T13:58:19.643 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 406, 2024-05-11T13:58:19.643 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 13969, 2024-05-11T13:58:19.643 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 846517, 2024-05-11T13:58:19.643 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 14, 2024-05-11T13:58:19.644 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-05-11T13:58:19.644 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.055739780, 2024-05-11T13:58:19.644 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.031806452, 2024-05-11T13:58:19.644 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.013298560, 2024-05-11T13:58:19.644 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000728267, 2024-05-11T13:58:19.644 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:19.645 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 402, 2024-05-11T13:58:19.645 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.068032443, 2024-05-11T13:58:19.645 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000169234 2024-05-11T13:58:19.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.645 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:19.646 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 11, 2024-05-11T13:58:19.646 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000003273, 2024-05-11T13:58:19.646 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000297 2024-05-11T13:58:19.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.646 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 4352, 2024-05-11T13:58:19.646 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 813840 2024-05-11T13:58:19.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.647 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:19.647 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 158883, 2024-05-11T13:58:19.647 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 223641, 2024-05-11T13:58:19.647 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 49148669, 2024-05-11T13:58:19.647 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 102918151, 2024-05-11T13:58:19.648 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 13, 2024-05-11T13:58:19.648 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-05-11T13:58:19.648 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 17.946896459, 2024-05-11T13:58:19.648 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 7.867267098, 2024-05-11T13:58:19.648 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 11.433840698, 2024-05-11T13:58:19.649 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.003669528, 2024-05-11T13:58:19.649 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:19.649 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 223639, 2024-05-11T13:58:19.649 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 16.609512991, 2024-05-11T13:58:19.649 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000074269 2024-05-11T13:58:19.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.650 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:19.650 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 370181, 2024-05-11T13:58:19.650 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.106169267, 2024-05-11T13:58:19.650 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000286 2024-05-11T13:58:19.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.651 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-11T13:58:19.651 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-05-11T13:58:19.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.651 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-11T13:58:19.651 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-11T13:58:19.651 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-11T13:58:19.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.652 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-11T13:58:19.652 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:19.652 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:19.652 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1040, 2024-05-11T13:58:19.652 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.615156484, 2024-05-11T13:58:19.653 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000591496 2024-05-11T13:58:19.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.653 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:19.653 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-11T13:58:19.653 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-11T13:58:19.654 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 31, 2024-05-11T13:58:19.654 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002023919, 2024-05-11T13:58:19.654 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000065287 2024-05-11T13:58:19.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.655 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-11T13:58:19.655 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 82708, 2024-05-11T13:58:19.655 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 82555, 2024-05-11T13:58:19.655 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-11T13:58:19.655 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 82555, 2024-05-11T13:58:19.655 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 124.617858514, 2024-05-11T13:58:19.656 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001509513 2024-05-11T13:58:19.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.656 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-11T13:58:19.656 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 153, 2024-05-11T13:58:19.656 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 1, 2024-05-11T13:58:19.657 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:19.657 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 11, 2024-05-11T13:58:19.657 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-05-11T13:58:19.657 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-11T13:58:19.657 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 573, 2024-05-11T13:58:19.657 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 9, 2024-05-11T13:58:19.658 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 548, 2024-05-11T13:58:19.658 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 16, 2024-05-11T13:58:19.658 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 557, 2024-05-11T13:58:19.658 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 712, 2024-05-11T13:58:19.658 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 256, 2024-05-11T13:58:19.659 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 256, 2024-05-11T13:58:19.659 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 38, 2024-05-11T13:58:19.659 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 165314, 2024-05-11T13:58:19.659 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 165078, 2024-05-11T13:58:19.659 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 153, 2024-05-11T13:58:19.660 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 28, 2024-05-11T13:58:19.660 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:19.660 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:19.660 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-05-11T13:58:19.660 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 84, 2024-05-11T13:58:19.660 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-11T13:58:19.661 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 11, 2024-05-11T13:58:19.661 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 696, 2024-05-11T13:58:19.661 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 20, 2024-05-11T13:58:19.661 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 991, 2024-05-11T13:58:19.661 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:19.662 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:19.662 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:19.662 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 1005, 2024-05-11T13:58:19.662 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 1252128184, 2024-05-11T13:58:19.662 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-05-11T13:58:19.662 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 3, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 74417, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 64519, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:19.663 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 75288, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 293, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 3, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 12 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 17, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 417, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 253, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 352, 2024-05-11T13:58:19.664 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 18, 2024-05-11T13:58:19.665 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-05-11T13:58:19.665 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:19.665 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:19.665 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 4, 2024-05-11T13:58:19.665 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 4, 2024-05-11T13:58:19.665 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-05-11T13:58:19.665 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 17, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 3586, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 13, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-11T13:58:19.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 1227, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 1227, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 3, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 3, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-05-11T13:58:19.667 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 6325174, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 436, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.217961570, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005087067 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 554, 2024-05-11T13:58:19.668 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 1252, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 698, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 105, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 232, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 127, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 573, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 1289, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 716, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 256, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 555, 2024-05-11T13:58:19.669 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 299, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 62360, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 223608 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 146884, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 586, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 82708, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 32, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 1551, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.024569064, 2024-05-11T13:58:19.670 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.008189688 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 82082, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 110.039041739, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001340598 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.671 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 209, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 14.166206879, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.067780894 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-11T13:58:19.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.673 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:19.674 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.674 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.674 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 239, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.329549269, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001378867 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 17, 2024-05-11T13:58:19.676 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.032536497, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001913911 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.677 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.678 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.005599114, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005599114 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.679 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.680 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.020355952, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005088988 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:19.681 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 2, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 1, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 5363, 2024-05-11T13:58:19.682 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 5363, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 1470, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:19.683 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:19.684 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 335703, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 102, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:19.685 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9608, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 184, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 1878464, 2024-05-11T13:58:19.686 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 12540, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 627, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 2160073, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 33, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 627, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 627, 2024-05-11T13:58:19.687 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 3.505938215, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005591607 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 627, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 724 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 627, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 4, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 623, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-05-11T13:58:19.688 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 3, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 439, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 73, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 20, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-11T13:58:19.690 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 129, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 41, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 9, 2024-05-11T13:58:19.691 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 9, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 127, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 30 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 4188 2024-05-11T13:58:19.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 4, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 2, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 8, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 4, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 297856, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 54912078, 2024-05-11T13:58:19.693 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 297856, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 297856, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 54912078, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.694 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 627, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2160685, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 627, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2160685, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.695 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 627, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 627, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 627, 2024-05-11T13:58:19.696 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 627, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 4, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 628, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 4, 2024-05-11T13:58:19.697 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 3, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 628, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x556a92590500": { 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-11T13:58:19.698 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 1227, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 2130870, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 1227, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 436, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2130870, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-11T13:58:19.700 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-11T13:58:20.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:20 smithi063 ceph-mon[29809]: pgmap v774: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 233 B/s wr, 11 op/s 2024-05-11T13:58:20.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:58:20.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:58:20.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.440 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:20 smithi071 ceph-mon[33293]: pgmap v774: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 233 B/s wr, 11 op/s 2024-05-11T13:58:20.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:58:20.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:58:20.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:20 smithi165 ceph-mon[33235]: pgmap v774: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 233 B/s wr, 11 op/s 2024-05-11T13:58:20.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:58:20.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:58:20.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:58:20.637 INFO:tasks.check_counter:Daemon mds.k mds.exported=11 2024-05-11T13:58:20.637 INFO:tasks.check_counter:Daemon mds.k mds.imported=20 2024-05-11T13:58:20.637 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=17 2024-05-11T13:58:20.637 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=417 2024-05-11T13:58:20.637 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-05-11T13:58:20.638 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:58:20.638 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:58:20.638 DEBUG:tasks.check_counter:Getting stats from c 2024-05-11T13:58:20.638 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-11T13:58:21.095 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:58:22.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:22 smithi063 ceph-mon[29809]: pgmap v775: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-11T13:58:22.440 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:22 smithi071 ceph-mon[33293]: pgmap v775: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-11T13:58:22.518 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:22 smithi165 ceph-mon[33235]: pgmap v775: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-11T13:58:23.404 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-11T13:58:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1827, 2024-05-11T13:58:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1762, 2024-05-11T13:58:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 27600132, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 699315, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.564014718, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.194005735, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.261354722, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.102248370, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1761, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.189914921, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000107844 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2064352, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 307856 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3960, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4327, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 47173826, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1981825, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.982101893, 2024-05-11T13:58:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.301901275, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.362049809, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.231695685, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4324, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.421278031, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000097427 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 723120 2024-05-11T13:58:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 576, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 576, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 29513700, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 182359, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.199018105, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.045445029, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.109320629, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.032884003, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 575, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.059356151, 2024-05-11T13:58:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000103228 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-11T13:58:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 9558, 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.508054354, 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000053154 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-05-11T13:58:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 526, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 44, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 482, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 463, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 12, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 1001, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 1218352994, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-05-11T13:58:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-11T13:58:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-11T13:58:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 10134, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 27, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-11T13:58:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 104566935, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 104566799, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 10134 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 176, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 1561, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 1385, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 30, 2024-05-11T13:58:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 104, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 74, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 526, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 1826, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 1300, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 55028, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-05-11T13:58:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-11T13:58:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 6, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 4, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 4, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 4774, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 232953, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 79, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9608, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 184, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 1066846, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 22394, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-11T13:58:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 5761, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 26, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 4, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 5761, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5761, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 9.004865756, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001563073 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5761, 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5762 2024-05-11T13:58:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 5761, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 5761, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 3400, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 2358, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-05-11T13:58:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 2, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-11T13:58:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 129, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 41, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 10, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 11, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 1, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 6359, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 103682045, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 6359, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6359, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 103682045, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5761, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 100618388, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5761, 2024-05-11T13:58:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 100618388, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5761, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 5761, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5761, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5761, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55bc06ef6780": { 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:23.448 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-11T13:58:24.288 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:24 smithi165 ceph-mon[33235]: pgmap v776: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-11T13:58:24.363 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-11T13:58:24.363 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-11T13:58:24.363 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-11T13:58:24.363 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-11T13:58:24.363 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-11T13:58:24.363 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:58:24.363 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:58:24.364 DEBUG:tasks.check_counter:Getting stats from f 2024-05-11T13:58:24.364 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-11T13:58:24.440 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:24 smithi071 ceph-mon[33293]: pgmap v776: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-11T13:58:24.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:24 smithi063 ceph-mon[29809]: pgmap v776: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-11T13:58:24.814 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:58:26.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:26 smithi071 ceph-mon[33293]: pgmap v777: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-11T13:58:26.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:26 smithi165 ceph-mon[33235]: pgmap v777: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-11T13:58:26.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:26 smithi063 ceph-mon[29809]: pgmap v777: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-11T13:58:27.106 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2299, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2234, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2636608, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 850212, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 5, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 1, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.585728070, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.207047471, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.231625006, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.128354264, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2233, 2024-05-11T13:58:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.243467394, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000109031 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2064736, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 308976 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 4, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 372, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 876, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 723994, 2024-05-11T13:58:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.046083960, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.031858919, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.004495711, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000326235, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 369, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.058604595, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000158820 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 723296 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3478, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3478, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 6325581, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1105187, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.624133072, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.224486239, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.171742844, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.184164844, 2024-05-11T13:58:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3477, 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.301196608, 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000086625 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-11T13:58:27.111 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8738, 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.448430034, 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000051319 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.114 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-11T13:58:27.114 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-05-11T13:58:27.114 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-11T13:58:27.114 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:27.114 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-05-11T13:58:27.114 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 629, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 31, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 598, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 584, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 20, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:27.115 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:27.116 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-05-11T13:58:27.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-05-11T13:58:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 3278, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 5, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 9634236, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 9634175, 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-11T13:58:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 3278 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 557, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 1294, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 737, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 34, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 68, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 34, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 629, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 1411, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 782, 2024-05-11T13:58:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 53780, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-05-11T13:58:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-11T13:58:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 2, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 4775, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-11T13:58:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 182081, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 48, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9608, 2024-05-11T13:58:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 184, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 1695084, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 5305, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-11T13:58:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 5178, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 5178, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5178, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 6.996895655, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001351273 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5178, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5179 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 5178, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 5178, 2024-05-11T13:58:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 3175, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 2001, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-11T13:58:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-11T13:58:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 129, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 41, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 4, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 4, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-05-11T13:58:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-05-11T13:58:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 5777, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 8409774, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 5777, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5777, 2024-05-11T13:58:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 8409774, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5178, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 5439937, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5178, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5439937, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5178, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 5178, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5178, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5178, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55973596a780": { 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:27.151 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-11T13:58:28.045 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-11T13:58:28.046 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-11T13:58:28.046 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-11T13:58:28.046 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-11T13:58:28.046 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-05-11T13:58:28.046 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:58:28.046 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:58:28.046 DEBUG:tasks.check_counter:Getting stats from i 2024-05-11T13:58:28.046 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-11T13:58:28.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:28 smithi165 ceph-mon[33235]: pgmap v778: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-05-11T13:58:28.440 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:28 smithi071 ceph-mon[33293]: pgmap v778: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-05-11T13:58:28.497 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:58:28.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:28 smithi063 ceph-mon[29809]: pgmap v778: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-05-11T13:58:30.451 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:30 smithi165 ceph-mon[33235]: pgmap v779: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-11T13:58:30.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:30 smithi063 ceph-mon[29809]: pgmap v779: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-11T13:58:30.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:30 smithi071 ceph-mon[33293]: pgmap v779: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-11T13:58:30.812 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3367, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2937, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 4308208, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 3598370, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 10, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.725555107, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.300372467, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.441552066, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.059280318, 2024-05-11T13:58:30.813 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2937, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.358276775, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000121987 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3408, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001508395, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000442 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2079872, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 933072 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:30.814 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 7355, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 5866, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2558588, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 2445002, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 15, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 1.203062041, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.521569188, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.882677632, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.014434522, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5867, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.458259123, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000078107 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.815 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 12123, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.005928627, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000489 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 7792, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 174000 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2779, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2824, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1173878, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1915730, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 18, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-05-11T13:58:30.816 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.549030218, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.237371981, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.375749701, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.003678280, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2818, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.276904039, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000098262 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4989, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.003107910, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000622 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.817 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 928, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 736 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1106, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.806683375, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000729370 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.818 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000019065, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000009532 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 753, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 546, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 546, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.319942638, 2024-05-11T13:58:30.819 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002417477 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 207, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 547, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 36, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 511, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 494, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 176, 2024-05-11T13:58:30.820 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 225, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 225, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 32, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 1259, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 998, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 205, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 32, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 7, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 2, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 162, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 12, 2024-05-11T13:58:30.821 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 334, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 1005, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 1252128184, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:30.822 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 4, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 686, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 276, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 1398, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 111, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 1, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 1 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 17, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 401, 2024-05-11T13:58:30.823 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 244, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 343, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 17, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-11T13:58:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 17, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 3972, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 2, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 1283, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 1283, 2024-05-11T13:58:30.825 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 3, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 3, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 5947651, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 494, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.889710386, 2024-05-11T13:58:30.826 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.007873907 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 553, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 722, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 169, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 106, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 233, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 127, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 547, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 726, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 179, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 225, 2024-05-11T13:58:30.827 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 309, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 84, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 59308, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 223608 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 818, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 392, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 753, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 44, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 1168, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.014577691, 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.014577691 2024-05-11T13:58:30.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 12, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.314369515, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.026197459 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 192, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.470265328, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002449298 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.832 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 312, 2024-05-11T13:58:30.833 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.456617364, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001463517 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 29, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.064112740, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002210784 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:30.834 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:30.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:30.836 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-11T13:58:30.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.838 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 18, 2024-05-11T13:58:30.839 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 9, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1476, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:30.840 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:30.841 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 296023, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 101, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:30.842 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9608, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 184, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 1855236, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 9210, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-11T13:58:30.843 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 641, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 1776823, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 32, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 641, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 641, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4.997925604, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.007797075 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.844 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 641, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 721 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 641, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 0, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 641, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 0, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 494, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 60, 2024-05-11T13:58:30.845 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 6, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-11T13:58:30.846 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-11T13:58:30.847 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 129, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 41, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 10, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 14, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 4, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 105, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 26 2024-05-11T13:58:30.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 4371 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-05-11T13:58:30.849 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 13497, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 6789625, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 13497, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 13497, 2024-05-11T13:58:30.850 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 6789625, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:30.851 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 641, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1776823, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 641, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1776823, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:30.852 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 641, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 641, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 641, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 641, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:30.853 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:30.854 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x556ab05b8500": { 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1283, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1753347, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1283, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 494, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1753347, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:30.855 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:30.856 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.856 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:30.856 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-11T13:58:31.935 INFO:tasks.check_counter:Daemon mds.i mds.exported=2 2024-05-11T13:58:31.935 INFO:tasks.check_counter:Daemon mds.i mds.imported=12 2024-05-11T13:58:31.935 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=17 2024-05-11T13:58:31.935 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=401 2024-05-11T13:58:31.935 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-11T13:58:31.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:58:31.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:58:31.935 DEBUG:tasks.check_counter:Getting stats from l 2024-05-11T13:58:31.936 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-11T13:58:32.382 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T13:58:32.527 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:32 smithi165 ceph-mon[33235]: pgmap v780: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-11T13:58:32.537 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:32 smithi063 ceph-mon[29809]: pgmap v780: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-11T13:58:32.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:32 smithi071 ceph-mon[33293]: pgmap v780: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-11T13:58:34.492 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:34 smithi165 ceph-mon[33235]: pgmap v781: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:58:34.497 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-11T13:58:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-11T13:58:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 726, 2024-05-11T13:58:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 661, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 6292450, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 352580, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.249725574, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.085394654, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.151218419, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.032227718, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 660, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.086447068, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000130980 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2066912, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 314096 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1618, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1993, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 17185894, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1243269, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.452151223, 2024-05-11T13:58:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.168843216, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.136740553, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.102006945, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1990, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.210812046, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000105935 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 729376 2024-05-11T13:58:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3614, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3614, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 18582683, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1148486, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.784447749, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.236379691, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.261993443, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.211144595, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3613, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.338117167, 2024-05-11T13:58:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000093583 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-11T13:58:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8825, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.476391901, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000053982 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 1018, 2024-05-11T13:58:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 149, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 869, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 805, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 49, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-11T13:58:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-11T13:58:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-05-11T13:58:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-11T13:58:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-11T13:58:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 5255, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 14, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-11T13:58:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 42649815, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 42649241, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 5255 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 929, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 3469, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 2540, 2024-05-11T13:58:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 121, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 139, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 18, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 1018, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 3647, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 2629, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 58308, 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-05-11T13:58:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-11T13:58:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-11T13:58:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 3, 2024-05-11T13:58:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 3, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 6434, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-11T13:58:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-11T13:58:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-11T13:58:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 225663, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 77, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9608, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 184, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-11T13:58:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 3073479, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 9497, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 5347, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 5347, 2024-05-11T13:58:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5347, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 9.402860100, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001758530 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5347, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5348 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 5347, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 5347, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-11T13:58:34.530 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 3311, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 2034, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-05-11T13:58:34.531 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-11T13:58:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 129, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 41, 2024-05-11T13:58:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 9, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 10, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 1, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 5954, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 41491390, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 5954, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5954, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 41491390, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5347, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 38492637, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5347, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 38492637, 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5347, 2024-05-11T13:58:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 5347, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5347, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5347, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x5590949d2780": { 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-11T13:58:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-11T13:58:34.541 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-11T13:58:34.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:34 smithi063 ceph-mon[29809]: pgmap v781: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:58:34.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:34 smithi071 ceph-mon[33293]: pgmap v781: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:58:35.509 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-11T13:58:35.509 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-11T13:58:35.509 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-11T13:58:35.509 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-11T13:58:35.509 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-11T13:58:35.509 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-11T13:58:35.509 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-11T13:58:35.509 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-11T13:58:35.520 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-11T13:58:35.520 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:58:35.520 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T13:58:35.596 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:58:35.596 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T13:58:35.656 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-11T13:58:36.130 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:58:36.492 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:36 smithi063 ceph-mon[29809]: pgmap v782: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-11T13:58:36.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:36 smithi165 ceph-mon[33235]: pgmap v782: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-11T13:58:36.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:36 smithi071 ceph-mon[33293]: pgmap v782: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-11T13:58:38.347 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:38 smithi063 ceph-mon[29809]: pgmap v783: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:58:38.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:38 smithi165 ceph-mon[33235]: pgmap v783: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:58:38.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 -- 172.21.15.63:0/3795899851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c0075780 msgr2=0x7ff5c0075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:38.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/3795899851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c0075780 0x7ff5c0075b60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000b0a0 tx=0x7ff5b000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 -- 172.21.15.63:0/3795899851 shutdown_connections 2024-05-11T13:58:38.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/3795899851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c010ff50 0x7ff5c0112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/3795899851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c0075780 0x7ff5c0075b60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/3795899851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c00760a0 0x7ff5c010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 -- 172.21.15.63:0/3795899851 >> 172.21.15.63:0/3795899851 conn(0x7ff5c007a6b0 msgr2=0x7ff5c007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:38.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 -- 172.21.15.63:0/3795899851 shutdown_connections 2024-05-11T13:58:38.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.701+0000 7ff5c6c00700 1 -- 172.21.15.63:0/3795899851 wait complete. 2024-05-11T13:58:38.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.702+0000 7ff5c6c00700 1 Processor -- start 2024-05-11T13:58:38.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.702+0000 7ff5c6c00700 1 -- start start 2024-05-11T13:58:38.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5c6c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c0075780 0x7ff5c01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:38.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5c6c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00760a0 0x7ff5c01a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:38.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5c6c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c010ff50 0x7ff5c019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:38.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5c6c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01157d0 con 0x7ff5c00760a0 2024-05-11T13:58:38.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5c6c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0115650 con 0x7ff5c010ff50 2024-05-11T13:58:38.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5c6c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0115950 con 0x7ff5c0075780 2024-05-11T13:58:38.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5bf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00760a0 0x7ff5c01a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:38.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5bf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00760a0 0x7ff5c01a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50600/0 (socket says 172.21.15.63:50600) 2024-05-11T13:58:38.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5bf400700 1 -- 172.21.15.63:0/537075272 learned_addr learned my addr 172.21.15.63:0/537075272 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:58:38.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.703+0000 7ff5bfe00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c0075780 0x7ff5c01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:38.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:38 smithi071 ceph-mon[33293]: pgmap v783: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-11T13:58:38.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5c4c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c010ff50 0x7ff5c019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:38.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5bf400700 1 -- 172.21.15.63:0/537075272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c0075780 msgr2=0x7ff5c01a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:38.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5bf400700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c0075780 0x7ff5c01a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5bf400700 1 -- 172.21.15.63:0/537075272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c010ff50 msgr2=0x7ff5c019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5bf400700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c010ff50 0x7ff5c019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5bf400700 1 -- 172.21.15.63:0/537075272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c019c270 con 0x7ff5c00760a0 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5c4c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c010ff50 0x7ff5c019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.704+0000 7ff5bf400700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00760a0 0x7ff5c01a1e40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c990 tx=0x7ff5ac00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.705+0000 7ff5bcc00700 1 -- 172.21.15.63:0/537075272 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac007b00 con 0x7ff5c00760a0 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.705+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c019c500 con 0x7ff5c00760a0 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.705+0000 7ff5bcc00700 1 -- 172.21.15.63:0/537075272 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5ac007ca0 con 0x7ff5c00760a0 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.705+0000 7ff5bcc00700 1 -- 172.21.15.63:0/537075272 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac018660 con 0x7ff5c00760a0 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.705+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c01a87d0 con 0x7ff5c00760a0 2024-05-11T13:58:38.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.706+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c006a240 con 0x7ff5c00760a0 2024-05-11T13:58:38.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.710+0000 7ff5bcc00700 1 -- 172.21.15.63:0/537075272 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff5ac00f450 con 0x7ff5c00760a0 2024-05-11T13:58:38.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.712+0000 7ff5bcc00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5a40b2a70 0x7ff5a40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:38.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.712+0000 7ff5bcc00700 1 -- 172.21.15.63:0/537075272 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9525+0+0 (secure 0 0 0) 0x7ff5ac0e2b00 con 0x7ff5c00760a0 2024-05-11T13:58:38.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.712+0000 7ff5bfe00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5a40b2a70 0x7ff5a40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:38.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.713+0000 7ff5bcc00700 1 -- 172.21.15.63:0/537075272 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5ac014030 con 0x7ff5c00760a0 2024-05-11T13:58:38.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.713+0000 7ff5bfe00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5a40b2a70 0x7ff5a40b4f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009e60 tx=0x7ff5b000b7b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:38.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.979+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff5c002e570 con 0x7ff5c00760a0 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.980+0000 7ff5bcc00700 1 -- 172.21.15.63:0/537075272 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v129) ==== 81+0+940 (secure 0 0 0) 0x7ff5ac0aa110 con 0x7ff5c00760a0 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1742389159 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/712822368 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2407940590 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3897275629 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1742389159 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/66647058 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:38.968 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2606154945 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:38.969 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2618629485 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:38.969 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2351804499 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:38.969 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/531186954 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:38.969 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4232658320 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:38.969 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:38.969 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:38.969 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1483448731 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:38.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5a40b2a70 msgr2=0x7ff5a40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:38.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5a40b2a70 0x7ff5a40b4f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009e60 tx=0x7ff5b000b7b0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00760a0 msgr2=0x7ff5c01a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:38.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00760a0 0x7ff5c01a1e40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c990 tx=0x7ff5ac00cca0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 shutdown_connections 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7ff5a40b2a70 0x7ff5a40b4f30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c010ff50 0x7ff5c019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c0075780 0x7ff5c01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 --2- 172.21.15.63:0/537075272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00760a0 0x7ff5c01a1e40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.984+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 >> 172.21.15.63:0/537075272 conn(0x7ff5c007a6b0 msgr2=0x7ff5c0100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.985+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 shutdown_connections 2024-05-11T13:58:38.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:38.985+0000 7ff5c6c00700 1 -- 172.21.15.63:0/537075272 wait complete. 2024-05-11T13:58:38.973 INFO:teuthology.orchestra.run.smithi063.stderr:listed 16 entries 2024-05-11T13:58:39.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:39 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/537075272' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:58:39.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:39 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/537075272' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:58:39.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:39 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/537075272' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:58:39.807 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-11T13:58:39.807 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T13:58:39.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-11T13:58:40.290 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:58:40.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:40 smithi063 ceph-mon[29809]: pgmap v784: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-11T13:58:40.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:40 smithi165 ceph-mon[33235]: pgmap v784: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-11T13:58:40.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:40 smithi071 ceph-mon[33293]: pgmap v784: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-11T13:58:42.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:42 smithi063 ceph-mon[29809]: pgmap v785: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-11T13:58:42.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:42 smithi165 ceph-mon[33235]: pgmap v785: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-11T13:58:42.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 -- 172.21.15.63:0/593432079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa010b730 msgr2=0x7f2aa010bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 --2- 172.21.15.63:0/593432079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa010b730 0x7f2aa010bb10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009990 tx=0x7f2a94009ca0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 -- 172.21.15.63:0/593432079 shutdown_connections 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 --2- 172.21.15.63:0/593432079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa01060f0 0x7f2aa0112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 --2- 172.21.15.63:0/593432079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa010b730 0x7f2aa010bb10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 --2- 172.21.15.63:0/593432079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa0105730 0x7f2aa0105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 -- 172.21.15.63:0/593432079 >> 172.21.15.63:0/593432079 conn(0x7f2aa0101200 msgr2=0x7f2aa0103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 -- 172.21.15.63:0/593432079 shutdown_connections 2024-05-11T13:58:42.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.615+0000 7f2aa7600700 1 -- 172.21.15.63:0/593432079 wait complete. 2024-05-11T13:58:42.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.616+0000 7f2aa7600700 1 Processor -- start 2024-05-11T13:58:42.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.616+0000 7f2aa7600700 1 -- start start 2024-05-11T13:58:42.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2aa7600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa0105730 0x7f2aa019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:42.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2aa7600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa01060f0 0x7f2aa019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:42.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2aa7600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa010b730 0x7f2aa01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:42.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2aa7600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0115810 con 0x7f2aa01060f0 2024-05-11T13:58:42.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2aa7600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0115690 con 0x7f2aa010b730 2024-05-11T13:58:42.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2aa7600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0115990 con 0x7f2aa0105730 2024-05-11T13:58:42.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2a9fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa01060f0 0x7f2aa019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:42.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2a9fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa01060f0 0x7f2aa019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60642/0 (socket says 172.21.15.63:60642) 2024-05-11T13:58:42.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2a9fe00700 1 -- 172.21.15.63:0/1498106488 learned_addr learned my addr 172.21.15.63:0/1498106488 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:58:42.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.617+0000 7f2aa4c00700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa0105730 0x7f2aa019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2aa5600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa010b730 0x7f2aa01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2a9fe00700 1 -- 172.21.15.63:0/1498106488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa0105730 msgr2=0x7f2aa019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2a9fe00700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa0105730 0x7f2aa019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2a9fe00700 1 -- 172.21.15.63:0/1498106488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa010b730 msgr2=0x7f2aa01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2a9fe00700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa010b730 0x7f2aa01a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2a9fe00700 1 -- 172.21.15.63:0/1498106488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa01a0d70 con 0x7f2aa01060f0 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2aa5600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa010b730 0x7f2aa01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.618+0000 7f2a9fe00700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa01060f0 0x7f2aa019c2a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c920 tx=0x7f2a8c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.619+0000 7f2a9d600700 1 -- 172.21.15.63:0/1498106488 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c007b00 con 0x7f2aa01060f0 2024-05-11T13:58:42.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.619+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa01a1060 con 0x7f2aa01060f0 2024-05-11T13:58:42.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.619+0000 7f2a9d600700 1 -- 172.21.15.63:0/1498106488 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a8c004cc0 con 0x7f2aa01060f0 2024-05-11T13:58:42.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.619+0000 7f2a9d600700 1 -- 172.21.15.63:0/1498106488 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c01d060 con 0x7f2aa01060f0 2024-05-11T13:58:42.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.619+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa01a88e0 con 0x7f2aa01060f0 2024-05-11T13:58:42.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.620+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa006a240 con 0x7f2aa01060f0 2024-05-11T13:58:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.622+0000 7f2a9d600700 1 -- 172.21.15.63:0/1498106488 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2a8c005410 con 0x7f2aa01060f0 2024-05-11T13:58:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.624+0000 7f2a9d600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2a880b29f0 0x7f2a880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.624+0000 7f2a9d600700 1 -- 172.21.15.63:0/1498106488 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9525+0+0 (secure 0 0 0) 0x7f2a8c0e25a0 con 0x7f2aa01060f0 2024-05-11T13:58:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.624+0000 7f2aa4c00700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2a880b29f0 0x7f2a880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.625+0000 7f2aa4c00700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2a880b29f0 0x7f2a880b4eb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a94006010 tx=0x7f2a9400b540 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:42.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.629+0000 7f2a9d600700 1 -- 172.21.15.63:0/1498106488 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a8c014030 con 0x7f2aa01060f0 2024-05-11T13:58:42.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:42 smithi071 ceph-mon[33293]: pgmap v785: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-11T13:58:42.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.882+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2aa002e570 con 0x7f2aa01060f0 2024-05-11T13:58:42.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.883+0000 7f2a9d600700 1 -- 172.21.15.63:0/1498106488 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v129) ==== 81+0+940 (secure 0 0 0) 0x7f2a8c0a9b90 con 0x7f2aa01060f0 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1742389159 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/712822368 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2407940590 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3897275629 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1742389159 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/66647058 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/405023443 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:42.871 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2606154945 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:42.872 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2618629485 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:42.872 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2351804499 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:42.872 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/531186954 2024-05-12T13:33:51.933837+0000 2024-05-11T13:58:42.872 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4232658320 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:42.872 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:42.872 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1250505383 2024-05-12T13:18:04.171969+0000 2024-05-11T13:58:42.872 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1483448731 2024-05-12T13:18:39.354713+0000 2024-05-11T13:58:42.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2a880b29f0 msgr2=0x7f2a880b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:42.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2a880b29f0 0x7f2a880b4eb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a94006010 tx=0x7f2a9400b540 comp rx=0 tx=0).stop 2024-05-11T13:58:42.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa01060f0 msgr2=0x7f2aa019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:42.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa01060f0 0x7f2aa019c2a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c920 tx=0x7f2a8c00cc30 comp rx=0 tx=0).stop 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 shutdown_connections 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f2a880b29f0 0x7f2a880b4eb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa010b730 0x7f2aa01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa0105730 0x7f2aa019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 --2- 172.21.15.63:0/1498106488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aa01060f0 0x7f2aa019c2a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.887+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 >> 172.21.15.63:0/1498106488 conn(0x7f2aa0101200 msgr2=0x7f2aa0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.888+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 shutdown_connections 2024-05-11T13:58:42.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:42.888+0000 7f2aa7600700 1 -- 172.21.15.63:0/1498106488 wait complete. 2024-05-11T13:58:42.876 INFO:teuthology.orchestra.run.smithi063.stderr:listed 16 entries 2024-05-11T13:58:43.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:43 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1498106488' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:58:43.592 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:43 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1498106488' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:58:43.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:43 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1498106488' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T13:58:43.770 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-05-11T13:58:43.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:58:43.770 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /tmp/cephfs 2024-05-11T13:58:43.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[87027]: fuse finished with error 0 and tester_r 0 2024-05-11T13:58:44.101 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T13:58:44.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:44 smithi063 ceph-mon[29809]: pgmap v786: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:58:44.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:44 smithi165 ceph-mon[33235]: pgmap v786: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:58:44.624 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:44 smithi071 ceph-mon[33293]: pgmap v786: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:58:46.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:46 smithi165 ceph-mon[33235]: pgmap v787: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-11T13:58:46.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:46 smithi063 ceph-mon[29809]: pgmap v787: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-11T13:58:46.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:46 smithi071 ceph-mon[33293]: pgmap v787: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-11T13:58:48.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:48 smithi165 ceph-mon[33235]: pgmap v788: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:58:48.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:48 smithi063 ceph-mon[29809]: pgmap v788: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:58:48.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:48 smithi071 ceph-mon[33293]: pgmap v788: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-11T13:58:50.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T13:58:50.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:58:50.203 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-11T13:58:50.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:58:50.290 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-11T13:58:50.391 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-11T13:58:50.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:58:50.392 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-11T13:58:50.420 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:50 smithi063 ceph-mon[29809]: pgmap v789: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-05-11T13:58:50.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T13:58:50.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-11T13:58:50.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:50 smithi165 ceph-mon[33235]: pgmap v789: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-05-11T13:58:50.631 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-11T13:58:50.644 INFO:tasks.cephadm:Teardown begin 2024-05-11T13:58:50.644 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T13:58:50.644 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:58:50.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:50 smithi071 ceph-mon[33293]: pgmap v789: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-05-11T13:58:51.082 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:58:52.422 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:52 smithi063 ceph-mon[29809]: pgmap v790: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-11T13:58:52.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:52 smithi165 ceph-mon[33235]: pgmap v790: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-11T13:58:52.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:52 smithi071 ceph-mon[33293]: pgmap v790: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-11T13:58:53.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 -- 172.21.15.63:0/3697291972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 msgr2=0x7fc08c105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 --2- 172.21.15.63:0/3697291972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c105c00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc078009990 tx=0x7fc078009ca0 comp rx=0 tx=0).stop 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 -- 172.21.15.63:0/3697291972 shutdown_connections 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 --2- 172.21.15.63:0/3697291972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc08c106140 0x7fc08c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 --2- 172.21.15.63:0/3697291972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc08c10b780 0x7fc08c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 --2- 172.21.15.63:0/3697291972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c105c00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 -- 172.21.15.63:0/3697291972 >> 172.21.15.63:0/3697291972 conn(0x7fc08c101270 msgr2=0x7fc08c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 -- 172.21.15.63:0/3697291972 shutdown_connections 2024-05-11T13:58:53.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.727+0000 7fc094600700 1 -- 172.21.15.63:0/3697291972 wait complete. 2024-05-11T13:58:53.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.728+0000 7fc094600700 1 Processor -- start 2024-05-11T13:58:53.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc094600700 1 -- start start 2024-05-11T13:58:53.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc094600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc094600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc08c106140 0x7fc08c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc094600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc08c10b780 0x7fc08c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc094600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c115810 con 0x7fc08c105780 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc094600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c115690 con 0x7fc08c106140 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc094600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c115990 con 0x7fc08c10b780 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc091c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc091c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c1a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45484/0 (socket says 172.21.15.63:45484) 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.729+0000 7fc091c00700 1 -- 172.21.15.63:0/3618307884 learned_addr learned my addr 172.21.15.63:0/3618307884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc092600700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc08c10b780 0x7fc08c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:53.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc091200700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc08c106140 0x7fc08c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:53.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc091c00700 1 -- 172.21.15.63:0/3618307884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc08c10b780 msgr2=0x7fc08c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:53.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc091c00700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc08c10b780 0x7fc08c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:53.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc091c00700 1 -- 172.21.15.63:0/3618307884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc08c106140 msgr2=0x7fc08c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:53.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc091c00700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc08c106140 0x7fc08c1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:53.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc091c00700 1 -- 172.21.15.63:0/3618307884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c19c230 con 0x7fc08c105780 2024-05-11T13:58:53.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.730+0000 7fc091c00700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c1a1910 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc08000c9b0 tx=0x7fc08000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:53.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.731+0000 7fc08a000700 1 -- 172.21.15.63:0/3618307884 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080007b00 con 0x7fc08c105780 2024-05-11T13:58:53.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.731+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c19c520 con 0x7fc08c105780 2024-05-11T13:58:53.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.731+0000 7fc08a000700 1 -- 172.21.15.63:0/3618307884 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc080007ca0 con 0x7fc08c105780 2024-05-11T13:58:53.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.731+0000 7fc08a000700 1 -- 172.21.15.63:0/3618307884 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080018660 con 0x7fc08c105780 2024-05-11T13:58:53.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.731+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a87e0 con 0x7fc08c105780 2024-05-11T13:58:53.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.732+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c06a240 con 0x7fc08c105780 2024-05-11T13:58:53.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.735+0000 7fc08a000700 1 -- 172.21.15.63:0/3618307884 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc08000f450 con 0x7fc08c105780 2024-05-11T13:58:53.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.736+0000 7fc08a000700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc0740b2a70 0x7fc0740b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:53.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.737+0000 7fc08a000700 1 -- 172.21.15.63:0/3618307884 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9525+0+0 (secure 0 0 0) 0x7fc0800e2ce0 con 0x7fc08c105780 2024-05-11T13:58:53.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.737+0000 7fc091200700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc0740b2a70 0x7fc0740b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:53.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.737+0000 7fc091200700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc0740b2a70 0x7fc0740b4f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc078005850 tx=0x7fc07801a040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:53.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:53.739+0000 7fc08a000700 1 -- 172.21.15.63:0/3618307884 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc080014030 con 0x7fc08c105780 2024-05-11T13:58:54.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.028+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc08c19cfc0 con 0x7fc08c105780 2024-05-11T13:58:54.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.029+0000 7fc08a000700 1 -- 172.21.15.63:0/3618307884 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+11943 (secure 0 0 0) 0x7fc0800aa380 con 0x7fc08c105780 2024-05-11T13:58:54.016 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:58:54.017 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":82,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:57:36.639239+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:58:54.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc0740b2a70 msgr2=0x7fc0740b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:54.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc0740b2a70 0x7fc0740b4f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc078005850 tx=0x7fc07801a040 comp rx=0 tx=0).stop 2024-05-11T13:58:54.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 msgr2=0x7fc08c1a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:54.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c1a1910 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc08000c9b0 tx=0x7fc08000cd70 comp rx=0 tx=0).stop 2024-05-11T13:58:54.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 shutdown_connections 2024-05-11T13:58:54.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7fc0740b2a70 0x7fc0740b4f30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:54.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc08c10b780 0x7fc08c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:54.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08c105780 0x7fc08c1a1910 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:54.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 --2- 172.21.15.63:0/3618307884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc08c106140 0x7fc08c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:54.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 >> 172.21.15.63:0/3618307884 conn(0x7fc08c101270 msgr2=0x7fc08c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:54.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 shutdown_connections 2024-05-11T13:58:54.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:54.033+0000 7fc094600700 1 -- 172.21.15.63:0/3618307884 wait complete. 2024-05-11T13:58:54.022 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 82 2024-05-11T13:58:54.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:54 smithi165 ceph-mon[33235]: pgmap v791: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-11T13:58:54.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:54 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3618307884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:58:54.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:54 smithi063 ceph-mon[29809]: pgmap v791: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-11T13:58:54.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:54 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3618307884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:58:54.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:54 smithi071 ceph-mon[33293]: pgmap v791: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-11T13:58:54.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:54 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3618307884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:58:54.876 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T13:58:55.324 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:58:56.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:56 smithi165 ceph-mon[33235]: pgmap v792: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-11T13:58:56.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:56 smithi063 ceph-mon[29809]: pgmap v792: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-11T13:58:56.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:56 smithi071 ceph-mon[33293]: pgmap v792: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.810+0000 7f158f000700 1 -- 172.21.15.63:0/1552717097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158810b660 msgr2=0x7f158810ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.810+0000 7f158f000700 1 --2- 172.21.15.63:0/1552717097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158810b660 0x7f158810ba40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f157c009990 tx=0x7f157c009ca0 comp rx=0 tx=0).stop 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.810+0000 7f158f000700 1 -- 172.21.15.63:0/1552717097 shutdown_connections 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.811+0000 7f158f000700 1 --2- 172.21.15.63:0/1552717097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1588106020 0x7f1588112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.811+0000 7f158f000700 1 --2- 172.21.15.63:0/1552717097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158810b660 0x7f158810ba40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.811+0000 7f158f000700 1 --2- 172.21.15.63:0/1552717097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1588105660 0x7f1588105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.811+0000 7f158f000700 1 -- 172.21.15.63:0/1552717097 >> 172.21.15.63:0/1552717097 conn(0x7f1588101130 msgr2=0x7f1588103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.811+0000 7f158f000700 1 -- 172.21.15.63:0/1552717097 shutdown_connections 2024-05-11T13:58:57.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.811+0000 7f158f000700 1 -- 172.21.15.63:0/1552717097 wait complete. 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 Processor -- start 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 -- start start 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1588105660 0x7f15881a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1588106020 0x7f15881a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f158810b660 0x7f158819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15881156f0 con 0x7f1588105660 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1588115570 con 0x7f158810b660 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.812+0000 7f158f000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1588115870 con 0x7f1588106020 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1588105660 0x7f15881a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1588105660 0x7f15881a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45492/0 (socket says 172.21.15.63:45492) 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 -- 172.21.15.63:0/1718778135 learned_addr learned my addr 172.21.15.63:0/1718778135 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158d200700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1588106020 0x7f15881a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:57.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158e600700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f158810b660 0x7f158819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 -- 172.21.15.63:0/1718778135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1588106020 msgr2=0x7f15881a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1588106020 0x7f15881a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 -- 172.21.15.63:0/1718778135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f158810b660 msgr2=0x7f158819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f158810b660 0x7f158819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158dc00700 1 -- 172.21.15.63:0/1718778135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158819c190 con 0x7f1588105660 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.813+0000 7f158e600700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f158810b660 0x7f158819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.814+0000 7f158dc00700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1588105660 0x7f15881a1870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f157c0057c0 tx=0x7f157c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.814+0000 7f1586000700 1 -- 172.21.15.63:0/1718778135 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c027070 con 0x7f1588105660 2024-05-11T13:58:57.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.814+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158819c420 con 0x7f1588105660 2024-05-11T13:58:57.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.814+0000 7f1586000700 1 -- 172.21.15.63:0/1718778135 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f157c00fda0 con 0x7f1588105660 2024-05-11T13:58:57.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.814+0000 7f1586000700 1 -- 172.21.15.63:0/1718778135 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c022ca0 con 0x7f1588105660 2024-05-11T13:58:57.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.814+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15881a8740 con 0x7f1588105660 2024-05-11T13:58:57.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.815+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1568005330 con 0x7f1588105660 2024-05-11T13:58:57.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.818+0000 7f1586000700 1 -- 172.21.15.63:0/1718778135 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f157c02a070 con 0x7f1588105660 2024-05-11T13:58:57.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.819+0000 7f1586000700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f15700b2a70 0x7f15700b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:58:57.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.819+0000 7f1586000700 1 -- 172.21.15.63:0/1718778135 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9525+0+0 (secure 0 0 0) 0x7f157c0e3940 con 0x7f1588105660 2024-05-11T13:58:57.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.820+0000 7f158d200700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f15700b2a70 0x7f15700b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:58:57.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.820+0000 7f158d200700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f15700b2a70 0x7f15700b4f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1574005950 tx=0x7f15740058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:58:57.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:57.822+0000 7f1586000700 1 -- 172.21.15.63:0/1718778135 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f157c013070 con 0x7f1588105660 2024-05-11T13:58:58.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.115+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1568005620 con 0x7f1588105660 2024-05-11T13:58:58.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.117+0000 7f1586000700 1 -- 172.21.15.63:0/1718778135 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+11943 (secure 0 0 0) 0x7f157c0aaf60 con 0x7f1588105660 2024-05-11T13:58:58.104 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:58:58.105 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1042825923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1042825923},{"type":"v1","addr":"172.21.15.63:6835","nonce":1042825923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3240025359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":3240025359},{"type":"v1","addr":"172.21.15.63:6837","nonce":3240025359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":82,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T13:28:38.143704+0000","modified":"2024-05-11T13:57:36.639239+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34451,"mds_2":34427,"mds_3":34403,"mds_4":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6839/3189418422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3189418422},{"type":"v1","addr":"172.21.15.63:6839","nonce":3189418422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14577":{"gid":14577,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/3763187067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":3763187067},{"type":"v1","addr":"172.21.15.63:6841","nonce":3763187067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/2703259087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2703259087},{"type":"v1","addr":"172.21.15.165:6833","nonce":2703259087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/4024086251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4024086251},{"type":"v1","addr":"172.21.15.165:6835","nonce":4024086251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2308228126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2308228126},{"type":"v1","addr":"172.21.15.165:6837","nonce":2308228126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":30,"addr":"172.21.15.165:6839/1121013005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1121013005},{"type":"v1","addr":"172.21.15.165:6839","nonce":1121013005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34403":{"gid":34403,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":37,"addr":"172.21.15.71:6833/1870291573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1870291573},{"type":"v1","addr":"172.21.15.71:6833","nonce":1870291573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.71:6835/1316489545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1316489545},{"type":"v1","addr":"172.21.15.71:6835","nonce":1316489545}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":30,"addr":"172.21.15.71:6837/136473158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":136473158},{"type":"v1","addr":"172.21.15.71:6837","nonce":136473158}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6839/1525017608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1525017608},{"type":"v1","addr":"172.21.15.71:6839","nonce":1525017608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34403,24472,34427,34472,34451]},"id":1}]} 2024-05-11T13:58:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f15700b2a70 msgr2=0x7f15700b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f15700b2a70 0x7f15700b4f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1574005950 tx=0x7f15740058e0 comp rx=0 tx=0).stop 2024-05-11T13:58:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1588105660 msgr2=0x7f15881a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:58:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1588105660 0x7f15881a1870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f157c0057c0 tx=0x7f157c009f90 comp rx=0 tx=0).stop 2024-05-11T13:58:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 shutdown_connections 2024-05-11T13:58:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f15700b2a70 0x7f15700b4f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f158810b660 0x7f158819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1588105660 0x7f15881a1870 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 --2- 172.21.15.63:0/1718778135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1588106020 0x7f15881a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:58:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 >> 172.21.15.63:0/1718778135 conn(0x7f1588101130 msgr2=0x7f1588101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:58:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 shutdown_connections 2024-05-11T13:58:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:58:58.121+0000 7f158f000700 1 -- 172.21.15.63:0/1718778135 wait complete. 2024-05-11T13:58:58.111 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 82 2024-05-11T13:58:58.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:58 smithi063 ceph-mon[29809]: pgmap v793: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-05-11T13:58:58.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:58:58 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1718778135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:58:58.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:58 smithi071 ceph-mon[33293]: pgmap v793: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-05-11T13:58:58.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:58:58 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1718778135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:58:58.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:58 smithi165 ceph-mon[33235]: pgmap v793: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-05-11T13:58:58.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:58:58 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1718778135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T13:58:58.989 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T13:58:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:59:00.667 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:00 smithi063 ceph-mon[29809]: pgmap v794: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 KiB/s wr, 14 op/s 2024-05-11T13:59:00.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:00 smithi071 ceph-mon[33293]: pgmap v794: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 KiB/s wr, 14 op/s 2024-05-11T13:59:00.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:00 smithi165 ceph-mon[33235]: pgmap v794: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 KiB/s wr, 14 op/s 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 -- 172.21.15.63:0/1395574461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4105760 msgr2=0x7f4ef4105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 --2- 172.21.15.63:0/1395574461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4105760 0x7f4ef4105be0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009990 tx=0x7f4ee0009ca0 comp rx=0 tx=0).stop 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 -- 172.21.15.63:0/1395574461 shutdown_connections 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 --2- 172.21.15.63:0/1395574461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef4106120 0x7f4ef4112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 --2- 172.21.15.63:0/1395574461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef410b760 0x7f4ef410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 --2- 172.21.15.63:0/1395574461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4105760 0x7f4ef4105be0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 -- 172.21.15.63:0/1395574461 >> 172.21.15.63:0/1395574461 conn(0x7f4ef4101250 msgr2=0x7f4ef4103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 -- 172.21.15.63:0/1395574461 shutdown_connections 2024-05-11T13:59:01.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.822+0000 7f4efa400700 1 -- 172.21.15.63:0/1395574461 wait complete. 2024-05-11T13:59:01.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.823+0000 7f4efa400700 1 Processor -- start 2024-05-11T13:59:01.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.823+0000 7f4efa400700 1 -- start start 2024-05-11T13:59:01.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4efa400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef4105760 0x7f4ef419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:01.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4efa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4106120 0x7f4ef419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:01.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4efa400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef410b760 0x7f4ef41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:01.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4efa400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41158a0 con 0x7f4ef4106120 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4efa400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4115720 con 0x7f4ef4105760 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4efa400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4115a20 con 0x7f4ef410b760 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4ef2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4106120 0x7f4ef419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4ef2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4106120 0x7f4ef419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52464/0 (socket says 172.21.15.63:52464) 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.824+0000 7f4ef2a00700 1 -- 172.21.15.63:0/1492526076 learned_addr learned my addr 172.21.15.63:0/1492526076 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef3400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef4105760 0x7f4ef419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef3e00700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef410b760 0x7f4ef41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef2a00700 1 -- 172.21.15.63:0/1492526076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef410b760 msgr2=0x7f4ef41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef2a00700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef410b760 0x7f4ef41a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef2a00700 1 -- 172.21.15.63:0/1492526076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef4105760 msgr2=0x7f4ef419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef2a00700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef4105760 0x7f4ef419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef2a00700 1 -- 172.21.15.63:0/1492526076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef41a0da0 con 0x7f4ef4106120 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef3e00700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef410b760 0x7f4ef41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4ef2a00700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4106120 0x7f4ef419c2d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000c00 tx=0x7f4ee0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:01.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.825+0000 7f4eebe00700 1 -- 172.21.15.63:0/1492526076 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0027070 con 0x7f4ef4106120 2024-05-11T13:59:01.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.826+0000 7f4eebe00700 1 -- 172.21.15.63:0/1492526076 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee000fb80 con 0x7f4ef4106120 2024-05-11T13:59:01.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.826+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef41a1030 con 0x7f4ef4106120 2024-05-11T13:59:01.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.826+0000 7f4eebe00700 1 -- 172.21.15.63:0/1492526076 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0022c30 con 0x7f4ef4106120 2024-05-11T13:59:01.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.826+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef41a8910 con 0x7f4ef4106120 2024-05-11T13:59:01.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.827+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef406a240 con 0x7f4ef4106120 2024-05-11T13:59:01.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.829+0000 7f4eebe00700 1 -- 172.21.15.63:0/1492526076 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4ee002a070 con 0x7f4ef4106120 2024-05-11T13:59:01.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.831+0000 7f4eebe00700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4ed80b2a70 0x7f4ed80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:01.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.831+0000 7f4eebe00700 1 -- 172.21.15.63:0/1492526076 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9525+0+0 (secure 0 0 0) 0x7f4ee00e3b60 con 0x7f4ef4106120 2024-05-11T13:59:01.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.831+0000 7f4ef3400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4ed80b2a70 0x7f4ed80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:01.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.832+0000 7f4ef3400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4ed80b2a70 0x7f4ed80b4f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005fd0 tx=0x7f4ee4005db0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:01.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:01.834+0000 7f4eebe00700 1 -- 172.21.15.63:0/1492526076 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ee0031080 con 0x7f4ef4106120 2024-05-11T13:59:02.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.075+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ef402e570 con 0x7f4ef4106120 2024-05-11T13:59:02.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.077+0000 7f4eebe00700 1 -- 172.21.15.63:0/1492526076 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+25322 (secure 0 0 0) 0x7f4ee0013070 con 0x7f4ef4106120 2024-05-11T13:59:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-11T13:59:02.065 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":129,"fsid":"a0d7c9ac-0f98-11ef-bc98-c7b262605968","created":"2024-05-11T13:17:22.623485+0000","modified":"2024-05-11T13:58:15.447129+0000","last_up_change":"2024-05-11T13:27:12.438796+0000","last_in_change":"2024-05-11T13:26:46.704853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T13:21:55.534075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T13:28:29.069877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T13:28:33.385508+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":128,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T13:28:47.644228+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":128,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22f2805e-feb6-4126-b7f7-d4567818ce77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6803","nonce":1611555800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6805","nonce":1611555800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6809","nonce":1611555800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1611555800},{"type":"v1","addr":"172.21.15.63:6807","nonce":1611555800}]},"public_addr":"172.21.15.63:6803/1611555800","cluster_addr":"172.21.15.63:6805/1611555800","heartbeat_back_addr":"172.21.15.63:6809/1611555800","heartbeat_front_addr":"172.21.15.63:6807/1611555800","state":["exists","up"]},{"osd":1,"uuid":"29a3b52e-3ed5-403c-b9c4-0a69b8d1b989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6801","nonce":969700086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6803","nonce":969700086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6807","nonce":969700086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":969700086},{"type":"v1","addr":"172.21.15.71:6805","nonce":969700086}]},"public_addr":"172.21.15.71:6801/969700086","cluster_addr":"172.21.15.71:6803/969700086","heartbeat_back_addr":"172.21.15.71:6807/969700086","heartbeat_front_addr":"172.21.15.71:6805/969700086","state":["exists","up"]},{"osd":2,"uuid":"5f94b33d-6308-46a0-9f5c-63bd1bcaa477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6801","nonce":1296073597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6803","nonce":1296073597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6807","nonce":1296073597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1296073597},{"type":"v1","addr":"172.21.15.165:6805","nonce":1296073597}]},"public_addr":"172.21.15.165:6801/1296073597","cluster_addr":"172.21.15.165:6803/1296073597","heartbeat_back_addr":"172.21.15.165:6807/1296073597","heartbeat_front_addr":"172.21.15.165:6805/1296073597","state":["exists","up"]},{"osd":3,"uuid":"d387295c-4396-466f-92e3-400391f0e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6811","nonce":206479443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6813","nonce":206479443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6817","nonce":206479443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":206479443},{"type":"v1","addr":"172.21.15.63:6815","nonce":206479443}]},"public_addr":"172.21.15.63:6811/206479443","cluster_addr":"172.21.15.63:6813/206479443","heartbeat_back_addr":"172.21.15.63:6817/206479443","heartbeat_front_addr":"172.21.15.63:6815/206479443","state":["exists","up"]},{"osd":4,"uuid":"d209500e-da38-45b9-9ae7-eddaafae3127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6809","nonce":2625617400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6811","nonce":2625617400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6815","nonce":2625617400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2625617400},{"type":"v1","addr":"172.21.15.71:6813","nonce":2625617400}]},"public_addr":"172.21.15.71:6809/2625617400","cluster_addr":"172.21.15.71:6811/2625617400","heartbeat_back_addr":"172.21.15.71:6815/2625617400","heartbeat_front_addr":"172.21.15.71:6813/2625617400","state":["exists","up"]},{"osd":5,"uuid":"7dd5fca3-e67c-4824-af4e-2e6efa55db80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6809","nonce":2071542018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6811","nonce":2071542018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6815","nonce":2071542018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2071542018},{"type":"v1","addr":"172.21.15.165:6813","nonce":2071542018}]},"public_addr":"172.21.15.165:6809/2071542018","cluster_addr":"172.21.15.165:6811/2071542018","heartbeat_back_addr":"172.21.15.165:6815/2071542018","heartbeat_front_addr":"172.21.15.165:6813/2071542018","state":["exists","up"]},{"osd":6,"uuid":"18fec429-9a88-4389-b373-438e9cad3dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6819","nonce":1008047147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6821","nonce":1008047147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6825","nonce":1008047147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1008047147},{"type":"v1","addr":"172.21.15.63:6823","nonce":1008047147}]},"public_addr":"172.21.15.63:6819/1008047147","cluster_addr":"172.21.15.63:6821/1008047147","heartbeat_back_addr":"172.21.15.63:6825/1008047147","heartbeat_front_addr":"172.21.15.63:6823/1008047147","state":["exists","up"]},{"osd":7,"uuid":"ad08b6e9-8303-4d9b-8a92-f837c203f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6817","nonce":3194467426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6819","nonce":3194467426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6823","nonce":3194467426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3194467426},{"type":"v1","addr":"172.21.15.71:6821","nonce":3194467426}]},"public_addr":"172.21.15.71:6817/3194467426","cluster_addr":"172.21.15.71:6819/3194467426","heartbeat_back_addr":"172.21.15.71:6823/3194467426","heartbeat_front_addr":"172.21.15.71:6821/3194467426","state":["exists","up"]},{"osd":8,"uuid":"a5ddd20e-3176-42ba-853e-47ca81ab5ef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6817","nonce":4105128904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6819","nonce":4105128904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6823","nonce":4105128904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4105128904},{"type":"v1","addr":"172.21.15.165:6821","nonce":4105128904}]},"public_addr":"172.21.15.165:6817/4105128904","cluster_addr":"172.21.15.165:6819/4105128904","heartbeat_back_addr":"172.21.15.165:6823/4105128904","heartbeat_front_addr":"172.21.15.165:6821/4105128904","state":["exists","up"]},{"osd":9,"uuid":"4d377b63-0bda-4801-8f95-5e2e9b6bf9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6827","nonce":1803317888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6829","nonce":1803317888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6833","nonce":1803317888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1803317888},{"type":"v1","addr":"172.21.15.63:6831","nonce":1803317888}]},"public_addr":"172.21.15.63:6827/1803317888","cluster_addr":"172.21.15.63:6829/1803317888","heartbeat_back_addr":"172.21.15.63:6833/1803317888","heartbeat_front_addr":"172.21.15.63:6831/1803317888","state":["exists","up"]},{"osd":10,"uuid":"cb80b66c-5c56-420b-8b5d-a0128ec05742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6825","nonce":3023940928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6827","nonce":3023940928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6831","nonce":3023940928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3023940928},{"type":"v1","addr":"172.21.15.71:6829","nonce":3023940928}]},"public_addr":"172.21.15.71:6825/3023940928","cluster_addr":"172.21.15.71:6827/3023940928","heartbeat_back_addr":"172.21.15.71:6831/3023940928","heartbeat_front_addr":"172.21.15.71:6829/3023940928","state":["exists","up"]},{"osd":11,"uuid":"222725ae-5dca-47ba-9146-2c4ef02a3272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6825","nonce":2607132784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6827","nonce":2607132784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6831","nonce":2607132784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2607132784},{"type":"v1","addr":"172.21.15.165:6829","nonce":2607132784}]},"public_addr":"172.21.15.165:6825/2607132784","cluster_addr":"172.21.15.165:6827/2607132784","heartbeat_back_addr":"172.21.15.165:6831/2607132784","heartbeat_front_addr":"172.21.15.165:6829/2607132784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:20:45.009235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:18.291967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:21:51.470321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:22:26.668198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:01.150952+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:23:35.002428+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:10.720127+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:24:46.425008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:21.676044+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:25:57.672577+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:26:33.777580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T13:27:09.621882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/1742389159":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/712822368":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/2407940590":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/3897275629":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6801/1742389159":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/66647058":"2024-05-12T13:33:51.933837+0000","172.21.15.63:6801/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:6800/405023443":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2606154945":"2024-05-12T13:18:39.354713+0000","172.21.15.63:0/2618629485":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/2351804499":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/531186954":"2024-05-12T13:33:51.933837+0000","172.21.15.63:0/4232658320":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6800/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:6801/1250505383":"2024-05-12T13:18:04.171969+0000","172.21.15.63:0/1483448731":"2024-05-12T13:18:39.354713+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]},{"pool":4,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T13:59:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4ed80b2a70 msgr2=0x7f4ed80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4ed80b2a70 0x7f4ed80b4f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005fd0 tx=0x7f4ee4005db0 comp rx=0 tx=0).stop 2024-05-11T13:59:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4106120 msgr2=0x7f4ef419c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4106120 0x7f4ef419c2d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000c00 tx=0x7f4ee0005ef0 comp rx=0 tx=0).stop 2024-05-11T13:59:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 shutdown_connections 2024-05-11T13:59:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f4ed80b2a70 0x7f4ed80b4f30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef410b760 0x7f4ef41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef4105760 0x7f4ef419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 --2- 172.21.15.63:0/1492526076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ef4106120 0x7f4ef419c2d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 >> 172.21.15.63:0/1492526076 conn(0x7f4ef4101250 msgr2=0x7f4ef4101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.081+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 shutdown_connections 2024-05-11T13:59:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:02.082+0000 7f4efa400700 1 -- 172.21.15.63:0/1492526076 wait complete. 2024-05-11T13:59:02.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:02 smithi063 ceph-mon[29809]: pgmap v795: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-05-11T13:59:02.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:02 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/1492526076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:59:02.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:02 smithi071 ceph-mon[33293]: pgmap v795: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-05-11T13:59:02.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:02 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/1492526076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:59:02.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:02 smithi165 ceph-mon[33235]: pgmap v795: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-05-11T13:59:02.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:02 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/1492526076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T13:59:03.029 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-11T13:59:03.509 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:59:04.578 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:04 smithi063 ceph-mon[29809]: pgmap v796: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-05-11T13:59:04.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:04 smithi071 ceph-mon[33293]: pgmap v796: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-05-11T13:59:04.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:04 smithi165 ceph-mon[33235]: pgmap v796: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.938+0000 7f60c7200700 1 -- 172.21.15.63:0/136789395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c806c6d0 msgr2=0x7f60c8108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.938+0000 7f60c7200700 1 --2- 172.21.15.63:0/136789395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c806c6d0 0x7f60c8108110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f60b40099c0 tx=0x7f60b4009cd0 comp rx=0 tx=0).stop 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.939+0000 7f60c7200700 1 -- 172.21.15.63:0/136789395 shutdown_connections 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.939+0000 7f60c7200700 1 --2- 172.21.15.63:0/136789395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c8108840 0x7f60c81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.939+0000 7f60c7200700 1 --2- 172.21.15.63:0/136789395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60c806bd20 0x7f60c806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.939+0000 7f60c7200700 1 --2- 172.21.15.63:0/136789395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c806c6d0 0x7f60c8108110 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.939+0000 7f60c7200700 1 -- 172.21.15.63:0/136789395 >> 172.21.15.63:0/136789395 conn(0x7f60c8079290 msgr2=0x7f60c80796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.939+0000 7f60c7200700 1 -- 172.21.15.63:0/136789395 shutdown_connections 2024-05-11T13:59:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.939+0000 7f60c7200700 1 -- 172.21.15.63:0/136789395 wait complete. 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 Processor -- start 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 -- start start 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60c806bd20 0x7f60c81026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c806c6d0 0x7f60c8102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c8108840 0x7f60c8103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8113540 con 0x7f60c8108840 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60c81133c0 con 0x7f60c806bd20 2024-05-11T13:59:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.940+0000 7f60c7200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60c81136c0 con 0x7f60c806c6d0 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c8108840 0x7f60c8103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c8108840 0x7f60c8103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52482/0 (socket says 172.21.15.63:52482) 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 -- 172.21.15.63:0/2943401272 learned_addr learned my addr 172.21.15.63:0/2943401272 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c5400700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c806c6d0 0x7f60c8102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c5e00700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60c806bd20 0x7f60c81026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 -- 172.21.15.63:0/2943401272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c806c6d0 msgr2=0x7f60c8102c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c806c6d0 0x7f60c8102c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 -- 172.21.15.63:0/2943401272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60c806bd20 msgr2=0x7f60c81026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60c806bd20 0x7f60c81026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c6800700 1 -- 172.21.15.63:0/2943401272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81a7f20 con 0x7f60c8108840 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.941+0000 7f60c5e00700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60c806bd20 0x7f60c81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:59:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.942+0000 7f60c6800700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c8108840 0x7f60c8103160 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f60c0007f10 tx=0x7f60c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.942+0000 7f60bea00700 1 -- 172.21.15.63:0/2943401272 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0009e20 con 0x7f60c8108840 2024-05-11T13:59:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.942+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81a8210 con 0x7f60c8108840 2024-05-11T13:59:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.942+0000 7f60bea00700 1 -- 172.21.15.63:0/2943401272 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60c000dbd0 con 0x7f60c8108840 2024-05-11T13:59:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.942+0000 7f60bea00700 1 -- 172.21.15.63:0/2943401272 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00058d0 con 0x7f60c8108840 2024-05-11T13:59:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.942+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c81a8720 con 0x7f60c8108840 2024-05-11T13:59:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.943+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c806a240 con 0x7f60c8108840 2024-05-11T13:59:05.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.946+0000 7f60bea00700 1 -- 172.21.15.63:0/2943401272 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f60c001a020 con 0x7f60c8108840 2024-05-11T13:59:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.948+0000 7f60bea00700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60ac0b29f0 0x7f60ac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.948+0000 7f60bea00700 1 -- 172.21.15.63:0/2943401272 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9525+0+0 (secure 0 0 0) 0x7f60c00e2830 con 0x7f60c8108840 2024-05-11T13:59:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.948+0000 7f60c5e00700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60ac0b29f0 0x7f60ac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.949+0000 7f60c5e00700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60ac0b29f0 0x7f60ac0b4eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60b8005fd0 tx=0x7f60b8005e30 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:05.950+0000 7f60bea00700 1 -- 172.21.15.63:0/2943401272 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60c0010070 con 0x7f60c8108840 2024-05-11T13:59:06.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.242+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f60c802e730 con 0x7f60c8108840 2024-05-11T13:59:06.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.247+0000 7f60bea00700 1 -- 172.21.15.63:0/2943401272 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f60c00a9d20 con 0x7f60c8108840 2024-05-11T13:59:06.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60ac0b29f0 msgr2=0x7f60ac0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:06.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60ac0b29f0 0x7f60ac0b4eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60b8005fd0 tx=0x7f60b8005e30 comp rx=0 tx=0).stop 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c8108840 msgr2=0x7f60c8103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c8108840 0x7f60c8103160 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f60c0007f10 tx=0x7f60c000d3b0 comp rx=0 tx=0).stop 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 shutdown_connections 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f60ac0b29f0 0x7f60ac0b4eb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60c8108840 0x7f60c8103160 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60c806bd20 0x7f60c81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 --2- 172.21.15.63:0/2943401272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c806c6d0 0x7f60c8102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.255+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 >> 172.21.15.63:0/2943401272 conn(0x7f60c8079290 msgr2=0x7f60c81073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.256+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 shutdown_connections 2024-05-11T13:59:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:06.256+0000 7f60c7200700 1 -- 172.21.15.63:0/2943401272 wait complete. 2024-05-11T13:59:06.246 INFO:teuthology.orchestra.run.smithi063.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a[29805]: 2024-05-11T13:59:06.243+0000 7f341ea00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mon[29809]: pgmap v797: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2943401272' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mon[29809]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mon[29809]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/2943401272' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mon[29809]: osdmap e130: 12 total, 12 up, 12 in 2024-05-11T13:59:06.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mon[29809]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a[69379]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a[69379]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mds[69383]: -- 172.21.15.63:0/1474563903 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a70711a40 con 0x563a706e4c00 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a[69379]: starting mds.a at 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a[69379]: 2024-05-11T13:59:06.435+0000 7f4c79200700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g[68957]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g[68957]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.673 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:59:06 smithi063 ceph-mds[68961]: -- 172.21.15.63:0/2647172352 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558f8795a40 con 0x5558f8768c00 2024-05-11T13:59:06.674 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g[68957]: starting mds.g at 2024-05-11T13:59:06.674 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 13:59:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g[68957]: 2024-05-11T13:59:06.438+0000 7f51aae00700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6840/0: (98) Address already in use 2024-05-11T13:59:06.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mon[33293]: pgmap v797: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2943401272' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mon[33293]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mon[33293]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/2943401272' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mon[33293]: osdmap e130: 12 total, 12 up, 12 in 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mon[33293]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b[63166]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b[63166]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.691 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mds[63175]: -- 172.21.15.71:0/2860217117 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618ecaf1a40 con 0x5618ecac5000 2024-05-11T13:59:06.692 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b[63166]: starting mds.b at 2024-05-11T13:59:06.692 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e[62544]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.692 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e[62544]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.692 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mds[62548]: -- 172.21.15.71:0/3459295666 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4ce931a40 con 0x55a4ce904800 2024-05-11T13:59:06.692 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e[62544]: starting mds.e at 2024-05-11T13:59:06.692 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k[62232]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.692 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k[62232]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.693 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mds[62241]: -- 172.21.15.71:0/2216527321 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564094387a40 con 0x56409435ac00 2024-05-11T13:59:06.693 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k[62232]: starting mds.k at 2024-05-11T13:59:06.693 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h[62856]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.693 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h[62856]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.693 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:59:06 smithi071 ceph-mds[62862]: -- 172.21.15.71:0/816056366 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dbaa8b1a40 con 0x55dbaa884800 2024-05-11T13:59:06.693 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 13:59:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h[62856]: starting mds.h at 2024-05-11T13:59:06.803 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c[63181]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.803 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c[63181]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.803 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mds[63185]: -- 172.21.15.165:0/1676422472 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593782d1a40 con 0x5593782a4800 2024-05-11T13:59:06.803 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c[63181]: starting mds.c at 2024-05-11T13:59:06.803 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i[63485]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.804 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i[63485]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.804 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mds[63489]: -- 172.21.15.165:0/147450144 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560168d0ba40 con 0x560168cdec00 2024-05-11T13:59:06.804 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i[63485]: starting mds.i at 2024-05-11T13:59:06.804 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l[62564]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.804 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l[62564]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.804 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mds[62568]: -- 172.21.15.165:0/1719242907 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fddfb7fa40 con 0x55fddfb52c00 2024-05-11T13:59:06.804 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l[62564]: starting mds.l at 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mon[33235]: pgmap v797: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2943401272' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mon[33235]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mon[33235]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/2943401272' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mon[33235]: osdmap e130: 12 total, 12 up, 12 in 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mon[33235]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f[62874]: ignoring --setuser ceph since I am not root 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f[62874]: ignoring --setgroup ceph since I am not root 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:59:06 smithi165 ceph-mds[62880]: -- 172.21.15.165:0/2699247312 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1f9729a40 con 0x55c1f96fc800 2024-05-11T13:59:06.805 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 13:59:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f[62874]: starting mds.f at 2024-05-11T13:59:07.245 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T13:59:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: pgmap v799: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6832/1593335932,v1:172.21.15.165:6834/1593335932] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6833/1244130543,v1:172.21.15.165:6835/1244130543] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6832/1626836367,v1:172.21.15.71:6834/1626836367] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6833/1155064403,v1:172.21.15.71:6835/1155064403] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6837/2403479441,v1:172.21.15.165:6839/2403479441] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.165:6836/2768885676,v1:172.21.15.165:6838/2768885676] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6836/3260258818,v1:172.21.15.71:6838/3260258818] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.71:6837/399704471,v1:172.21.15.71:6839/399704471] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.63:6839/3199425923,v1:172.21.15.63:6840/3199425923] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: mds.? [v2:172.21.15.63:6838/1298506220,v1:172.21.15.63:6841/1298506220] up:boot 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:59:08.672 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:59:08.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:08 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: pgmap v799: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6832/1593335932,v1:172.21.15.165:6834/1593335932] up:boot 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6833/1244130543,v1:172.21.15.165:6835/1244130543] up:boot 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6832/1626836367,v1:172.21.15.71:6834/1626836367] up:boot 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6833/1155064403,v1:172.21.15.71:6835/1155064403] up:boot 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6837/2403479441,v1:172.21.15.165:6839/2403479441] up:boot 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.165:6836/2768885676,v1:172.21.15.165:6838/2768885676] up:boot 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6836/3260258818,v1:172.21.15.71:6838/3260258818] up:boot 2024-05-11T13:59:08.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.71:6837/399704471,v1:172.21.15.71:6839/399704471] up:boot 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.63:6839/3199425923,v1:172.21.15.63:6840/3199425923] up:boot 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: mds.? [v2:172.21.15.63:6838/1298506220,v1:172.21.15.63:6841/1298506220] up:boot 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:59:08.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:08 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:59:08.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: pgmap v799: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-11T13:59:08.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6832/1593335932,v1:172.21.15.165:6834/1593335932] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6833/1244130543,v1:172.21.15.165:6835/1244130543] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6832/1626836367,v1:172.21.15.71:6834/1626836367] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6833/1155064403,v1:172.21.15.71:6835/1155064403] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6837/2403479441,v1:172.21.15.165:6839/2403479441] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.165:6836/2768885676,v1:172.21.15.165:6838/2768885676] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6836/3260258818,v1:172.21.15.71:6838/3260258818] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.71:6837/399704471,v1:172.21.15.71:6839/399704471] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.63:6839/3199425923,v1:172.21.15.63:6840/3199425923] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: mds.? [v2:172.21.15.63:6838/1298506220,v1:172.21.15.63:6841/1298506220] up:boot 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T13:59:08.804 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T13:59:08.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T13:59:08.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T13:59:08.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T13:59:08.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T13:59:08.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T13:59:08.805 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:08 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T13:59:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 -- 172.21.15.63:0/1769446721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68381095e0 msgr2=0x7f68381099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 --2- 172.21.15.63:0/1769446721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68381095e0 0x7f68381099c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6828009990 tx=0x7f6828009ca0 comp rx=0 tx=0).stop 2024-05-11T13:59:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 -- 172.21.15.63:0/1769446721 shutdown_connections 2024-05-11T13:59:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 --2- 172.21.15.63:0/1769446721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f683806c5e0 0x7f6838110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 --2- 172.21.15.63:0/1769446721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68381095e0 0x7f68381099c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 --2- 172.21.15.63:0/1769446721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683806bc20 0x7f683806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 -- 172.21.15.63:0/1769446721 >> 172.21.15.63:0/1769446721 conn(0x7f6838079400 msgr2=0x7f6838079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 -- 172.21.15.63:0/1769446721 shutdown_connections 2024-05-11T13:59:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.954+0000 7f683d200700 1 -- 172.21.15.63:0/1769446721 wait complete. 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.956+0000 7f683d200700 1 Processor -- start 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.956+0000 7f683d200700 1 -- start start 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f683d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f683806bc20 0x7f683819be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f683d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683806c5e0 0x7f683819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f683d200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68381095e0 0x7f68381a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f683d200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6838113620 con 0x7f683806bc20 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f683d200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68381134a0 con 0x7f683806c5e0 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f683d200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68381137a0 con 0x7f68381095e0 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f6835e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f683806bc20 0x7f683819be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f6835e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f683806bc20 0x7f683819be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41098/0 (socket says 172.21.15.63:41098) 2024-05-11T13:59:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f6835e00700 1 -- 172.21.15.63:0/321425822 learned_addr learned my addr 172.21.15.63:0/321425822 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f6835400700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683806c5e0 0x7f683819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.957+0000 7f6836800700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68381095e0 0x7f68381a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f6835e00700 1 -- 172.21.15.63:0/321425822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68381095e0 msgr2=0x7f68381a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f6835e00700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68381095e0 0x7f68381a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f6835e00700 1 -- 172.21.15.63:0/321425822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683806c5e0 msgr2=0x7f683819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f6835e00700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683806c5e0 0x7f683819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f6835e00700 1 -- 172.21.15.63:0/321425822 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68381a0d90 con 0x7f683806bc20 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f6835e00700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f683806bc20 0x7f683819be70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f682800b650 tx=0x7f682800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f682ea00700 1 -- 172.21.15.63:0/321425822 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828027070 con 0x7f683806bc20 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68381a1020 con 0x7f683806bc20 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f682ea00700 1 -- 172.21.15.63:0/321425822 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6828022470 con 0x7f683806bc20 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.958+0000 7f682ea00700 1 -- 172.21.15.63:0/321425822 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828017650 con 0x7f683806bc20 2024-05-11T13:59:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.959+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68381a8900 con 0x7f683806bc20 2024-05-11T13:59:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.960+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6838104ca0 con 0x7f683806bc20 2024-05-11T13:59:09.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.962+0000 7f682ea00700 1 -- 172.21.15.63:0/321425822 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6828009d10 con 0x7f683806bc20 2024-05-11T13:59:09.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.963+0000 7f682ea00700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f681c0b2a70 0x7f681c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:09.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.964+0000 7f682ea00700 1 -- 172.21.15.63:0/321425822 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9867+0+0 (secure 0 0 0) 0x7f68280e37f0 con 0x7f683806bc20 2024-05-11T13:59:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.964+0000 7f6835400700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f681c0b2a70 0x7f681c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.965+0000 7f6835400700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f681c0b2a70 0x7f681c0b4f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6824005950 tx=0x7f68240058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:09.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:09.969+0000 7f682ea00700 1 -- 172.21.15.63:0/321425822 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6828013070 con 0x7f683806bc20 2024-05-11T13:59:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.265+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f68381a8ad0 con 0x7f683806bc20 2024-05-11T13:59:10.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.377+0000 7f682ea00700 1 -- 172.21.15.63:0/321425822 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) ==== 104+0+0 (secure 0 0 0) 0x7f68280aacc0 con 0x7f683806bc20 2024-05-11T13:59:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.381+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f681c0b2a70 msgr2=0x7f681c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.381+0000 7f683d200700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f681c0b2a70 0x7f681c0b4f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6824005950 tx=0x7f68240058e0 comp rx=0 tx=0).stop 2024-05-11T13:59:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.381+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f683806bc20 msgr2=0x7f683819be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.381+0000 7f683d200700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f683806bc20 0x7f683819be70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f682800b650 tx=0x7f682800bfc0 comp rx=0 tx=0).stop 2024-05-11T13:59:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 shutdown_connections 2024-05-11T13:59:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f681c0b2a70 0x7f681c0b4f30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:10.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68381095e0 0x7f68381a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:10.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f683806bc20 0x7f683819be70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:10.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 --2- 172.21.15.63:0/321425822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683806c5e0 0x7f683819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:10.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 >> 172.21.15.63:0/321425822 conn(0x7f6838079400 msgr2=0x7f6838101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:10.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 shutdown_connections 2024-05-11T13:59:10.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:10.382+0000 7f683d200700 1 -- 172.21.15.63:0/321425822 wait complete. 2024-05-11T13:59:10.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:10 smithi063 ceph-mon[29809]: pgmap v800: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:10.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:10 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/321425822' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:59:10.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:10 smithi071 ceph-mon[33293]: pgmap v800: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:10.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:10 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/321425822' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:59:10.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:10 smithi165 ceph-mon[33235]: pgmap v800: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:10.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:10 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/321425822' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T13:59:11.184 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T13:59:11.660 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:59:11.670 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:11 smithi063 ceph-mon[29809]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-11T13:59:11.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:11 smithi063 ceph-mon[29809]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-11T13:59:11.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:11 smithi063 ceph-mon[29809]: Cluster is now healthy 2024-05-11T13:59:11.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:11 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/321425822' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-11T13:59:11.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:11 smithi063 ceph-mon[29809]: fsmap 12 up:standby 2024-05-11T13:59:11.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:11 smithi071 ceph-mon[33293]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-11T13:59:11.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:11 smithi071 ceph-mon[33293]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-11T13:59:11.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:11 smithi071 ceph-mon[33293]: Cluster is now healthy 2024-05-11T13:59:11.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:11 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/321425822' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-11T13:59:11.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:11 smithi071 ceph-mon[33293]: fsmap 12 up:standby 2024-05-11T13:59:11.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:11 smithi165 ceph-mon[33235]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-11T13:59:11.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:11 smithi165 ceph-mon[33235]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-11T13:59:11.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:11 smithi165 ceph-mon[33235]: Cluster is now healthy 2024-05-11T13:59:11.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:11 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/321425822' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-11T13:59:11.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:11 smithi165 ceph-mon[33235]: fsmap 12 up:standby 2024-05-11T13:59:12.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:12 smithi063 ceph-mon[29809]: pgmap v801: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:12.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:12 smithi071 ceph-mon[33293]: pgmap v801: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:12.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:12 smithi165 ceph-mon[33235]: pgmap v801: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 -- 172.21.15.63:0/3719696184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f841810ffb0 msgr2=0x7f84181123a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 --2- 172.21.15.63:0/3719696184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f841810ffb0 0x7f84181123a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f841000b0a0 tx=0x7f841000b3b0 comp rx=0 tx=0).stop 2024-05-11T13:59:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 -- 172.21.15.63:0/3719696184 shutdown_connections 2024-05-11T13:59:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 --2- 172.21.15.63:0/3719696184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f841810ffb0 0x7f84181123a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 --2- 172.21.15.63:0/3719696184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8418075780 0x7f8418075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 --2- 172.21.15.63:0/3719696184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84180760a0 0x7f841810fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 -- 172.21.15.63:0/3719696184 >> 172.21.15.63:0/3719696184 conn(0x7f841807a6b0 msgr2=0x7f841807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 -- 172.21.15.63:0/3719696184 shutdown_connections 2024-05-11T13:59:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.988+0000 7f841e600700 1 -- 172.21.15.63:0/3719696184 wait complete. 2024-05-11T13:59:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.989+0000 7f841e600700 1 Processor -- start 2024-05-11T13:59:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.989+0000 7f841e600700 1 -- start start 2024-05-11T13:59:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f841e600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8418075780 0x7f84181a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f841e600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84180760a0 0x7f84181a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f841e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f841810ffb0 0x7f841819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f841e600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84181157d0 con 0x7f8418075780 2024-05-11T13:59:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f841e600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8418115650 con 0x7f841810ffb0 2024-05-11T13:59:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f841e600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8418115950 con 0x7f84180760a0 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f8417400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8418075780 0x7f84181a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f8417400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8418075780 0x7f84181a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41114/0 (socket says 172.21.15.63:41114) 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f8417400700 1 -- 172.21.15.63:0/49182494 learned_addr learned my addr 172.21.15.63:0/49182494 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.990+0000 7f840ea00700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84180760a0 0x7f84181a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f8417e00700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f841810ffb0 0x7f841819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f8417400700 1 -- 172.21.15.63:0/49182494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84180760a0 msgr2=0x7f84181a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f8417400700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84180760a0 0x7f84181a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f8417400700 1 -- 172.21.15.63:0/49182494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f841810ffb0 msgr2=0x7f841819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f8417400700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f841810ffb0 0x7f841819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f8417400700 1 -- 172.21.15.63:0/49182494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841819c170 con 0x7f8418075780 2024-05-11T13:59:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f840ea00700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84180760a0 0x7f84181a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T13:59:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.991+0000 7f8417400700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8418075780 0x7f84181a1850 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f840800c9b0 tx=0x7f840800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.992+0000 7f8414c00700 1 -- 172.21.15.63:0/49182494 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408007b00 con 0x7f8418075780 2024-05-11T13:59:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.992+0000 7f8414c00700 1 -- 172.21.15.63:0/49182494 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8408007ca0 con 0x7f8418075780 2024-05-11T13:59:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.992+0000 7f8414c00700 1 -- 172.21.15.63:0/49182494 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408018660 con 0x7f8418075780 2024-05-11T13:59:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.992+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841819c460 con 0x7f8418075780 2024-05-11T13:59:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.992+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84181a8720 con 0x7f8418075780 2024-05-11T13:59:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.993+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841806a240 con 0x7f8418075780 2024-05-11T13:59:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.996+0000 7f8414c00700 1 -- 172.21.15.63:0/49182494 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f840800f450 con 0x7f8418075780 2024-05-11T13:59:13.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.998+0000 7f8414c00700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83fc0b2a70 0x7f83fc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:13.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.998+0000 7f8414c00700 1 -- 172.21.15.63:0/49182494 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9867+0+0 (secure 0 0 0) 0x7f84080e2c00 con 0x7f8418075780 2024-05-11T13:59:13.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.999+0000 7f840ea00700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83fc0b2a70 0x7f83fc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:13.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:13.999+0000 7f840ea00700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83fc0b2a70 0x7f83fc0b4f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8400005950 tx=0x7f84000058e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.003+0000 7f8414c00700 1 -- 172.21.15.63:0/49182494 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8408014030 con 0x7f8418075780 2024-05-11T13:59:14.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.251+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f841802e570 con 0x7f8418075780 2024-05-11T13:59:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.403+0000 7f8414c00700 1 -- 172.21.15.63:0/49182494 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) ==== 181+0+0 (secure 0 0 0) 0x7f84080aa040 con 0x7f8418075780 2024-05-11T13:59:14.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.409+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83fc0b2a70 msgr2=0x7f83fc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:14.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.409+0000 7f841e600700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83fc0b2a70 0x7f83fc0b4f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8400005950 tx=0x7f84000058e0 comp rx=0 tx=0).stop 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.409+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8418075780 msgr2=0x7f84181a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.409+0000 7f841e600700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8418075780 0x7f84181a1850 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f840800c9b0 tx=0x7f840800cd70 comp rx=0 tx=0).stop 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.410+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 shutdown_connections 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.410+0000 7f841e600700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f83fc0b2a70 0x7f83fc0b4f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.410+0000 7f841e600700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f841810ffb0 0x7f841819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.410+0000 7f841e600700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8418075780 0x7f84181a1850 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.410+0000 7f841e600700 1 --2- 172.21.15.63:0/49182494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84180760a0 0x7f84181a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.410+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 >> 172.21.15.63:0/49182494 conn(0x7f841807a6b0 msgr2=0x7f8418100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.411+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 shutdown_connections 2024-05-11T13:59:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:14.411+0000 7f841e600700 1 -- 172.21.15.63:0/49182494 wait complete. 2024-05-11T13:59:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_metadata' removed 2024-05-11T13:59:14.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:14 smithi063 ceph-mon[29809]: pgmap v802: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:14.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:14 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/49182494' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:14.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:14 smithi071 ceph-mon[33293]: pgmap v802: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:14.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:14 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/49182494' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:14.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:14 smithi165 ceph-mon[33235]: pgmap v802: 129 pgs: 129 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 29 op/s 2024-05-11T13:59:14.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:14 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/49182494' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:15.273 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T13:59:15.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:15 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/49182494' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:15.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:15 smithi063 ceph-mon[29809]: osdmap e131: 12 total, 12 up, 12 in 2024-05-11T13:59:15.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:15 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/49182494' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:15.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:15 smithi071 ceph-mon[33293]: osdmap e131: 12 total, 12 up, 12 in 2024-05-11T13:59:15.762 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:59:15.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:15 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/49182494' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:15.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:15 smithi165 ceph-mon[33235]: osdmap e131: 12 total, 12 up, 12 in 2024-05-11T13:59:16.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:16 smithi063 ceph-mon[29809]: pgmap v804: 97 pgs: 97 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:16.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:16 smithi071 ceph-mon[33293]: pgmap v804: 97 pgs: 97 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:16.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:16 smithi165 ceph-mon[33235]: pgmap v804: 97 pgs: 97 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.077+0000 7f92b9a00700 1 -- 172.21.15.63:0/346902430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b40758f0 msgr2=0x7f92b4075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.077+0000 7f92b9a00700 1 --2- 172.21.15.63:0/346902430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b40758f0 0x7f92b4075cd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009990 tx=0x7f92a4009ca0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.078+0000 7f92b9a00700 1 -- 172.21.15.63:0/346902430 shutdown_connections 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.078+0000 7f92b9a00700 1 --2- 172.21.15.63:0/346902430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92b4114490 0x7f92b4116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.078+0000 7f92b9a00700 1 --2- 172.21.15.63:0/346902430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b40758f0 0x7f92b4075cd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.078+0000 7f92b9a00700 1 --2- 172.21.15.63:0/346902430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92b4076210 0x7f92b4113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.078+0000 7f92b9a00700 1 -- 172.21.15.63:0/346902430 >> 172.21.15.63:0/346902430 conn(0x7f92b40fefd0 msgr2=0x7f92b41013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.078+0000 7f92b9a00700 1 -- 172.21.15.63:0/346902430 shutdown_connections 2024-05-11T13:59:18.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.078+0000 7f92b9a00700 1 -- 172.21.15.63:0/346902430 wait complete. 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 Processor -- start 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 -- start start 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92b40758f0 0x7f92b41a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92b4076210 0x7f92b41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b4114490 0x7f92b41a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4119b80 con 0x7f92b4114490 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4119a00 con 0x7f92b40758f0 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.079+0000 7f92b9a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4119d00 con 0x7f92b4076210 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b4114490 0x7f92b41a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b4114490 0x7f92b41a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41134/0 (socket says 172.21.15.63:41134) 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 -- 172.21.15.63:0/3891058806 learned_addr learned my addr 172.21.15.63:0/3891058806 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b3e00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92b40758f0 0x7f92b41a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:18.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b3400700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92b4076210 0x7f92b41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:18.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 -- 172.21.15.63:0/3891058806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92b4076210 msgr2=0x7f92b41a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:18.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92b4076210 0x7f92b41a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 -- 172.21.15.63:0/3891058806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92b40758f0 msgr2=0x7f92b41a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92b40758f0 0x7f92b41a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.080+0000 7f92b9000700 1 -- 172.21.15.63:0/3891058806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b41a50b0 con 0x7f92b4114490 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.081+0000 7f92b9000700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b4114490 0x7f92b41a4970 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0094f0 tx=0x7f92ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.081+0000 7f92b0c00700 1 -- 172.21.15.63:0/3891058806 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00ddd0 con 0x7f92b4114490 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.081+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41a53a0 con 0x7f92b4114490 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.081+0000 7f92b0c00700 1 -- 172.21.15.63:0/3891058806 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92ac009e20 con 0x7f92b4114490 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.081+0000 7f92b0c00700 1 -- 172.21.15.63:0/3891058806 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac015400 con 0x7f92b4114490 2024-05-11T13:59:18.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.081+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b41acc20 con 0x7f92b4114490 2024-05-11T13:59:18.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.082+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b406a240 con 0x7f92b4114490 2024-05-11T13:59:18.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.085+0000 7f92b0c00700 1 -- 172.21.15.63:0/3891058806 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f92ac005020 con 0x7f92b4114490 2024-05-11T13:59:18.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.086+0000 7f92b0c00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f92980b2a70 0x7f92980b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:18.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.086+0000 7f92b0c00700 1 -- 172.21.15.63:0/3891058806 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9313+0+0 (secure 0 0 0) 0x7f92ac0e2340 con 0x7f92b4114490 2024-05-11T13:59:18.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.087+0000 7f92b3e00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f92980b2a70 0x7f92980b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:18.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.087+0000 7f92b3e00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f92980b2a70 0x7f92980b4f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92a4006010 tx=0x7f92a400b540 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:18.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.091+0000 7f92b0c00700 1 -- 172.21.15.63:0/3891058806 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92ac010070 con 0x7f92b4114490 2024-05-11T13:59:18.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.334+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f92b402e570 con 0x7f92b4114490 2024-05-11T13:59:18.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.436+0000 7f92b0c00700 1 -- 172.21.15.63:0/3891058806 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) ==== 169+0+0 (secure 0 0 0) 0x7f92ac0a99a0 con 0x7f92b4114490 2024-05-11T13:59:18.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.441+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f92980b2a70 msgr2=0x7f92980b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:18.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.441+0000 7f92b9a00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f92980b2a70 0x7f92980b4f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92a4006010 tx=0x7f92a400b540 comp rx=0 tx=0).stop 2024-05-11T13:59:18.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.441+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b4114490 msgr2=0x7f92b41a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:18.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.441+0000 7f92b9a00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b4114490 0x7f92b41a4970 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0094f0 tx=0x7f92ac00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.443+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 shutdown_connections 2024-05-11T13:59:18.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.443+0000 7f92b9a00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f92980b2a70 0x7f92980b4f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.443+0000 7f92b9a00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b4114490 0x7f92b41a4970 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.443+0000 7f92b9a00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92b40758f0 0x7f92b41a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.443+0000 7f92b9a00700 1 --2- 172.21.15.63:0/3891058806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92b4076210 0x7f92b41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:18.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.443+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 >> 172.21.15.63:0/3891058806 conn(0x7f92b40fefd0 msgr2=0x7f92b41008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:18.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.445+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 shutdown_connections 2024-05-11T13:59:18.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:18.445+0000 7f92b9a00700 1 -- 172.21.15.63:0/3891058806 wait complete. 2024-05-11T13:59:18.436 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_data' removed 2024-05-11T13:59:18.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:18 smithi063 ceph-mon[29809]: pgmap v805: 97 pgs: 97 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:18.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:18 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3891058806' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:18.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:18 smithi071 ceph-mon[33293]: pgmap v805: 97 pgs: 97 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:18.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:18 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3891058806' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:18.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:18 smithi165 ceph-mon[33235]: pgmap v805: 97 pgs: 97 active+clean; 2.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:18.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:18 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3891058806' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:19.301 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-11T13:59:19.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:19 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/3891058806' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:19.671 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:19 smithi063 ceph-mon[29809]: osdmap e132: 12 total, 12 up, 12 in 2024-05-11T13:59:19.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:19 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/3891058806' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:19.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:19 smithi071 ceph-mon[33293]: osdmap e132: 12 total, 12 up, 12 in 2024-05-11T13:59:19.775 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T13:59:19.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:19 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/3891058806' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:19.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:19 smithi165 ceph-mon[33235]: osdmap e132: 12 total, 12 up, 12 in 2024-05-11T13:59:20.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:20 smithi071 ceph-mon[33293]: pgmap v807: 33 pgs: 33 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:20.690 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:20 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:59:20.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:20 smithi165 ceph-mon[33235]: pgmap v807: 33 pgs: 33 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:20.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:20 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:59:20.853 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:20 smithi063 ceph-mon[29809]: pgmap v807: 33 pgs: 33 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:20.853 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:20 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T13:59:21.524 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:21 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:59:21.525 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:21 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:59:21.525 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:21 smithi063 ceph-mon[29809]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:59:21.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:21 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:59:21.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:21 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:59:21.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:21 smithi165 ceph-mon[33235]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:59:21.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:21 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T13:59:21.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:21 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T13:59:21.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:21 smithi071 ceph-mon[33293]: from='mgr.15279 172.21.15.63:0/2263336788' entity='mgr.x' 2024-05-11T13:59:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.119+0000 7f879ac00700 1 -- 172.21.15.63:0/4260835826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 msgr2=0x7f879410bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.119+0000 7f879ac00700 1 --2- 172.21.15.63:0/4260835826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879410bb70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8784009990 tx=0x7f8784009ca0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.120+0000 7f879ac00700 1 -- 172.21.15.63:0/4260835826 shutdown_connections 2024-05-11T13:59:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.120+0000 7f879ac00700 1 --2- 172.21.15.63:0/4260835826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794106150 0x7f8794112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.120+0000 7f879ac00700 1 --2- 172.21.15.63:0/4260835826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879410bb70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.120+0000 7f879ac00700 1 --2- 172.21.15.63:0/4260835826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8794105790 0x7f8794105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.120+0000 7f879ac00700 1 -- 172.21.15.63:0/4260835826 >> 172.21.15.63:0/4260835826 conn(0x7f8794101280 msgr2=0x7f87941036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.120+0000 7f879ac00700 1 -- 172.21.15.63:0/4260835826 shutdown_connections 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.120+0000 7f879ac00700 1 -- 172.21.15.63:0/4260835826 wait complete. 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.121+0000 7f879ac00700 1 Processor -- start 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.121+0000 7f879ac00700 1 -- start start 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f879ac00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794105790 0x7f87941a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f879ac00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8794106150 0x7f87941a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f879ac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f879ac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8794115820 con 0x7f879410b790 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f879ac00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87941156a0 con 0x7f8794105790 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f879ac00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87941159a0 con 0x7f8794106150 2024-05-11T13:59:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8798c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8798c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879419bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44636/0 (socket says 172.21.15.63:44636) 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8798c00700 1 -- 172.21.15.63:0/220008091 learned_addr learned my addr 172.21.15.63:0/220008091 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8793e00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794105790 0x7f87941a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f878b400700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8794106150 0x7f87941a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8798c00700 1 -- 172.21.15.63:0/220008091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8794106150 msgr2=0x7f87941a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8798c00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8794106150 0x7f87941a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8798c00700 1 -- 172.21.15.63:0/220008091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794105790 msgr2=0x7f87941a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.122+0000 7f8798c00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794105790 0x7f87941a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.123+0000 7f8798c00700 1 -- 172.21.15.63:0/220008091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879419c260 con 0x7f879410b790 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.123+0000 7f8798c00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879419bb00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f878c007ee0 tx=0x7f878c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.123+0000 7f8791600700 1 -- 172.21.15.63:0/220008091 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c00dbd0 con 0x7f879410b790 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.123+0000 7f8791600700 1 -- 172.21.15.63:0/220008091 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f878c00dd70 con 0x7f879410b790 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.123+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879419c550 con 0x7f879410b790 2024-05-11T13:59:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.123+0000 7f8791600700 1 -- 172.21.15.63:0/220008091 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0058d0 con 0x7f879410b790 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.123+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941a87c0 con 0x7f879410b790 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.125+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879410dd20 con 0x7f879410b790 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.127+0000 7f8791600700 1 -- 172.21.15.63:0/220008091 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f878c01a020 con 0x7f879410b790 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.129+0000 7f8791600700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87780b29f0 0x7f87780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.129+0000 7f8791600700 1 -- 172.21.15.63:0/220008091 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 8754+0+0 (secure 0 0 0) 0x7f878c0e1e60 con 0x7f879410b790 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.129+0000 7f8793e00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87780b29f0 0x7f87780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.129+0000 7f8793e00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87780b29f0 0x7f87780b4eb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8794103360 tx=0x7f8784005960 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T13:59:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.133+0000 7f8791600700 1 -- 172.21.15.63:0/220008091 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f878c010070 con 0x7f879410b790 2024-05-11T13:59:22.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.365+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f879410d700 con 0x7f879410b790 2024-05-11T13:59:22.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.475+0000 7f8791600700 1 -- 172.21.15.63:0/220008091 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v133) ==== 178+0+0 (secure 0 0 0) 0x7f878c0a96f0 con 0x7f879410b790 2024-05-11T13:59:22.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87780b29f0 msgr2=0x7f87780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:22.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87780b29f0 0x7f87780b4eb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8794103360 tx=0x7f8784005960 comp rx=0 tx=0).stop 2024-05-11T13:59:22.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 msgr2=0x7f879419bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T13:59:22.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879419bb00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f878c007ee0 tx=0x7f878c00d3b0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 shutdown_connections 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:6800/3203389144,v1:172.21.15.63:6801/3203389144] conn(0x7f87780b29f0 0x7f87780b4eb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f879410b790 0x7f879419bb00 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794105790 0x7f87941a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 --2- 172.21.15.63:0/220008091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8794106150 0x7f87941a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.481+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 >> 172.21.15.63:0/220008091 conn(0x7f8794101280 msgr2=0x7f8794101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.482+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 shutdown_connections 2024-05-11T13:59:22.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-11T13:59:22.483+0000 7f879ac00700 1 -- 172.21.15.63:0/220008091 wait complete. 2024-05-11T13:59:22.473 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_data_ec' removed 2024-05-11T13:59:22.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:22 smithi165 ceph-mon[33235]: pgmap v808: 33 pgs: 33 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:22.803 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:22 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/220008091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:22.864 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:22 smithi063 ceph-mon[29809]: pgmap v808: 33 pgs: 33 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:22.864 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:22 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/220008091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:22.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:22 smithi071 ceph-mon[33293]: pgmap v808: 33 pgs: 33 active+clean; 2.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:22.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:22 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/220008091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T13:59:23.552 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T13:59:23.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T13:59:23.753 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:23 smithi071 ceph-mon[33293]: from='client.? 172.21.15.63:0/220008091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:23.753 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:23 smithi071 ceph-mon[33293]: osdmap e133: 12 total, 12 up, 12 in 2024-05-11T13:59:23.777 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T13:59:23.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:23 smithi165 ceph-mon[33235]: from='client.? 172.21.15.63:0/220008091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:23.807 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:23 smithi165 ceph-mon[33235]: osdmap e133: 12 total, 12 up, 12 in 2024-05-11T13:59:23.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:23 smithi063 ceph-mon[29809]: from='client.? 172.21.15.63:0/220008091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T13:59:23.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:23 smithi063 ceph-mon[29809]: osdmap e133: 12 total, 12 up, 12 in 2024-05-11T13:59:23.950 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-11T13:59:23.950 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-11T13:59:24.013 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-11T13:59:24.063 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-11T13:59:24.116 INFO:tasks.cephadm:Stopping all daemons... 2024-05-11T13:59:24.117 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-11T13:59:24.117 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a 2024-05-11T13:59:24.473 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:24 smithi063 systemd[1]: Stopping Ceph mon.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:24.733 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:24 smithi063 ceph-mon[29809]: pgmap v810: 1 pgs: 1 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:24.733 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:24 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a[29805]: 2024-05-11T13:59:24.683+0000 7f3425800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:59:24.733 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:24 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a[29805]: 2024-05-11T13:59:24.683+0000 7f3425800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-11T13:59:24.802 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:24 smithi165 ceph-mon[33235]: pgmap v810: 1 pgs: 1 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:24.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:24 smithi071 ceph-mon[33293]: pgmap v810: 1 pgs: 1 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T13:59:25.064 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:24 smithi063 podman[115628]: 2024-05-11 13:59:24.746188999 +0000 UTC m=+0.223691358 container died d455117ba008f0b2924b5cb0a0290d1fdd607316082ec64de728a46d41012be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:59:25.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:25 smithi063 podman[115628]: 2024-05-11 13:59:25.219217021 +0000 UTC m=+0.696719403 container cleanup d455117ba008f0b2924b5cb0a0290d1fdd607316082ec64de728a46d41012be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-11T13:59:25.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:25 smithi063 bash[115628]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a 2024-05-11T13:59:25.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 11 13:59:25 smithi063 podman[115640]: 2024-05-11 13:59:25.587245216 +0000 UTC m=+0.823365417 container remove d455117ba008f0b2924b5cb0a0290d1fdd607316082ec64de728a46d41012be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-11T13:59:25.850 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.a.service' 2024-05-11T13:59:25.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:25.985 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-11T13:59:25.985 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-11T13:59:25.985 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.b 2024-05-11T13:59:26.440 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:26 smithi071 systemd[1]: Stopping Ceph mon.b for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:26.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:26 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b[33289]: 2024-05-11T13:59:26.694+0000 7f6bcea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:59:26.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:26 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b[33289]: 2024-05-11T13:59:26.694+0000 7f6bcea00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-11T13:59:26.941 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:26 smithi071 podman[70465]: 2024-05-11 13:59:26.79744398 +0000 UTC m=+0.268575618 container died 48814d2bd8db14edce7d0ec0bb86d4244d10ffac176bbacaa22dd6e88ed71275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux ) 2024-05-11T13:59:27.440 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:27 smithi071 podman[70465]: 2024-05-11 13:59:27.226951758 +0000 UTC m=+0.698083389 container cleanup 48814d2bd8db14edce7d0ec0bb86d4244d10ffac176bbacaa22dd6e88ed71275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:59:27.441 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:27 smithi071 bash[70465]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b 2024-05-11T13:59:27.940 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 11 13:59:27 smithi071 podman[70479]: 2024-05-11 13:59:27.503463865 +0000 UTC m=+0.694838461 container remove 48814d2bd8db14edce7d0ec0bb86d4244d10ffac176bbacaa22dd6e88ed71275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-b, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:59:28.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.b.service' 2024-05-11T13:59:28.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:28.136 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-11T13:59:28.136 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-11T13:59:28.136 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.c 2024-05-11T13:59:28.552 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:28 smithi165 systemd[1]: Stopping Ceph mon.c for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:29.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c[33212]: 2024-05-11T13:59:29.117+0000 7fe5fdc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:59:29.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c[33212]: 2024-05-11T13:59:29.117+0000 7fe5fdc00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-11T13:59:29.553 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:29 smithi165 podman[75601]: 2024-05-11 13:59:29.282976694 +0000 UTC m=+0.636051116 container died a0e327cdd6b074b910834abbfcc18dfa31cb407673967d2ecc17c8367a4b16a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:59:30.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:29 smithi165 podman[75601]: 2024-05-11 13:59:29.79609302 +0000 UTC m=+1.149167482 container cleanup a0e327cdd6b074b910834abbfcc18dfa31cb407673967d2ecc17c8367a4b16a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:59:30.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:29 smithi165 bash[75601]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c 2024-05-11T13:59:31.052 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 11 13:59:30 smithi165 podman[75615]: 2024-05-11 13:59:30.573398684 +0000 UTC m=+1.208594622 container remove a0e327cdd6b074b910834abbfcc18dfa31cb407673967d2ecc17c8367a4b16a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-11T13:59:31.102 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mon.c.service' 2024-05-11T13:59:31.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:31.212 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-11T13:59:31.213 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-11T13:59:31.213 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.x 2024-05-11T13:59:31.671 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:59:31 smithi063 systemd[1]: Stopping Ceph mgr.x for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:32.171 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:59:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:59:31.839+0000 7f17b7a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:59:32.171 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:59:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x[30038]: 2024-05-11T13:59:31.839+0000 7f17b7a00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-11T13:59:32.172 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:59:31 smithi063 podman[115744]: 2024-05-11 13:59:31.94351556 +0000 UTC m=+0.242307877 container died a5aa873b8ad4258bf72651b5ea8ae131be23601e25ed6a5092f547c52fa480b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-11T13:59:32.921 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:59:32 smithi063 podman[115744]: 2024-05-11 13:59:32.808643478 +0000 UTC m=+1.107435811 container cleanup a5aa873b8ad4258bf72651b5ea8ae131be23601e25ed6a5092f547c52fa480b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-11T13:59:32.921 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:59:32 smithi063 bash[115744]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x 2024-05-11T13:59:33.421 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 11 13:59:33 smithi063 podman[115758]: 2024-05-11 13:59:33.060381345 +0000 UTC m=+1.108631693 container remove a5aa873b8ad4258bf72651b5ea8ae131be23601e25ed6a5092f547c52fa480b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True) 2024-05-11T13:59:33.591 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-11T13:59:33.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:33.711 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-11T13:59:33.711 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-11T13:59:33.711 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.y 2024-05-11T13:59:34.190 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:59:33 smithi071 systemd[1]: Stopping Ceph mgr.y for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:34.690 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:59:34 smithi071 podman[70588]: 2024-05-11 13:59:34.428228728 +0000 UTC m=+0.218737316 container died bdd917e054d80db94f8c82d2b3bd1fc8859cee87ba7540f9418733b710e49a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T13:59:35.191 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:59:34 smithi071 podman[70588]: 2024-05-11 13:59:34.906497354 +0000 UTC m=+0.697005931 container cleanup bdd917e054d80db94f8c82d2b3bd1fc8859cee87ba7540f9418733b710e49a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-11T13:59:35.191 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:59:34 smithi071 bash[70588]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y 2024-05-11T13:59:35.940 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:59:35 smithi071 podman[70602]: 2024-05-11 13:59:35.566522519 +0000 UTC m=+1.119022440 container remove bdd917e054d80db94f8c82d2b3bd1fc8859cee87ba7540f9418733b710e49a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-11T13:59:35.940 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 11 13:59:35 smithi071 systemd[1]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-11T13:59:36.112 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-11T13:59:36.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:36.215 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-11T13:59:36.215 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-11T13:59:36.216 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.z 2024-05-11T13:59:36.553 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:59:36 smithi165 systemd[1]: Stopping Ceph mgr.z for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:37.197 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:59:36 smithi165 podman[75733]: 2024-05-11 13:59:36.890069684 +0000 UTC m=+0.215927260 container died 2186da7408dd913de20435c42f229dd7d9acad00197fe2530ed71a5c007b7ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-11T13:59:37.552 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:59:37 smithi165 podman[75733]: 2024-05-11 13:59:37.386548869 +0000 UTC m=+0.712406428 container cleanup 2186da7408dd913de20435c42f229dd7d9acad00197fe2530ed71a5c007b7ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T13:59:37.552 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:59:37 smithi165 bash[75733]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z 2024-05-11T13:59:38.052 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:59:37 smithi165 podman[75747]: 2024-05-11 13:59:37.654839944 +0000 UTC m=+0.739189271 container remove 2186da7408dd913de20435c42f229dd7d9acad00197fe2530ed71a5c007b7ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T13:59:38.053 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 11 13:59:37 smithi165 systemd[1]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-11T13:59:38.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-11T13:59:38.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:38.334 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-11T13:59:38.334 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-11T13:59:38.337 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.0 2024-05-11T13:59:38.671 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:38 smithi063 systemd[1]: Stopping Ceph osd.0 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:39.671 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0[41949]: 2024-05-11T13:59:39.254+0000 7f72f9e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:59:39.671 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0[41949]: 2024-05-11T13:59:39.254+0000 7f72f9e00700 -1 osd.0 133 *** Got signal Terminated *** 2024-05-11T13:59:39.671 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0[41949]: 2024-05-11T13:59:39.254+0000 7f72f9e00700 -1 osd.0 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T13:59:45.421 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:45 smithi063 podman[115865]: 2024-05-11 13:59:45.166592199 +0000 UTC m=+6.395444269 container died 84e49e906380c15d3834a3e76738de463578cfc01ec38b56f845c7cfd7e6c831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, CEPH_POINT_RELEASE=) 2024-05-11T13:59:46.036 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:45 smithi063 podman[115865]: 2024-05-11 13:59:45.831212453 +0000 UTC m=+7.060064560 container cleanup 84e49e906380c15d3834a3e76738de463578cfc01ec38b56f845c7cfd7e6c831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=) 2024-05-11T13:59:46.036 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:45 smithi063 bash[115865]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0 2024-05-11T13:59:46.421 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:46 smithi063 podman[115884]: 2024-05-11 13:59:46.049085316 +0000 UTC m=+0.734004369 container remove 84e49e906380c15d3834a3e76738de463578cfc01ec38b56f845c7cfd7e6c831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-11T13:59:47.171 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:46 smithi063 podman[115971]: 2024-05-11 13:59:46.823746632 +0000 UTC m=+0.038828077 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:59:47.171 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:46 smithi063 podman[115971]: 2024-05-11 13:59:46.958524326 +0000 UTC m=+0.173605808 container create b8755d828fcbce2c141ba61a2c0dbccf5c5bde1043d53109ff9f3236635c66dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T13:59:47.671 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:47 smithi063 podman[115971]: 2024-05-11 13:59:47.220995814 +0000 UTC m=+0.436077272 container init b8755d828fcbce2c141ba61a2c0dbccf5c5bde1043d53109ff9f3236635c66dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-11T13:59:47.671 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:47 smithi063 podman[115971]: 2024-05-11 13:59:47.234044507 +0000 UTC m=+0.449125962 container start b8755d828fcbce2c141ba61a2c0dbccf5c5bde1043d53109ff9f3236635c66dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:59:47.671 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:47 smithi063 podman[115971]: 2024-05-11 13:59:47.352672704 +0000 UTC m=+0.567754164 container attach b8755d828fcbce2c141ba61a2c0dbccf5c5bde1043d53109ff9f3236635c66dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-11T13:59:48.421 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 11 13:59:48 smithi063 podman[115971]: 2024-05-11 13:59:48.068294905 +0000 UTC m=+1.283376399 container died b8755d828fcbce2c141ba61a2c0dbccf5c5bde1043d53109ff9f3236635c66dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-11T13:59:48.883 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.0.service' 2024-05-11T13:59:48.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:48.996 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-11T13:59:48.996 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-11T13:59:48.996 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.1 2024-05-11T13:59:49.440 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:49 smithi071 systemd[1]: Stopping Ceph osd.1 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:49.940 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1[37852]: 2024-05-11T13:59:49.565+0000 7f37b3200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:59:49.940 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1[37852]: 2024-05-11T13:59:49.565+0000 7f37b3200700 -1 osd.1 133 *** Got signal Terminated *** 2024-05-11T13:59:49.941 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1[37852]: 2024-05-11T13:59:49.565+0000 7f37b3200700 -1 osd.1 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T13:59:55.441 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:55 smithi071 podman[70707]: 2024-05-11 13:59:55.155400091 +0000 UTC m=+5.743485178 container died e5f5311e5f5b148c9c26c6db482ed4ac493b431afff4496a0d5fee6c3e6ef3fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True) 2024-05-11T13:59:55.847 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:55 smithi071 podman[70707]: 2024-05-11 13:59:55.583970735 +0000 UTC m=+6.172055849 container cleanup e5f5311e5f5b148c9c26c6db482ed4ac493b431afff4496a0d5fee6c3e6ef3fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-11T13:59:55.847 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:55 smithi071 bash[70707]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1 2024-05-11T13:59:56.190 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:55 smithi071 podman[70735]: 2024-05-11 13:59:55.860071172 +0000 UTC m=+0.678111027 container remove e5f5311e5f5b148c9c26c6db482ed4ac493b431afff4496a0d5fee6c3e6ef3fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T13:59:56.941 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:56 smithi071 podman[70821]: 2024-05-11 13:59:56.590952722 +0000 UTC m=+0.036225093 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T13:59:56.941 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:56 smithi071 podman[70821]: 2024-05-11 13:59:56.702788341 +0000 UTC m=+0.148060749 container create 8a7cff271a7a476ad5d399a1aa688d30a628de050b42024b6dce4d2a3ed0fbb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T13:59:57.441 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:56 smithi071 podman[70821]: 2024-05-11 13:59:56.998405717 +0000 UTC m=+0.443678128 container init 8a7cff271a7a476ad5d399a1aa688d30a628de050b42024b6dce4d2a3ed0fbb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , ceph=True) 2024-05-11T13:59:57.441 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:57 smithi071 podman[70821]: 2024-05-11 13:59:57.009319057 +0000 UTC m=+0.454591468 container start 8a7cff271a7a476ad5d399a1aa688d30a628de050b42024b6dce4d2a3ed0fbb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-11T13:59:57.441 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:57 smithi071 podman[70821]: 2024-05-11 13:59:57.13050237 +0000 UTC m=+0.575774834 container attach 8a7cff271a7a476ad5d399a1aa688d30a628de050b42024b6dce4d2a3ed0fbb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-11T13:59:58.190 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 11 13:59:57 smithi071 podman[70821]: 2024-05-11 13:59:57.919256115 +0000 UTC m=+1.364528525 container died 8a7cff271a7a476ad5d399a1aa688d30a628de050b42024b6dce4d2a3ed0fbb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-11T13:59:58.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.1.service' 2024-05-11T13:59:58.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T13:59:58.752 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-11T13:59:58.753 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-11T13:59:58.753 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.2 2024-05-11T13:59:59.303 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:59:58 smithi165 systemd[1]: Stopping Ceph osd.2 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T13:59:59.802 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:59:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2[38393]: 2024-05-11T13:59:59.379+0000 7f75f8e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T13:59:59.803 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:59:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2[38393]: 2024-05-11T13:59:59.379+0000 7f75f8e00700 -1 osd.2 133 *** Got signal Terminated *** 2024-05-11T13:59:59.803 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 13:59:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2[38393]: 2024-05-11T13:59:59.379+0000 7f75f8e00700 -1 osd.2 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:00:05.303 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:05 smithi165 podman[75853]: 2024-05-11 14:00:05.050302807 +0000 UTC m=+5.847464814 container died fab6e274f9e78428a8ac02ada9d5914268fe753d3cd7fb4578f2588b5aa7a80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-11T14:00:05.695 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:05 smithi165 podman[75853]: 2024-05-11 14:00:05.490036803 +0000 UTC m=+6.287198806 container cleanup fab6e274f9e78428a8ac02ada9d5914268fe753d3cd7fb4578f2588b5aa7a80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:00:05.695 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:05 smithi165 bash[75853]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2 2024-05-11T14:00:06.052 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:05 smithi165 podman[75882]: 2024-05-11 14:00:05.708418544 +0000 UTC m=+0.627029035 container remove fab6e274f9e78428a8ac02ada9d5914268fe753d3cd7fb4578f2588b5aa7a80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-11T14:00:06.128 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:05.857+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:06.129 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:05.697+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:06.421 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:06.140+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:06.691 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:06.203+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:06.803 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:06 smithi165 podman[75970]: 2024-05-11 14:00:06.449008311 +0000 UTC m=+0.032997061 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:00:06.803 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:06 smithi165 podman[75970]: 2024-05-11 14:00:06.568089788 +0000 UTC m=+0.152078575 container create 0ff0340ae60a85db0fd551f82b0c00bfd2d660916ba6baafe038039697cd4462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T14:00:06.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:06.614+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:06.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:06.555+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:07.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:06.822+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:07.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:06.721+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:07.302 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:06 smithi165 podman[75970]: 2024-05-11 14:00:06.880929003 +0000 UTC m=+0.464917797 container init 0ff0340ae60a85db0fd551f82b0c00bfd2d660916ba6baafe038039697cd4462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:00:07.302 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:06 smithi165 podman[75970]: 2024-05-11 14:00:06.891907883 +0000 UTC m=+0.475896677 container start 0ff0340ae60a85db0fd551f82b0c00bfd2d660916ba6baafe038039697cd4462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=) 2024-05-11T14:00:07.302 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:07 smithi165 podman[75970]: 2024-05-11 14:00:07.030110311 +0000 UTC m=+0.614099148 container attach 0ff0340ae60a85db0fd551f82b0c00bfd2d660916ba6baafe038039697cd4462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-11T14:00:07.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:07.195+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:07.670 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:07.188+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:07.793 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 11 14:00:07 smithi165 podman[75970]: 2024-05-11 14:00:07.437638236 +0000 UTC m=+1.021627030 container died 0ff0340ae60a85db0fd551f82b0c00bfd2d660916ba6baafe038039697cd4462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-11T14:00:07.794 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:07 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:07.519+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:07.794 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:07 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:07.565+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:08.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:07.862+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:08.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:07.710+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:08.230 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.2.service' 2024-05-11T14:00:08.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:00:08.347 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-11T14:00:08.347 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-11T14:00:08.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.3 2024-05-11T14:00:08.429 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:08.190+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:08.442 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:08.162+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:08.692 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:08 smithi063 systemd[1]: Stopping Ceph osd.3 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:00:09.054 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:08.566+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:09.054 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:08.567+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:09.054 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:08.678+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:09.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:08.705+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:09.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:08.873+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:09.171 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:08.883+0000 7f5c89c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:00:09.171 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:08.883+0000 7f5c89c00700 -1 osd.3 133 *** Got signal Terminated *** 2024-05-11T14:00:09.172 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:08.883+0000 7f5c89c00700 -1 osd.3 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:00:09.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:09.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:09.537 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:09.206+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:09.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:09.552+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:09.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:09.518+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:09.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:09.669+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:09.884 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:09.737+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:10.170 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:09.896+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:10.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:10.184+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:10.671 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:10.191+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:10.802 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:10.628+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:10.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:10.512+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:10.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:10.558+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:11.133 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:10.877+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:11.133 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:10.705+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:11.420 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:11.145+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:11.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:11.141+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:11.802 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:11.592+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:11.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:11.514+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:11.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:11.540+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:12.101 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:11.753+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:12.102 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:11.848+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:12.421 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:12.114+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:12.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:12.137+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:12.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:12.500+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:12.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:12.583+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:12.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:12.558+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:13.063 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:12.893+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:13.063 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:12.789+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:13.421 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:13 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3[48432]: 2024-05-11T14:00:13.076+0000 7f5c85400700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:40.565201+0000 front 2024-05-11T13:59:40.565061+0000 (oldest deadline 2024-05-11T14:00:05.265297+0000) 2024-05-11T14:00:13.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:13.175+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:13.802 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:13.586+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:13.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:13.605+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:13.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:13.485+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:14.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:13 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:13.762+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:14.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:13 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:13.857+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:14.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:14.186+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:14.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:14.596+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:14.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:14.449+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:14.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:14.607+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:15.106 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:14 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:14.902+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:15.107 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:14 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:14.797+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:15.107 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:14 smithi063 podman[116094]: 2024-05-11 14:00:14.777213532 +0000 UTC m=+6.032328177 container died b3d1adf47e1d3a16e0932147dca2c69a49b6e7156365a2f421015a4b2e32351b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-11T14:00:15.421 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:15 smithi063 podman[116094]: 2024-05-11 14:00:15.301947058 +0000 UTC m=+6.557061743 container cleanup b3d1adf47e1d3a16e0932147dca2c69a49b6e7156365a2f421015a4b2e32351b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-11T14:00:15.421 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:15 smithi063 bash[116094]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3 2024-05-11T14:00:15.690 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:15.208+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:15.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:15.585+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:15.802 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:15.575+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:15.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:15.419+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:15.840 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:15 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:15.785+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:15.840 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:15 smithi063 podman[116115]: 2024-05-11 14:00:15.578297933 +0000 UTC m=+0.786483829 container remove b3d1adf47e1d3a16e0932147dca2c69a49b6e7156365a2f421015a4b2e32351b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=) 2024-05-11T14:00:16.170 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:15 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:15.852+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:16.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:16.168+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:16.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:16.597+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:16.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:16.578+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:16.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:16.448+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:16.811 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:16 smithi063 podman[116201]: 2024-05-11 14:00:16.411002798 +0000 UTC m=+0.035084642 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:00:16.811 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:16 smithi063 podman[116201]: 2024-05-11 14:00:16.520734621 +0000 UTC m=+0.144816487 container create be5015a9e4b66f1aec3dfd8bdaef0a64916bfa34108809fb1c54c461c657ba9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:00:16.811 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:16 smithi063 podman[116201]: 2024-05-11 14:00:16.824220162 +0000 UTC m=+0.448302074 container init be5015a9e4b66f1aec3dfd8bdaef0a64916bfa34108809fb1c54c461c657ba9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, ceph=True) 2024-05-11T14:00:17.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:16 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:16.831+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:17.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:16 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:16.837+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:17.171 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:16 smithi063 podman[116201]: 2024-05-11 14:00:16.83582674 +0000 UTC m=+0.459908613 container start be5015a9e4b66f1aec3dfd8bdaef0a64916bfa34108809fb1c54c461c657ba9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-11T14:00:17.171 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:16 smithi063 podman[116201]: 2024-05-11 14:00:16.931455087 +0000 UTC m=+0.555537036 container attach be5015a9e4b66f1aec3dfd8bdaef0a64916bfa34108809fb1c54c461c657ba9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T14:00:17.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:17.138+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:17.670 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 11 14:00:17 smithi063 podman[116201]: 2024-05-11 14:00:17.309047676 +0000 UTC m=+0.933129615 container died be5015a9e4b66f1aec3dfd8bdaef0a64916bfa34108809fb1c54c461c657ba9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-11T14:00:17.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:17.645+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:17.413+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:17.413+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:17.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:17.587+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:17.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:17.587+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:17.921 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:17 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:17.832+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:17.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:17 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:17.843+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:17.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:17 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:17.843+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:18.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.3.service' 2024-05-11T14:00:18.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:00:18.200 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-11T14:00:18.200 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-11T14:00:18.200 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.4 2024-05-11T14:00:18.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:18.112+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:18.440 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:18 smithi071 systemd[1]: Stopping Ceph osd.4 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:00:18.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:18.633+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:18.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:18.448+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:18.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:18.448+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:18.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:18.559+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:18.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:18.559+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:18.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:18.632+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:18.941 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:18.767+0000 7f3ab6000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:00:18.941 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:18.767+0000 7f3ab6000700 -1 osd.4 133 *** Got signal Terminated *** 2024-05-11T14:00:18.941 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:18.767+0000 7f3ab6000700 -1 osd.4 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:00:18.941 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:18.824+0000 7f3aac800700 -1 osd.4 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.311744+0000 front 2024-05-11T13:59:54.311940+0000 (oldest deadline 2024-05-11T14:00:18.412023+0000) 2024-05-11T14:00:19.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:18 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:18.867+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:19.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:18 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:18.867+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:19.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:18 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:18.848+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:19.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:18 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:18.848+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:19.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:19.137+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:19.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:19.655+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:19.802 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:19.525+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:19.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:19.525+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:19.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:19.437+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:19.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:19.437+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:19.830 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:19.595+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:19.849 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:19 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:19.860+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:19.849 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:19 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:19.860+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:20.158 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:19.843+0000 7f3aac800700 -1 osd.4 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.311744+0000 front 2024-05-11T13:59:54.311940+0000 (oldest deadline 2024-05-11T14:00:18.412023+0000) 2024-05-11T14:00:20.170 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:19 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:19.905+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:20.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:19 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:19.905+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:20.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:20.171+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:20.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:20.171+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:20.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:20.658+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:20.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:20.556+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:20.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:20.556+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:20.435+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:20.435+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:20.824 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:20.596+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:21.141 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:20.872+0000 7f3aac800700 -1 osd.4 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.311744+0000 front 2024-05-11T13:59:54.311940+0000 (oldest deadline 2024-05-11T14:00:18.412023+0000) 2024-05-11T14:00:21.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:20 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:20.862+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:21.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:20 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:20.862+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:21.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:20 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:20.859+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:21.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:20 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:20.859+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:21.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:21.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:21.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:21.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:21.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:21.628+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:21.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:21.525+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:21.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:21.525+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:21.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:21.423+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:21.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:21.423+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:21.856 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:21.548+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:22.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:21 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:21.843+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:22.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:21 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:21.843+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:22.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:21 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:21.909+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:22.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:21 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:21.909+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:22.190 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:21.869+0000 7f3aac800700 -1 osd.4 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.311744+0000 front 2024-05-11T13:59:54.311940+0000 (oldest deadline 2024-05-11T14:00:18.412023+0000) 2024-05-11T14:00:22.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:22 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:22.115+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:22.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:22 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:22.115+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:22.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:22.593+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:22.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:22.499+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:22.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:22.499+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:22.393+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:22.393+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:22.891 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:22 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:22.585+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:23.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:22 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:22.861+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:23.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:22 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:22.861+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:23.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:22 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:22.960+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:23.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:22 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:22.960+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:23.190 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:22 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4[44345]: 2024-05-11T14:00:22.903+0000 7f3aac800700 -1 osd.4 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.311744+0000 front 2024-05-11T13:59:54.311940+0000 (oldest deadline 2024-05-11T14:00:18.412023+0000) 2024-05-11T14:00:23.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:23 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:23.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:23.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:23 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:23.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:23.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:23.548+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:23.802 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:23.527+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:23.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:23.527+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:23.396+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:23.396+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:23.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:23 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:23.615+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:24.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:23 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:23.834+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:24.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:23 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:23.834+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:24.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:23 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:23.911+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:24.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:23 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:23.911+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:24.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:24 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:24.155+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:24.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:24 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:24.155+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:24.749 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:24 smithi071 podman[70924]: 2024-05-11 14:00:24.469376215 +0000 UTC m=+5.847303110 container died 325ebabc81dfdb7f5f9daa2fdb42ca1f66a5ef3da689de1eba1716e24b8d77ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T14:00:24.749 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:24 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:24.594+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:24.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:24.568+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:24.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:24.443+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:24.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:24.443+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:24.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:24.512+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:24.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:24.512+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:25.099 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:24 smithi071 podman[70924]: 2024-05-11 14:00:24.909894513 +0000 UTC m=+6.287821422 container cleanup 325ebabc81dfdb7f5f9daa2fdb42ca1f66a5ef3da689de1eba1716e24b8d77ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-11T14:00:25.099 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:24 smithi071 bash[70924]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4 2024-05-11T14:00:25.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:24 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:24.811+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:25.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:24 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:24.811+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:25.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:24 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:24.888+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:25.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:24 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:24.888+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:25.440 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:25 smithi071 podman[70956]: 2024-05-11 14:00:25.21126576 +0000 UTC m=+0.727105214 container remove 325ebabc81dfdb7f5f9daa2fdb42ca1f66a5ef3da689de1eba1716e24b8d77ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:00:25.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:25 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:25.110+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:25.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:25 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:25.110+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:25.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:25.557+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:25.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:25.415+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:25.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:25.415+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:25.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:25.467+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:25.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:25.467+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:25.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:25.467+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:25.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:25 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:25.615+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:26.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:25 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:25.842+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:26.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:25 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:25.842+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:26.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:25 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:25.927+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:26.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:25 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:25.927+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:26.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:25 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:25.927+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:26.440 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:26 smithi071 podman[71044]: 2024-05-11 14:00:26.013276986 +0000 UTC m=+0.039199224 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:00:26.440 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:26 smithi071 podman[71044]: 2024-05-11 14:00:26.15387089 +0000 UTC m=+0.179793154 container create 7f2bb17c0975381d6641317713ac845253641881a44f2fdf43926c3dc2cd48ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-11T14:00:26.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:26 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:26.096+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:26.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:26 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:26.096+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:26.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:26.438+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:26.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:26.438+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:26.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:26.603+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:26.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:26.603+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:26.804 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:26.483+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:26.804 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:26.483+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:26.804 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:26.483+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:26.923 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:26 smithi071 podman[71044]: 2024-05-11 14:00:26.473976863 +0000 UTC m=+0.499899134 container init 7f2bb17c0975381d6641317713ac845253641881a44f2fdf43926c3dc2cd48ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:00:26.923 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:26 smithi071 podman[71044]: 2024-05-11 14:00:26.485084993 +0000 UTC m=+0.511007300 container start 7f2bb17c0975381d6641317713ac845253641881a44f2fdf43926c3dc2cd48ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:00:26.923 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:26 smithi071 podman[71044]: 2024-05-11 14:00:26.589695794 +0000 UTC m=+0.615618102 container attach 7f2bb17c0975381d6641317713ac845253641881a44f2fdf43926c3dc2cd48ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-11T14:00:26.923 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 11 14:00:26 smithi071 podman[71044]: 2024-05-11 14:00:26.93586284 +0000 UTC m=+0.961785124 container died 7f2bb17c0975381d6641317713ac845253641881a44f2fdf43926c3dc2cd48ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-11T14:00:26.924 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:26 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:26.639+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:26.924 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:26 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:26.639+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:27.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:26 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:26.832+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:27.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:26 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:26.832+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:27.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:26 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:26.967+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:27.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:26 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:26.967+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:27.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:26 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:26.967+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:27.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:27 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:27.112+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:27.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:27 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:27.112+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:27.679 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.4.service' 2024-05-11T14:00:27.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:00:27.779 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-11T14:00:27.780 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-11T14:00:27.780 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.5 2024-05-11T14:00:27.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:27.619+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:27.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:27.619+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:27.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:27.438+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:27.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:27.438+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:27.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:27.438+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:27.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:27.448+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:27.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:27.448+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:27.803 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:27.448+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:27.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:27 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:27.649+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:27.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:27 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:27.649+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:28.092 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:27 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:27.954+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:28.093 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:27 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:27.954+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:28.093 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:27 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:27.954+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:28.093 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:27 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:27.855+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:28.093 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:27 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:27.855+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:28.302 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:27 smithi165 systemd[1]: Stopping Ceph osd.5 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:00:28.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:28 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:28.116+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:28.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:28 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:28.116+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:28.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:28 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:28.116+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:28.635 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:28.367+0000 7fa481200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:00:28.635 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:28.367+0000 7fa481200700 -1 osd.5 133 *** Got signal Terminated *** 2024-05-11T14:00:28.635 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:28.367+0000 7fa481200700 -1 osd.5 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:00:28.635 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:28.477+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:28.636 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:28.477+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:28.636 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:28.477+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:28.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:28.421+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:28.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:28.421+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:28.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:28.421+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:28.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:28 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:28.643+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:28.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:28 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:28.643+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:29.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:28.646+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:29.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:28.646+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:29.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:28 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:28.808+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:29.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:28 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:28.808+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:29.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:28 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:28.808+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:29.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:28 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:28.927+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:29.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:28 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:28.927+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:29.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:28 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:28.927+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:29.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:29 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:29.141+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:29.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:29 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:29.141+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:29.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:29 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:29.141+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:29.671 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:29.506+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:29.672 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:29.506+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:29.672 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:29.506+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:29.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:29.408+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:29.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:29.408+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:29.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:29.408+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:29.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:29 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:29.667+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:29.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:29 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:29.667+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:30.045 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:29.684+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:30.045 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:29.684+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:30.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:29.763+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:30.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:29.763+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:30.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:29.763+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:30.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:29.942+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:30.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:29.942+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:30.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:29 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:29.942+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:30.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:30 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:30.122+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:30.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:30 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:30.122+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:30.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:30 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:30.122+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:30.682 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:30.495+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:30.682 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:30.495+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:30.682 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:30.495+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:30.683 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:30.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:30.683 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:30.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:30.683 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:30.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:30.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:30 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:30.618+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:30.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:30 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:30.618+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:31.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:30.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:31.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:30.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:31.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:30.803+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:31.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:30.803+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:31.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:30.803+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:31.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:30.955+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:31.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:30.955+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:31.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:30 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:30.955+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:31.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:31 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:31.109+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:31.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:31 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:31.109+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:31.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:31 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:31.109+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:31.704 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:31.509+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:31.704 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:31.509+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:31.704 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:31.509+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:31.704 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:31.407+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:31.704 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:31.407+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:31.704 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:31.407+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:31.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:31 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:31.580+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:31.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:31 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:31.580+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:32.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:31.716+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:32.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:31.716+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:32.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:31.802+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:32.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:31.802+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:32.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:31.802+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:32.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:31.984+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:32.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:31.984+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:32.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:31 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:31.984+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:32.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:32 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:32.137+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:32.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:32 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:32.137+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:32.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:32 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:32.137+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:32.654 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:32.531+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.439645+0000 front 2024-05-11T13:59:43.439726+0000 (oldest deadline 2024-05-11T14:00:08.640708+0000) 2024-05-11T14:00:32.655 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:32.531+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.545752+0000 front 2024-05-11T13:59:52.546013+0000 (oldest deadline 2024-05-11T14:00:16.645942+0000) 2024-05-11T14:00:32.655 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5[44974]: 2024-05-11T14:00:32.531+0000 7fa47ca00700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.547669+0000 front 2024-05-11T14:00:02.547777+0000 (oldest deadline 2024-05-11T14:00:25.447799+0000) 2024-05-11T14:00:32.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:32.387+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:32.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:32.387+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:32.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:32.387+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:32.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:32 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:32.559+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:32.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:32 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:32.559+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:33.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:32.667+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:33.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:32 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:32.667+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:33.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:32.774+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:33.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:32.774+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:33.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:32.774+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:33.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:32.956+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:33.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:32.956+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:33.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:32 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:32.956+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:33.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:33 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:33.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:33.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:33 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:33.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:33.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:33 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:33.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:33.666 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:33 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:33.343+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:33.666 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:33 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:33.343+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:33.666 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:33 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:33.343+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:33.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:33 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:33.592+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:33.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:33 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:33.592+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:34.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:33 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:33.679+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:34.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:33 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:33.679+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:34.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:33.780+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:34.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:33.780+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:34.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:33.780+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:34.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:33.949+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:34.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:33.949+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:34.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:33 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:33.949+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:34.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:34 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:34.080+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:34.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:34 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:34.080+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:34.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:34 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:34.080+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:34.660 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:34 smithi165 podman[76076]: 2024-05-11 14:00:34.379564068 +0000 UTC m=+6.166021613 container died 9dc946963ef7e34b26bb843eeb1b7b3cf9744f596904045bbbfb92f0fe0f2952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-11T14:00:34.660 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:34 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:34.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:34.660 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:34 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:34.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:34.660 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:34 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:34.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:34.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:34 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:34.595+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:34.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:34 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:34.595+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:35.052 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:34 smithi165 podman[76076]: 2024-05-11 14:00:34.786862893 +0000 UTC m=+6.573320431 container cleanup 9dc946963ef7e34b26bb843eeb1b7b3cf9744f596904045bbbfb92f0fe0f2952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T14:00:35.052 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:34 smithi165 bash[76076]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5 2024-05-11T14:00:35.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:34 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:34.672+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:35.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:34 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:34.672+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:35.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:34 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:34.755+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:35.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:34 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:34.755+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:35.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:34 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:34.755+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:35.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:34.999+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:35.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:34.999+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:35.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:34.999+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:35.359 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:35 smithi165 podman[76108]: 2024-05-11 14:00:35.071478381 +0000 UTC m=+0.673464570 container remove 9dc946963ef7e34b26bb843eeb1b7b3cf9744f596904045bbbfb92f0fe0f2952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-05-11T14:00:35.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:35.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:35.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:35.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:35.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:35.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:35.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:35.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:35.611 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:35 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:35.371+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:35.611 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:35 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:35.371+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:35.611 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:35 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:35.372+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:35.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:35 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:35.624+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:35.887 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:35 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:35.624+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:35.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:35.584+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:35.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:35 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:35.584+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:36.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:35.769+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:36.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:35.769+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:36.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:35 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:35.769+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:36.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:36.011+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:36.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:36.011+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:36.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:36.011+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:36.197 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:35 smithi165 podman[76194]: 2024-05-11 14:00:35.80575579 +0000 UTC m=+0.036373322 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:00:36.198 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:35 smithi165 podman[76194]: 2024-05-11 14:00:35.931033697 +0000 UTC m=+0.161651267 container create 0b6c66601adfac30c3eabe7abeb1a8c3964a83cd07a3e8cc88d692c7048ceeb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507) 2024-05-11T14:00:36.198 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:36 smithi165 podman[76194]: 2024-05-11 14:00:36.210542654 +0000 UTC m=+0.441160234 container init 0b6c66601adfac30c3eabe7abeb1a8c3964a83cd07a3e8cc88d692c7048ceeb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-11T14:00:36.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:36 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:36.068+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:36.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:36 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:36.068+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:36.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:36 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:36.068+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:36.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:36 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:36.068+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:36.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:36 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:36.391+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:36.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:36 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:36.391+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:36.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:36 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:36.391+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:36.552 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:36 smithi165 podman[76194]: 2024-05-11 14:00:36.221847199 +0000 UTC m=+0.452464739 container start 0b6c66601adfac30c3eabe7abeb1a8c3964a83cd07a3e8cc88d692c7048ceeb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-11T14:00:36.553 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:36 smithi165 podman[76194]: 2024-05-11 14:00:36.334736234 +0000 UTC m=+0.565353778 container attach 0b6c66601adfac30c3eabe7abeb1a8c3964a83cd07a3e8cc88d692c7048ceeb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:00:36.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:36 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:36.600+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:36.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:36 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:36.600+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:37.049 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 11 14:00:36 smithi165 podman[76194]: 2024-05-11 14:00:36.688848673 +0000 UTC m=+0.919466249 container died 0b6c66601adfac30c3eabe7abeb1a8c3964a83cd07a3e8cc88d692c7048ceeb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-11T14:00:37.050 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:36 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:36.602+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:37.050 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:36 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:36.602+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:37.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:36.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:37.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:36.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:37.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:36.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:37.172 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:37.002+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:37.172 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:37.002+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:37.172 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:37.002+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:37.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:37 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:37.107+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:37.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:37 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:37.107+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:37.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:37 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:37.107+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:37.442 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:37 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:37.107+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:37.488 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.5.service' 2024-05-11T14:00:37.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:00:37.646 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-11T14:00:37.646 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-11T14:00:37.646 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.6 2024-05-11T14:00:37.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:37 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:37.594+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:37.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:37 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:37.594+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:37.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:37 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:37.428+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:37.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:37 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:37.428+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:37.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:37 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:37.428+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:37.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:37 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:37.561+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:37.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:37 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:37.561+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:38.149 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 systemd[1]: Stopping Ceph osd.6 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:00:38.149 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:37.981+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:38.149 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:37.981+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:38.149 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:37.981+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:38.149 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:37.981+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:12.403835+0000 front 2024-05-11T14:00:12.403780+0000 (oldest deadline 2024-05-11T14:00:37.604639+0000) 2024-05-11T14:00:38.149 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:37.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:38.149 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:37.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:38.150 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:37.787+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:38.150 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:37 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:37.787+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:38.421 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:38.254+0000 7fbc64200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:00:38.421 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:38.254+0000 7fbc64200700 -1 osd.6 133 *** Got signal Terminated *** 2024-05-11T14:00:38.421 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:38.254+0000 7fbc64200700 -1 osd.6 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:00:38.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:38 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:38.092+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:38.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:38 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:38.092+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:38.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:38 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:38.092+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:38.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:38 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:38.092+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:38.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:38 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:38.611+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:38.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:38 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:38.611+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:38.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:38 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:38.451+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:38.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:38 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:38.451+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:38.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:38 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:38.451+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:38.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:38 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:38.520+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:38.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:38 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:38.520+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:39.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:38.779+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:39.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:38.779+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:39.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:38.779+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:39.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:38.779+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:39.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:38.949+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:39.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:38.949+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:39.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:38.949+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:39.172 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:38 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:38.949+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:12.403835+0000 front 2024-05-11T14:00:12.403780+0000 (oldest deadline 2024-05-11T14:00:37.604639+0000) 2024-05-11T14:00:39.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:39 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:39.109+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:39.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:39 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:39.109+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:39.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:39 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:39.109+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:39.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:39 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:39.109+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:39.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:39 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:39.646+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:39.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:39 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:39.646+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:39.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:39 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:39.646+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:39.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:39 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:39.483+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:39.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:39 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:39.483+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:39.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:39 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:39.483+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:39.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:39 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:39.508+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:39.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:39 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:39.508+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:40.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:39.920+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:40.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:39.920+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:40.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:39.920+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:40.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:39.920+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:12.403835+0000 front 2024-05-11T14:00:12.403780+0000 (oldest deadline 2024-05-11T14:00:37.604639+0000) 2024-05-11T14:00:40.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:39.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:40.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:39.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:40.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:39.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:40.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:39 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:39.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:40.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:40 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:40.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:40.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:40 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:40.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:40.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:40 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:40.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:40.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:40 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:40.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:40.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:40 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:40.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:40.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:40 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:40.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:40.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:40 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:40.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:40.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:40 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:40.459+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:40.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:40 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:40.459+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:40.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:40 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:40.459+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:40.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:40 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:40.545+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:40.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:40 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:40.545+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:40.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:40 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:40.545+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:41.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:40.936+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:41.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:40.936+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:41.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:40.936+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:41.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:40.936+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:12.403835+0000 front 2024-05-11T14:00:12.403780+0000 (oldest deadline 2024-05-11T14:00:37.604639+0000) 2024-05-11T14:00:41.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:40.833+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:41.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:40.833+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:41.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:40.833+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:41.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:40.833+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:41.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:41 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:41.118+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:41.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:41 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:41.118+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:41.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:41 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:41.118+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:41.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:41 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:41.118+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:41.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:41 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:41.494+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:41.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:41 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:41.494+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:41.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:41 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:41.494+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:41.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:41 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:41.739+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:41.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:41 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:41.739+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:41.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:41 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:41.739+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:41.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:41 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:41.513+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:41.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:41 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:41.513+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:41.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:41 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:41.513+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:42.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:41.943+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:42.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:41.943+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:42.172 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:41.943+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:42.172 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:41.943+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:12.403835+0000 front 2024-05-11T14:00:12.403780+0000 (oldest deadline 2024-05-11T14:00:37.604639+0000) 2024-05-11T14:00:42.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:41.800+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:42.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:41.800+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:42.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:41.800+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:42.173 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:41 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:41.800+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:42.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:42.106+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:42.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:42.106+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:42.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:42.106+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:42.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:42.106+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:42.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:42.537+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:42.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:42.537+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:42.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:42.537+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:42.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:42.722+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:42.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:42.722+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:42.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:42 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:42.722+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:42.942 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:42.497+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:42.942 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:42.497+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:42.942 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:42 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:42.497+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:43.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:42.926+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:44.092779+0000 front 2024-05-11T13:59:44.093898+0000 (oldest deadline 2024-05-11T14:00:05.192857+0000) 2024-05-11T14:00:43.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:42.926+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:52.096016+0000 front 2024-05-11T13:59:52.096335+0000 (oldest deadline 2024-05-11T14:00:17.897443+0000) 2024-05-11T14:00:43.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:42.926+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.200012+0000 front 2024-05-11T14:00:00.199821+0000 (oldest deadline 2024-05-11T14:00:25.499746+0000) 2024-05-11T14:00:43.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6[55019]: 2024-05-11T14:00:42.926+0000 7fbc5fa00700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:12.403835+0000 front 2024-05-11T14:00:12.403780+0000 (oldest deadline 2024-05-11T14:00:37.604639+0000) 2024-05-11T14:00:43.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:42.775+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:43.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:42.775+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:43.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:42.775+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:43.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:42 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:42.775+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:43.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:43.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:43.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:43.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:43.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:43.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:43.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:43.154+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:43.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:43.698+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:43.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:43.698+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:43.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:43.698+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:43.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:43.535+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:43.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:43.535+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:43.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:43 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:43.535+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:43.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:43.505+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:43.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:43.505+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:43.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:43 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:43.505+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:44.146 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:43.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:44.147 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:43.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:44.147 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:43.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:44.147 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:43 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:43.794+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:44.421 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:44 smithi063 podman[116317]: 2024-05-11 14:00:44.159147207 +0000 UTC m=+6.055215522 container died 41e473a021a4397eedf199fbac097a394f6938616488581f8ba3be6aab20c22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-11T14:00:44.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:44.163+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:44.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:44.163+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:44.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:44.163+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:44.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:44.163+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:44.733 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:44 smithi063 podman[116317]: 2024-05-11 14:00:44.604393613 +0000 UTC m=+6.500461921 container cleanup 41e473a021a4397eedf199fbac097a394f6938616488581f8ba3be6aab20c22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD) 2024-05-11T14:00:44.733 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:44 smithi063 bash[116317]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6 2024-05-11T14:00:44.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:44.523+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:44.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:44.523+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:44.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:44 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:44.523+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:45.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:44 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:44.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:45.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:44 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:44.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:45.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:44 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:44.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:45.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:44 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:44.571+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:45.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:44 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:44.571+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:45.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:44 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:44.571+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:45.171 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:44 smithi063 podman[116335]: 2024-05-11 14:00:44.855850161 +0000 UTC m=+0.683466621 container remove 41e473a021a4397eedf199fbac097a394f6938616488581f8ba3be6aab20c22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:00:45.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:44 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:44.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:45.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:44 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:44.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:45.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:44 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:44.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:45.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:44 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:44.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:45.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:45.127+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:45.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:45.127+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:45.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:45.127+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:45.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:45.127+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:45.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:45.564+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:45.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:45.564+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:45.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:45.564+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:45.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:45.564+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:45.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:45.727+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:45.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:45.727+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:45.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:45 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:45.727+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:45.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:45.502+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:45.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:45.502+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:45.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:45.502+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:45.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:45 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:45.502+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:46.022 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:45.787+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:46.022 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:45.787+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:46.022 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:45.787+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:46.022 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:45 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:45.787+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:46.023 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:45 smithi063 podman[116424]: 2024-05-11 14:00:45.624277234 +0000 UTC m=+0.037647420 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:00:46.023 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:45 smithi063 podman[116424]: 2024-05-11 14:00:45.740286806 +0000 UTC m=+0.153657025 container create 9ccf2ba826b073636f6ba6a968b12a613364a59f25d14384d4e3b001aab4778f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-11T14:00:46.023 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:46 smithi063 podman[116424]: 2024-05-11 14:00:46.035476408 +0000 UTC m=+0.448846634 container init 9ccf2ba826b073636f6ba6a968b12a613364a59f25d14384d4e3b001aab4778f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:00:46.420 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:46 smithi063 podman[116424]: 2024-05-11 14:00:46.047503361 +0000 UTC m=+0.460873590 container start 9ccf2ba826b073636f6ba6a968b12a613364a59f25d14384d4e3b001aab4778f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-11T14:00:46.421 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:46 smithi063 podman[116424]: 2024-05-11 14:00:46.175919804 +0000 UTC m=+0.589289999 container attach 9ccf2ba826b073636f6ba6a968b12a613364a59f25d14384d4e3b001aab4778f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-11T14:00:46.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:46.152+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:46.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:46.152+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:46.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:46.152+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:46.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:46.152+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:46.760 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 11 14:00:46 smithi063 podman[116424]: 2024-05-11 14:00:46.511204421 +0000 UTC m=+0.924574604 container died 9ccf2ba826b073636f6ba6a968b12a613364a59f25d14384d4e3b001aab4778f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:00:46.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:46.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:46.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:46.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:46.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:46.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:46.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:46 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:46.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:47.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:46.696+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:47.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:46.696+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:47.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:46.696+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:47.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:46.612+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:47.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:46.612+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:47.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:46.612+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:47.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:46 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:46.612+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:47.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:46.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:47.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:46.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:47.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:46.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:47.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:46.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:47.332 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.6.service' 2024-05-11T14:00:47.430 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:47.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:47.535 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:47.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:47.535 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:47.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:47.535 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:47.121+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:47.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:00:47.537 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-11T14:00:47.538 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-11T14:00:47.538 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.7 2024-05-11T14:00:47.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:47.442+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:47.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:47.442+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:47.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:47.442+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:47.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:47 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:47.442+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:47.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:47 smithi071 systemd[1]: Stopping Ceph osd.7 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:00:48.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:47.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:48.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:47.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:48.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:47.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:48.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:47.584+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:48.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:47.584+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:48.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:47.584+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:48.053 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:47 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:47.584+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:48.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:47.785+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:48.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:47.785+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:48.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:47.785+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:48.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:47.785+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:48.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:47 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:47.785+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:48.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:48.077+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:48.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:48.077+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:48.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:48.077+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:48.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:48.077+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:48.441 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:48.084+0000 7f465c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:00:48.441 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:48.084+0000 7f465c000700 -1 osd.7 133 *** Got signal Terminated *** 2024-05-11T14:00:48.441 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:48.084+0000 7f465c000700 -1 osd.7 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:00:48.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:48.551+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:48.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:48.551+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:48.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:48.551+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:48.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:48.551+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:48.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:48.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:48.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:48.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:48.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:48.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:48.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:48 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:48.694+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:48.920 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:48.766+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:48.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:48.766+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:48.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:48.766+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:48.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:48.766+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:48.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:48 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:48.766+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:48.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:48.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:48.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:48.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:48.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:48.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:48.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:48 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:48.480+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:49.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:49.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:49.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:49.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:49.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:49.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:49.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:49.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:49.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:49.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:49.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:49.714+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:49.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:49.714+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:49.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:49.714+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:49.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:49.714+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:49.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:49.502+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:49.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:49.502+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:49.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:49.502+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:49.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:49 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:49.502+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:49.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:49.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:49.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:49.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:49.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:49.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:49.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:49 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:49.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:50.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:49.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:50.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:49.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:50.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:49.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:50.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:49.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:50.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:49 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:49.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:50.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:50.055+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:50.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:50.055+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:50.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:50.055+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:50.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:50.055+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:50.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:50.055+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:50.722 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:50.464+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:50.722 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:50.464+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:50.722 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:50.464+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:50.723 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:50.464+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:50.826 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:50.520+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:50.826 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:50.520+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:50.826 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:50.520+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:50.826 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:50 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:50.520+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:51.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:50.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:51.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:50.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:51.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:50.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:51.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:50 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:50.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:51.137 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:51.072+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:51.137 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:51.072+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:51.137 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:51.072+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:51.137 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:51.072+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:51.137 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:51.072+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:51.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:50.781+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:51.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:50.781+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:51.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:50.781+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:51.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:50.781+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:51.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:50 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:50.781+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:51.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:51.506+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:51.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:51.506+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:51.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:51.506+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:51.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:51.506+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:51.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:51.711+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:51.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:51.711+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:51.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:51.711+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:51.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:51 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:51.711+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:51.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:51.491+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:51.941 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:51.491+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:51.942 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:51.491+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:51.942 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:51 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:51.491+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:52.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:51 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:51.791+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:52.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:51 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:51.791+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:52.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:51 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:51.791+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:52.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:51 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:51.791+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:52.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:51 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:51.791+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:52.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:52.082+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:52.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:52.082+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:52.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:52.082+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:52.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:52.082+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:52.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:52.082+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:52.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:52.703+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:52.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:52.703+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:52.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:52.703+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:52.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:52.703+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:52.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:52.508+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:52.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:52.508+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:52.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:52.508+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:52.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:52 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:52.508+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:52.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:52.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.494958+0000 front 2024-05-11T13:59:54.495170+0000 (oldest deadline 2024-05-11T14:00:17.995154+0000) 2024-05-11T14:00:52.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:52.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.597698+0000 front 2024-05-11T14:00:02.598046+0000 (oldest deadline 2024-05-11T14:00:26.097500+0000) 2024-05-11T14:00:52.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:52.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.700192+0000 front 2024-05-11T14:00:13.699983+0000 (oldest deadline 2024-05-11T14:00:39.599934+0000) 2024-05-11T14:00:52.940 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:52 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7[51125]: 2024-05-11T14:00:52.484+0000 7f4657800700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:20.702217+0000 front 2024-05-11T14:00:20.702146+0000 (oldest deadline 2024-05-11T14:00:44.802010+0000) 2024-05-11T14:00:53.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:52.811+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:53.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:52.811+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:53.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:52.811+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:53.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:52.811+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:53.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:52.811+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:53.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:53 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:53.088+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:53.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:53 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:53.088+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:53.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:53 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:53.088+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:53.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:53 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:53.088+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:53.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:53 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:53.088+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:53.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:53.486+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:53.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:53.486+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:53.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:53.486+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:53.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:53.486+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:53.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:53.722+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:53.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:53.722+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:53.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:53.722+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:53.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:53 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:53.722+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:54.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:53 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:53.799+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:54.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:53 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:53.799+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:54.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:53 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:53.799+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:54.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:53 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:53.799+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:54.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:53 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:53.799+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:54.374 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:54 smithi071 podman[71148]: 2024-05-11 14:00:54.076704858 +0000 UTC m=+6.111424147 container died 9e0be783d57718b7b8d79ca88384514696f9a151fe682e6a6c5740943964eec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T14:00:54.374 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:54 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:54.046+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:54.374 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:54 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:54.046+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:54.375 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:54 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:54.046+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:54.375 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:54 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:54.046+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:54.375 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:54 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:54.046+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:54.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:54 smithi071 podman[71148]: 2024-05-11 14:00:54.519449888 +0000 UTC m=+6.554169207 container cleanup 9e0be783d57718b7b8d79ca88384514696f9a151fe682e6a6c5740943964eec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-11T14:00:54.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:54 smithi071 bash[71148]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7 2024-05-11T14:00:54.802 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:54.680+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:54.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:54.680+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:54.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:54.680+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:54.803 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:54.680+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:54.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:54.497+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:54.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:54.497+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:54.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:54.497+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:54.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:54 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:54.497+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:55.068 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:54 smithi071 podman[71178]: 2024-05-11 14:00:54.770868616 +0000 UTC m=+0.683949650 container remove 9e0be783d57718b7b8d79ca88384514696f9a151fe682e6a6c5740943964eec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-11T14:00:55.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:54.840+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:55.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:54.840+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:55.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:54.840+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:55.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:54.840+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:55.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:54.840+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:55.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:54 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:54.840+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:00:55.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:55 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:55.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:55.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:55 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:55.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:55.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:55 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:55.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:55.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:55 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:55.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:55.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:55 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:55.081+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:55.710 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:55.450+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:55.711 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:55.450+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:55.711 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:55.450+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:55.711 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:55.450+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:55.887 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:55 smithi071 podman[71267]: 2024-05-11 14:00:55.613298809 +0000 UTC m=+0.127393250 container create c4ac9081f76ba1f9c7f347145293eeffe2efb62acc6226a95a13ff2bfdf6f10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-11T14:00:55.887 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:55 smithi071 podman[71267]: 2024-05-11 14:00:55.519904142 +0000 UTC m=+0.033998546 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:00:55.887 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:55 smithi071 podman[71267]: 2024-05-11 14:00:55.900068251 +0000 UTC m=+0.414162735 container init c4ac9081f76ba1f9c7f347145293eeffe2efb62acc6226a95a13ff2bfdf6f10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-11T14:00:56.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:55.724+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:56.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:55.724+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:56.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:55.724+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:56.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:55 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:55.724+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:56.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:55.820+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:56.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:55.820+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:56.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:55.820+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:56.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:55.820+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:56.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:55.820+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:56.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:55 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:55.820+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:55 smithi071 podman[71267]: 2024-05-11 14:00:55.911151308 +0000 UTC m=+0.425245745 container start c4ac9081f76ba1f9c7f347145293eeffe2efb62acc6226a95a13ff2bfdf6f10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:56 smithi071 podman[71267]: 2024-05-11 14:00:56.007397883 +0000 UTC m=+0.521492331 container attach c4ac9081f76ba1f9c7f347145293eeffe2efb62acc6226a95a13ff2bfdf6f10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:56 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:56.074+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:56 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:56.074+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:56 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:56.074+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:56 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:56.074+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:56 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:56.074+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:56.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:56 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:56.074+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:00:56.690 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 11 14:00:56 smithi071 podman[71267]: 2024-05-11 14:00:56.369942576 +0000 UTC m=+0.884036980 container died c4ac9081f76ba1f9c7f347145293eeffe2efb62acc6226a95a13ff2bfdf6f10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-11T14:00:56.721 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:56.431+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:56.722 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:56.431+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:56.722 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:56.431+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:56.722 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:56.431+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:56.722 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:56.431+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:00:57.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:56.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:57.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:56.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:57.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:56.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:57.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:56.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:57.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:56 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:56.735+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:31.297562+0000 front 2024-05-11T14:00:31.297403+0000 (oldest deadline 2024-05-11T14:00:56.597461+0000) 2024-05-11T14:00:57.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:56 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:56.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:57.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:56 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:56.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:57.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:56 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:56.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:57.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:56 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:56.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:57.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:56 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:56.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:57.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:56 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:56.786+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:00:57.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:57 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:57.041+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:57.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:57 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:57.041+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:57.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:57 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:57.041+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:57.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:57 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:57.041+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:57.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:57 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:57.041+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:57.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:57 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:57.041+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:00:57.263 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.7.service' 2024-05-11T14:00:57.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:00:57.378 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-11T14:00:57.379 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-11T14:00:57.379 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.8 2024-05-11T14:00:57.705 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:57 smithi165 systemd[1]: Stopping Ceph osd.8 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:00:57.705 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:57.717+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:57.705 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:57.382+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:57.705 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:57.382+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:57.706 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:57.382+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:57.706 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:57.382+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:57.706 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:57.382+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:00:58.010 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:57.717+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:58.010 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:57.718+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:58.010 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:57.718+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:58.010 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:57 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:57.718+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:31.297562+0000 front 2024-05-11T14:00:31.297403+0000 (oldest deadline 2024-05-11T14:00:56.597461+0000) 2024-05-11T14:00:58.073 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:57.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:58.073 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:57.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:58.073 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:57.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:58.073 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:57.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:58.073 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:57.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:58.073 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:57 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:57.773+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:00:58.302 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.023+0000 7f1ee4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:00:58.302 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.023+0000 7f1ee4e00700 -1 osd.8 133 *** Got signal Terminated *** 2024-05-11T14:00:58.302 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.023+0000 7f1ee4e00700 -1 osd.8 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:00:58.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:58.070+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:58.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:58.070+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:58.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:58.070+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:58.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:58.070+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:58.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:58.070+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:58.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:58.070+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:00:58.657 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:58.339+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:58.657 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:58.339+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:58.657 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:58.339+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:58.657 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:58.339+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:58.657 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:58.339+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:00:59.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.670+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:00:59.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.670+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:00:59.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.670+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:00:59.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.670+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:00:59.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:58 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:58.670+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:31.297562+0000 front 2024-05-11T14:00:31.297403+0000 (oldest deadline 2024-05-11T14:00:56.597461+0000) 2024-05-11T14:00:59.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:58.801+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:59.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:58.801+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:59.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:58.801+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:59.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:58.801+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:59.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:58.801+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:59.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:58 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:58.801+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:00:59.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:59 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:59.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:00:59.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:59 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:59.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:00:59.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:59 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:59.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:00:59.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:59 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:59.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:00:59.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:59 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:59.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:00:59.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:00:59 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:00:59.067+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:00:59.647 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:59.371+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:00:59.648 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:59.371+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:00:59.648 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:59.371+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:00:59.648 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:59.371+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:00:59.648 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:00:59.371+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:00:59.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:59.812+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:00:59.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:59.812+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:00:59.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:59.812+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:00:59.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:59.812+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:00:59.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:59.812+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:00:59.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:00:59 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:00:59.812+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:00.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:59.660+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:01:00.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:59.660+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:01:00.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:59.660+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:01:00.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:59.660+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:01:00.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:00:59 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:00:59.660+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:31.297562+0000 front 2024-05-11T14:00:31.297403+0000 (oldest deadline 2024-05-11T14:00:56.597461+0000) 2024-05-11T14:01:00.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:00 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:00.032+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:00.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:00 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:00.032+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:00.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:00 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:00.032+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:00.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:00 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:00.032+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:00.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:00 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:00.032+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:00.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:00 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:00.032+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:00.676 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:00.392+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:00.676 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:00.392+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:00.676 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:00.392+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:00.677 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:00.392+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:00.677 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:00.392+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:01.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:00.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:01:01.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:00.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:01:01.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:00.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:01:01.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:00.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:01:01.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:00 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:00.689+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:31.297562+0000 front 2024-05-11T14:00:31.297403+0000 (oldest deadline 2024-05-11T14:00:56.597461+0000) 2024-05-11T14:01:01.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:00.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:01.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:00.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:01.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:00.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:01.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:00.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:01.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:00.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:01.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:00 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:00.790+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:01.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:01 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:01.030+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:01.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:01 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:01.030+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:01.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:01 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:01.030+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:01.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:01 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:01.030+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:01.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:01 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:01.030+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:01.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:01 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:01.030+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:01.712 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:01.415+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:01.713 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:01.415+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:01.713 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:01.415+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:01.713 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:01.415+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:01.713 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:01.415+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:01.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:01.749+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:01.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:01.749+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:01.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:01.749+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:01.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:01.749+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:01.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:01.749+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:01.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:01 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:01.749+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:02.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:01.725+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:01:02.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:01.725+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:01:02.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:01.725+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:01:02.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:01.725+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:01:02.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:01 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:01.725+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:31.297562+0000 front 2024-05-11T14:00:31.297403+0000 (oldest deadline 2024-05-11T14:00:56.597461+0000) 2024-05-11T14:01:02.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.004+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:02.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.004+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:02.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.004+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:02.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.004+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:02.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.004+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:02.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.004+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:02.679 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:02.396+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:02.680 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:02.396+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:02.680 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:02.396+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:02.680 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:02.396+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:02.680 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:02.396+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:03.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:02.692+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.678219+0000 front 2024-05-11T13:59:42.679219+0000 (oldest deadline 2024-05-11T14:00:05.578331+0000) 2024-05-11T14:01:03.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:02.692+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:00.085625+0000 front 2024-05-11T14:00:00.085933+0000 (oldest deadline 2024-05-11T14:00:25.985588+0000) 2024-05-11T14:01:03.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:02.692+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:08.187997+0000 front 2024-05-11T14:00:08.187794+0000 (oldest deadline 2024-05-11T14:00:39.388860+0000) 2024-05-11T14:01:03.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:02.692+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:22.594838+0000 front 2024-05-11T14:00:22.594568+0000 (oldest deadline 2024-05-11T14:00:48.494666+0000) 2024-05-11T14:01:03.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:02 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8[51484]: 2024-05-11T14:01:02.692+0000 7f1edb600700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:31.297562+0000 front 2024-05-11T14:00:31.297403+0000 (oldest deadline 2024-05-11T14:00:56.597461+0000) 2024-05-11T14:01:03.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:02.723+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:03.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:02.723+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:03.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:02.723+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:03.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:02.723+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:03.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:02.723+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:03.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:02 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:02.723+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:03.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.983+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:03.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.983+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:03.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.983+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:03.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.983+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:03.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.983+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:03.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:02 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:02.983+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:03.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:03 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:03.394+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:03.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:03 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:03.394+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:03.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:03 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:03.394+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:03.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:03 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:03.394+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:03.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:03 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:03.394+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:04.165 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:03 smithi165 podman[76299]: 2024-05-11 14:01:03.871814883 +0000 UTC m=+6.057068105 container died 1800be810d981c4810297ffdc81096d28f6fec66d7aeae8f2aa14929db058bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-11T14:01:04.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:03.720+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:04.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:03.720+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:04.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:03.720+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:04.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:03.720+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:04.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:03.720+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:04.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:03 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:03.720+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:04.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:03 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:03.973+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:04.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:03 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:03.973+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:04.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:03 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:03.973+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:04.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:03 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:03.973+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:04.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:03 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:03.973+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:04.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:03 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:03.973+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:04.552 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:04 smithi165 podman[76299]: 2024-05-11 14:01:04.307038338 +0000 UTC m=+6.492291557 container cleanup 1800be810d981c4810297ffdc81096d28f6fec66d7aeae8f2aa14929db058bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:01:04.552 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:04 smithi165 bash[76299]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8 2024-05-11T14:01:04.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:04 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:04.375+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:04.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:04 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:04.375+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:04.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:04 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:04.375+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:04.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:04 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:04.375+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:04.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:04 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:04.375+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:05.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:04 smithi165 podman[76331]: 2024-05-11 14:01:04.566744927 +0000 UTC m=+0.682128563 container remove 1800be810d981c4810297ffdc81096d28f6fec66d7aeae8f2aa14929db058bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:01:05.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:04 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:04.741+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:05.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:04 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:04.741+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:05.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:04 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:04.741+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:05.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:04 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:04.741+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:05.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:04 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:04.741+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:05.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:04 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:04.741+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:05.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:04.962+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:05.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:04.962+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:05.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:04.962+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:05.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:04.962+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:05.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:04.962+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:05.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:04.962+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:05 smithi165 podman[76419]: 2024-05-11 14:01:05.262175741 +0000 UTC m=+0.037669045 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:05 smithi165 podman[76419]: 2024-05-11 14:01:05.375777627 +0000 UTC m=+0.151270968 container create dc918160d451a2005e4c9dc05cf6b38618eff2adb10dab6bd29014dc2b8edf64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:05 smithi165 podman[76419]: 2024-05-11 14:01:05.6548413 +0000 UTC m=+0.430334648 container init dc918160d451a2005e4c9dc05cf6b38618eff2adb10dab6bd29014dc2b8edf64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=) 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:05 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:05.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:05 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:05.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:05 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:05.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:05 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:05.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:05.642 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:05 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:05.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:06.052 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:05 smithi165 podman[76419]: 2024-05-11 14:01:05.667677559 +0000 UTC m=+0.443170898 container start dc918160d451a2005e4c9dc05cf6b38618eff2adb10dab6bd29014dc2b8edf64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-11T14:01:06.053 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:05 smithi165 podman[76419]: 2024-05-11 14:01:05.78719367 +0000 UTC m=+0.562687007 container attach dc918160d451a2005e4c9dc05cf6b38618eff2adb10dab6bd29014dc2b8edf64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:01:06.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:05.709+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:06.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:05.709+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:06.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:05.709+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:06.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:05.709+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:06.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:05.709+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:06.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:05 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:05.709+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:06.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.006+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:06.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.006+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:06.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.006+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:06.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.006+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:06.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.006+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:06.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.006+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:06.474 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 11 14:01:06 smithi165 podman[76419]: 2024-05-11 14:01:06.122953397 +0000 UTC m=+0.898446758 container died dc918160d451a2005e4c9dc05cf6b38618eff2adb10dab6bd29014dc2b8edf64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:01:06.474 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:06.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:06.474 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:06.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:06.475 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:06.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:06.475 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:06.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:06.475 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:06 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:06.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:06.914 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.8.service' 2024-05-11T14:01:07.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:07.025 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-11T14:01:07.025 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-11T14:01:07.025 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.9 2024-05-11T14:01:07.110 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:06.756+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:07.111 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:06.756+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:07.111 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:06.756+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:07.111 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:06.756+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:07.111 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:06.756+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:07.111 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:06 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:06.756+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:07.421 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 systemd[1]: Stopping Ceph osd.9 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:07.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.967+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:07.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.967+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:07.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.967+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:07.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.967+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:07.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.967+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:07.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:06 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:06.967+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:07.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:07 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:07.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:07.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:07 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:07.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:07.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:07 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:07.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:07.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:07 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:07.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:07.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:07 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:07.361+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:08.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.729+0000 7fbe78000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:08.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.729+0000 7fbe78000700 -1 osd.9 133 *** Got signal Terminated *** 2024-05-11T14:01:08.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.729+0000 7fbe78000700 -1 osd.9 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:01:08.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:08.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:08.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:08.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:08.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:08.172 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:07 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:07.746+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:08.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:07.970+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:08.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:07.970+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:08.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:07.970+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:08.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:07.970+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:08.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:07.970+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:08.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:07.970+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:08.441 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:07 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:07.970+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:08.718 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:08.355+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:08.719 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:08.355+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:08.719 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:08.355+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:08.719 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:08.355+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:08.719 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:08.355+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:08.719 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:08 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:08.355+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:09.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:08.698+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:09.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:08.698+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:09.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:08.698+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:09.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:08.698+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:09.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:08.698+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:09.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:08 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:08.698+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:09.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:08.949+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:09.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:08.949+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:09.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:08.949+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:09.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:08.949+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:09.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:08.949+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:09.192 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:08.949+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:09.192 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:08.949+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:09.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:09.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:09.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:09.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:09.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:09.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:09.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:09.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:09.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:09.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:09.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:09 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:09.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:10.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:09.691+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:10.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:09.691+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:10.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:09.691+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:10.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:09.691+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:10.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:09.691+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:10.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:09 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:09.691+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:10.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:09.931+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:10.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:09.931+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:10.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:09.931+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:10.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:09.931+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:10.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:09.931+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:10.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:09.931+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:10.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:09 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:09.931+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:10.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:10.369+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:10.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:10.369+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:10.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:10.369+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:10.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:10.369+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:10.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:10.369+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:10.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:10 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:10.369+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:11.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:10.735+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:11.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:10.735+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:11.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:10.735+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:11.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:10.735+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:11.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:10.735+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:11.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:10 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:10.735+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:11.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:10.897+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:11.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:10.897+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:11.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:10.897+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:11.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:10.897+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:11.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:10.897+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:11.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:10.897+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:11.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:10 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:10.897+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:11.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:11.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:11.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:11.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:11.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:11.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:11.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:11.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:11.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:11.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:11.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:11 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:11.383+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:12.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:11.724+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:12.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:11.724+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:12.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:11.724+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:12.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:11.724+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:12.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:11.724+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:12.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:11 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:11.724+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:12.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:11.871+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:12.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:11.871+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:12.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:11.871+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:12.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:11.871+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:12.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:11.871+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:12.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:11.871+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:12.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:11 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:11.871+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:12.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:12.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:12.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:12.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:12.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:12.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:12.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:12.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:12.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:12.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:12.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:12 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:12.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:13.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:12.689+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:38.555959+0000 front 2024-05-11T13:59:38.555880+0000 (oldest deadline 2024-05-11T14:00:05.557642+0000) 2024-05-11T14:01:13.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:12.689+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.361048+0000 front 2024-05-11T13:59:54.361837+0000 (oldest deadline 2024-05-11T14:00:17.261108+0000) 2024-05-11T14:01:13.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:12.689+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:01.963179+0000 front 2024-05-11T14:00:01.963235+0000 (oldest deadline 2024-05-11T14:00:28.364722+0000) 2024-05-11T14:01:13.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:12.689+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.567597+0000 front 2024-05-11T14:00:13.567054+0000 (oldest deadline 2024-05-11T14:00:37.667047+0000) 2024-05-11T14:01:13.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:12.689+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:21.769335+0000 front 2024-05-11T14:00:21.768807+0000 (oldest deadline 2024-05-11T14:00:47.068905+0000) 2024-05-11T14:01:13.171 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:12 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9[61908]: 2024-05-11T14:01:12.689+0000 7fbe6e800700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.271874+0000 front 2024-05-11T14:00:32.272051+0000 (oldest deadline 2024-05-11T14:00:54.471892+0000) 2024-05-11T14:01:13.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:12.860+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:13.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:12.860+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:13.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:12.860+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:13.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:12.860+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:13.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:12.860+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:13.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:12.860+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:13.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:12.860+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:13.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:13.344+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:13.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:13.344+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:13.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:13.344+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:13.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:13.344+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:13.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:13.344+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:13.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:13 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:13.344+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:13.988 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:13 smithi063 podman[116557]: 2024-05-11 14:01:13.684987191 +0000 UTC m=+6.156974124 container died 3402f78966ac1c892271bfc0e39a5752973187a820267fd1b66778412cbffc27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:01:14.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:13.868+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:14.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:13.868+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:14.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:13.868+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:14.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:13.868+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:14.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:13.868+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:14.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:13.868+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:14.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:13 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:13.868+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:14.422 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:14 smithi063 podman[116557]: 2024-05-11 14:01:14.198508363 +0000 UTC m=+6.670495289 container cleanup 3402f78966ac1c892271bfc0e39a5752973187a820267fd1b66778412cbffc27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-11T14:01:14.422 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:14 smithi063 bash[116557]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9 2024-05-11T14:01:14.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:14.380+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:14.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:14.380+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:14.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:14.380+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:14.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:14.380+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:14.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:14.380+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:14.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:14 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:14.380+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:14.920 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:14 smithi063 podman[116577]: 2024-05-11 14:01:14.483195297 +0000 UTC m=+0.772678277 container remove 3402f78966ac1c892271bfc0e39a5752973187a820267fd1b66778412cbffc27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-11T14:01:15.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:14.903+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:15.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:14.903+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:15.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:14.903+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:15.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:14.903+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:15.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:14.903+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:15.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:14.903+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:15.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:14 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:14.903+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:15.526 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:15 smithi063 podman[116666]: 2024-05-11 14:01:15.173649873 +0000 UTC m=+0.032599905 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:01:15.527 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:15 smithi063 podman[116666]: 2024-05-11 14:01:15.275499161 +0000 UTC m=+0.134449234 container create 909cac741cb1ee377cb3a48a034695cbd532506aebfae50129e365d9b169c792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-11T14:01:15.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:15.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:15.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:15.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:15.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:15.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:15.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:15.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:15.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:15.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:15.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:15 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:15.363+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:15.920 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:15 smithi063 podman[116666]: 2024-05-11 14:01:15.539495527 +0000 UTC m=+0.398445649 container init 909cac741cb1ee377cb3a48a034695cbd532506aebfae50129e365d9b169c792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, ceph=True) 2024-05-11T14:01:15.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:15 smithi063 podman[116666]: 2024-05-11 14:01:15.552301315 +0000 UTC m=+0.411251380 container start 909cac741cb1ee377cb3a48a034695cbd532506aebfae50129e365d9b169c792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-05-11T14:01:15.921 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:15 smithi063 podman[116666]: 2024-05-11 14:01:15.677812473 +0000 UTC m=+0.536762569 container attach 909cac741cb1ee377cb3a48a034695cbd532506aebfae50129e365d9b169c792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T14:01:16.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:15.934+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:16.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:15.934+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:16.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:15.934+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:16.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:15.934+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:16.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:15.934+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:16.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:15.934+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:16.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:15 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:15.934+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:16.373 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 11 14:01:16 smithi063 podman[116666]: 2024-05-11 14:01:16.032083926 +0000 UTC m=+0.891034045 container died 909cac741cb1ee377cb3a48a034695cbd532506aebfae50129e365d9b169c792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507) 2024-05-11T14:01:16.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:16.364+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:16.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:16.364+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:16.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:16.364+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:16.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:16.364+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:16.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:16.364+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:16.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:16.364+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:16.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:16.364+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:16.880 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.9.service' 2024-05-11T14:01:17.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:17.001 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-11T14:01:17.001 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-11T14:01:17.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.10 2024-05-11T14:01:17.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:16.923+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:17.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:16.923+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:17.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:16.923+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:17.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:16.923+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:17.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:16.923+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:17.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:16.923+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:17.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:16 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:16.923+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:17.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 systemd[1]: Stopping Ceph osd.10 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:17.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:17.356+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:17.356+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:17.356+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:17.356+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:17.356+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:17.356+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:17.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:17 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:17.356+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:17.890 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.592+0000 7f76e6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:17.890 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.592+0000 7f76e6a00700 -1 osd.10 133 *** Got signal Terminated *** 2024-05-11T14:01:17.890 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.592+0000 7f76e6a00700 -1 osd.10 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:01:17.891 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.902+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:17.891 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.902+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:17.891 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.902+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:17.891 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.902+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:17.891 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.902+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:17.891 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.902+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:17.891 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:17 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:17.902+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:18.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:18.313+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:18.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:18.313+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:18.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:18.313+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:18.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:18.313+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:18.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:18.313+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:18.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:18.313+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:18.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:18 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:18.313+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:19.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:18.905+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:19.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:18.905+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:19.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:18.905+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:19.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:18.905+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:19.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:18.905+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:19.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:18.905+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:19.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:18 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:18.905+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:19.324 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:19.335+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:19.324 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:19.335+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:19.324 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:19.335+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:19.324 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:19.335+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:19.324 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:19.335+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:19.325 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:19.335+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:19.325 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:19 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:19.335+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:20.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:19.882+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:20.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:19.882+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:20.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:19.882+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:20.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:19.882+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:20.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:19.882+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:20.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:19.882+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:20.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:19 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:19.882+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:20.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:20.370+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:20.370+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:20.370+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:20.370+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:20.370+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:20.370+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:20.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:20.370+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:21.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:20.859+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:21.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:20.859+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:21.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:20.859+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:21.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:20.859+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:21.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:20.859+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:21.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:20.859+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:21.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:20 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:20.859+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:21.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:21.379+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:21.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:21.379+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:21.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:21.379+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:21.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:21.379+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:21.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:21.379+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:21.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:21.379+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:21.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:21 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:21.379+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:22.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:21.820+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:42.306286+0000 front 2024-05-11T13:59:42.306341+0000 (oldest deadline 2024-05-11T14:00:05.206424+0000) 2024-05-11T14:01:22.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:21.820+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:54.412660+0000 front 2024-05-11T13:59:54.412929+0000 (oldest deadline 2024-05-11T14:00:19.712654+0000) 2024-05-11T14:01:22.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:21.820+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:02.014938+0000 front 2024-05-11T14:00:02.015010+0000 (oldest deadline 2024-05-11T14:00:27.914782+0000) 2024-05-11T14:01:22.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:21.820+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-11T14:00:13.718123+0000 front 2024-05-11T14:00:13.718205+0000 (oldest deadline 2024-05-11T14:00:34.818017+0000) 2024-05-11T14:01:22.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:21.820+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:23.621109+0000 front 2024-05-11T14:00:23.621178+0000 (oldest deadline 2024-05-11T14:00:48.921287+0000) 2024-05-11T14:01:22.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:21.820+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:32.324104+0000 front 2024-05-11T14:00:32.323808+0000 (oldest deadline 2024-05-11T14:00:55.823424+0000) 2024-05-11T14:01:22.191 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:21 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10[57972]: 2024-05-11T14:01:21.820+0000 7f76dd200700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.125280+0000 front 2024-05-11T14:00:41.125395+0000 (oldest deadline 2024-05-11T14:01:07.025289+0000) 2024-05-11T14:01:22.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:22.384+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:22.384+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:22.384+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:22.384+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:22.384+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:22.384+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:22.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:22 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:22.384+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:23.440 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:23 smithi071 podman[71386]: 2024-05-11 14:01:23.156670427 +0000 UTC m=+5.710436015 container died 230106416b193e495e0907652209a0da03982ee91b20af14eb19d33aeb5c0ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-11T14:01:23.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:23.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:23.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:23.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:23.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:23.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:23.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:23.803 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:23 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:23.354+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:23.830 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:23 smithi071 podman[71386]: 2024-05-11 14:01:23.601606884 +0000 UTC m=+6.155372466 container cleanup 230106416b193e495e0907652209a0da03982ee91b20af14eb19d33aeb5c0ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-11T14:01:23.831 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:23 smithi071 bash[71386]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10 2024-05-11T14:01:24.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:23 smithi071 podman[71416]: 2024-05-11 14:01:23.844099682 +0000 UTC m=+0.658223841 container remove 230106416b193e495e0907652209a0da03982ee91b20af14eb19d33aeb5c0ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS) 2024-05-11T14:01:24.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:24.305+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:24.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:24.305+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:24.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:24.305+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:24.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:24.305+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:24.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:24.305+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:24.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:24.305+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:24.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:24.305+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:24.928 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:24 smithi071 podman[71504]: 2024-05-11 14:01:24.559664399 +0000 UTC m=+0.033229162 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:01:24.928 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:24 smithi071 podman[71504]: 2024-05-11 14:01:24.66990782 +0000 UTC m=+0.143472619 container create ef58f6d4973d2e06d977e7e3db8b5bed1ca19dfa48a8445e3e1397b09f236023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-11T14:01:25.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:24 smithi071 podman[71504]: 2024-05-11 14:01:24.940347012 +0000 UTC m=+0.413911821 container init ef58f6d4973d2e06d977e7e3db8b5bed1ca19dfa48a8445e3e1397b09f236023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:01:25.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:24 smithi071 podman[71504]: 2024-05-11 14:01:24.953182397 +0000 UTC m=+0.426747239 container start ef58f6d4973d2e06d977e7e3db8b5bed1ca19dfa48a8445e3e1397b09f236023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-11T14:01:25.190 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:25 smithi071 podman[71504]: 2024-05-11 14:01:25.047059828 +0000 UTC m=+0.520624604 container attach ef58f6d4973d2e06d977e7e3db8b5bed1ca19dfa48a8445e3e1397b09f236023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-11T14:01:25.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:25.272+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:25.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:25.272+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:25.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:25.272+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:25.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:25.272+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:25.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:25.272+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:25.554 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:25.272+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:25.554 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:25 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:25.272+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:25.677 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 11 14:01:25 smithi071 podman[71504]: 2024-05-11 14:01:25.393990373 +0000 UTC m=+0.867555162 container died ef58f6d4973d2e06d977e7e3db8b5bed1ca19dfa48a8445e3e1397b09f236023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:01:26.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.10.service' 2024-05-11T14:01:26.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:26.152 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-11T14:01:26.153 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-11T14:01:26.153 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.11 2024-05-11T14:01:26.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 systemd[1]: Stopping Ceph osd.11 for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:26.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.284+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:26.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.284+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:26.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.284+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:26.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.284+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:26.554 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.284+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:26.554 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.284+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:26.554 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.284+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:27.052 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.745+0000 7fafcda00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:27.052 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.745+0000 7fafcda00700 -1 osd.11 133 *** Got signal Terminated *** 2024-05-11T14:01:27.052 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:26 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:26.745+0000 7fafcda00700 -1 osd.11 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T14:01:27.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:27.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:27.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:27.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:27.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:27.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:27.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:27.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:27 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:27.238+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-05-11T14:01:02.894457+0000 front 2024-05-11T14:01:02.894363+0000 (oldest deadline 2024-05-11T14:01:26.994256+0000) 2024-05-11T14:01:28.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:28.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:28.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:28.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:28.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:28.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:28.554 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:28.554 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:28.254+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-05-11T14:01:02.894457+0000 front 2024-05-11T14:01:02.894363+0000 (oldest deadline 2024-05-11T14:01:26.994256+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:29.455 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:29 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:29.207+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-05-11T14:01:02.894457+0000 front 2024-05-11T14:01:02.894363+0000 (oldest deadline 2024-05-11T14:01:26.994256+0000) 2024-05-11T14:01:30.552 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.244+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:30.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.245+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:30.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.245+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:30.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.245+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:30.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.245+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:30.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.245+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:30.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.245+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:30.553 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:30 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:30.245+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-05-11T14:01:02.894457+0000 front 2024-05-11T14:01:02.894363+0000 (oldest deadline 2024-05-11T14:01:26.994256+0000) 2024-05-11T14:01:31.302 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-11T13:59:43.070998+0000 front 2024-05-11T13:59:43.071579+0000 (oldest deadline 2024-05-11T14:00:05.971257+0000) 2024-05-11T14:01:31.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-11T13:59:51.873329+0000 front 2024-05-11T13:59:51.873416+0000 (oldest deadline 2024-05-11T14:00:17.172838+0000) 2024-05-11T14:01:31.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-11T14:00:04.176266+0000 front 2024-05-11T14:00:04.176475+0000 (oldest deadline 2024-05-11T14:00:26.476494+0000) 2024-05-11T14:01:31.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-11T14:00:19.882496+0000 front 2024-05-11T14:00:19.882110+0000 (oldest deadline 2024-05-11T14:00:44.582354+0000) 2024-05-11T14:01:31.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-11T14:00:33.187018+0000 front 2024-05-11T14:00:33.187153+0000 (oldest deadline 2024-05-11T14:00:56.087219+0000) 2024-05-11T14:01:31.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.63:6822 osd.6 since back 2024-05-11T14:00:41.890219+0000 front 2024-05-11T14:00:41.890086+0000 (oldest deadline 2024-05-11T14:01:07.789805+0000) 2024-05-11T14:01:31.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-11T14:00:50.691496+0000 front 2024-05-11T14:00:50.691549+0000 (oldest deadline 2024-05-11T14:01:15.991387+0000) 2024-05-11T14:01:31.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:31 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11[58149]: 2024-05-11T14:01:31.295+0000 7fafc4200700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-05-11T14:01:02.894457+0000 front 2024-05-11T14:01:02.894363+0000 (oldest deadline 2024-05-11T14:01:26.994256+0000) 2024-05-11T14:01:32.904 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:32 smithi165 podman[76535]: 2024-05-11 14:01:32.636137997 +0000 UTC m=+6.036070801 container died d35dc1d0e064ee14179c67f65797a956a6567192d0d8f5fb13a7e214ee07c6f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-11T14:01:33.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:33 smithi165 podman[76535]: 2024-05-11 14:01:33.0297413 +0000 UTC m=+6.429674135 container cleanup d35dc1d0e064ee14179c67f65797a956a6567192d0d8f5fb13a7e214ee07c6f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-11T14:01:33.303 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:33 smithi165 bash[76535]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11 2024-05-11T14:01:33.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:33 smithi165 podman[76568]: 2024-05-11 14:01:33.322530333 +0000 UTC m=+0.674833706 container remove d35dc1d0e064ee14179c67f65797a956a6567192d0d8f5fb13a7e214ee07c6f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True) 2024-05-11T14:01:34.383 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:34 smithi165 podman[76657]: 2024-05-11 14:01:34.037847518 +0000 UTC m=+0.033620630 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T14:01:34.384 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:34 smithi165 podman[76657]: 2024-05-11 14:01:34.148732936 +0000 UTC m=+0.144506079 container create abbfcb65faf264ce42d5d8cd822ec45ea550107ede6bf83446aae7e3b0026939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-11T14:01:34.384 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:34 smithi165 podman[76657]: 2024-05-11 14:01:34.395797014 +0000 UTC m=+0.391570200 container init abbfcb65faf264ce42d5d8cd822ec45ea550107ede6bf83446aae7e3b0026939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-11T14:01:34.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:34 smithi165 podman[76657]: 2024-05-11 14:01:34.407394885 +0000 UTC m=+0.403168031 container start abbfcb65faf264ce42d5d8cd822ec45ea550107ede6bf83446aae7e3b0026939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-11T14:01:34.802 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:34 smithi165 podman[76657]: 2024-05-11 14:01:34.501107153 +0000 UTC m=+0.496880305 container attach abbfcb65faf264ce42d5d8cd822ec45ea550107ede6bf83446aae7e3b0026939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-11T14:01:35.186 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 11 14:01:34 smithi165 podman[76657]: 2024-05-11 14:01:34.849523242 +0000 UTC m=+0.845296349 container died abbfcb65faf264ce42d5d8cd822ec45ea550107ede6bf83446aae7e3b0026939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T14:01:35.532 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@osd.11.service' 2024-05-11T14:01:35.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:35.646 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-11T14:01:35.646 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-11T14:01:35.646 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.a 2024-05-11T14:01:36.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 14:01:35 smithi063 systemd[1]: Stopping Ceph mds.a for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:36.420 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 14:01:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a[69379]: 2024-05-11T14:01:36.255+0000 7f4c76a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:36.421 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 14:01:36 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a[69379]: 2024-05-11T14:01:36.255+0000 7f4c76a00700 -1 mds.a *** got signal Terminated *** 2024-05-11T14:01:38.725 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 14:01:38 smithi063 podman[116769]: 2024-05-11 14:01:38.482867985 +0000 UTC m=+2.361366107 container died 8e87e1d62b62dff0374d1b53d5baf74bcb60ff09f3f46c6dc05df1f71c45e941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-11T14:01:39.112 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 14:01:38 smithi063 podman[116769]: 2024-05-11 14:01:38.915435981 +0000 UTC m=+2.793934096 container cleanup 8e87e1d62b62dff0374d1b53d5baf74bcb60ff09f3f46c6dc05df1f71c45e941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:01:39.112 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 14:01:38 smithi063 bash[116769]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a 2024-05-11T14:01:39.420 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 11 14:01:39 smithi063 podman[116795]: 2024-05-11 14:01:39.124935249 +0000 UTC m=+0.622484893 container remove 8e87e1d62b62dff0374d1b53d5baf74bcb60ff09f3f46c6dc05df1f71c45e941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-11T14:01:39.654 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.a.service' 2024-05-11T14:01:39.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:39.773 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-11T14:01:39.774 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-11T14:01:39.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.d 2024-05-11T14:01:40.171 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 14:01:39 smithi063 systemd[1]: Stopping Ceph mds.d for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:40.670 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 14:01:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d[68025]: 2024-05-11T14:01:40.417+0000 7f8639a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:40.671 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 14:01:40 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d[68025]: 2024-05-11T14:01:40.417+0000 7f8639a00700 -1 mds.d *** got signal Terminated *** 2024-05-11T14:01:44.727 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 14:01:44 smithi063 podman[116900]: 2024-05-11 14:01:44.444935834 +0000 UTC m=+4.152277038 container died 4d1e9d1e3aab7a5cda8671a02e4165c096b70cc7fdb6ae243ea20d2100225bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True) 2024-05-11T14:01:45.107 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 14:01:44 smithi063 podman[116900]: 2024-05-11 14:01:44.86105529 +0000 UTC m=+4.568396508 container cleanup 4d1e9d1e3aab7a5cda8671a02e4165c096b70cc7fdb6ae243ea20d2100225bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-11T14:01:45.107 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 14:01:44 smithi063 bash[116900]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d 2024-05-11T14:01:45.420 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 11 14:01:45 smithi063 podman[116914]: 2024-05-11 14:01:45.1201409 +0000 UTC m=+0.663920999 container remove 4d1e9d1e3aab7a5cda8671a02e4165c096b70cc7fdb6ae243ea20d2100225bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T14:01:45.641 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.d.service' 2024-05-11T14:01:45.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:45.753 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-11T14:01:45.753 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-11T14:01:45.753 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.g 2024-05-11T14:01:46.170 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 14:01:45 smithi063 systemd[1]: Stopping Ceph mds.g for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:46.670 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 14:01:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g[68957]: 2024-05-11T14:01:46.395+0000 7f51a8600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:46.670 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 14:01:46 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g[68957]: 2024-05-11T14:01:46.395+0000 7f51a8600700 -1 mds.g *** got signal Terminated *** 2024-05-11T14:01:50.670 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 14:01:50 smithi063 podman[117022]: 2024-05-11 14:01:50.416513659 +0000 UTC m=+4.171913132 container died dda168a2ca7950a53174d4db6f5fa87a7aa1b57c5fb4d613551ae71e1693fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T14:01:50.970 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 14:01:50 smithi063 podman[117022]: 2024-05-11 14:01:50.862223901 +0000 UTC m=+4.617623334 container cleanup dda168a2ca7950a53174d4db6f5fa87a7aa1b57c5fb4d613551ae71e1693fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-11T14:01:50.970 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 14:01:50 smithi063 bash[117022]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g 2024-05-11T14:01:51.420 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 11 14:01:51 smithi063 podman[117036]: 2024-05-11 14:01:51.038480422 +0000 UTC m=+0.601140917 container remove dda168a2ca7950a53174d4db6f5fa87a7aa1b57c5fb4d613551ae71e1693fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-11T14:01:51.595 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.g.service' 2024-05-11T14:01:51.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:51.708 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-11T14:01:51.708 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-11T14:01:51.708 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.j 2024-05-11T14:01:51.920 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 14:01:51 smithi063 systemd[1]: Stopping Ceph mds.j for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:52.670 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 14:01:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j[67559]: 2024-05-11T14:01:52.310+0000 7f0ebf800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:52.670 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 14:01:52 smithi063 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j[67559]: 2024-05-11T14:01:52.310+0000 7f0ebf800700 -1 mds.j *** got signal Terminated *** 2024-05-11T14:01:56.622 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 14:01:56 smithi063 podman[117140]: 2024-05-11 14:01:56.198374898 +0000 UTC m=+4.011934756 container died bba3825e142c3e46d11a4ce8fef13ce0d41b97fc1e4aab7c9576a812d5c30fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-11T14:01:56.920 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 14:01:56 smithi063 podman[117140]: 2024-05-11 14:01:56.828859868 +0000 UTC m=+4.642419811 container cleanup bba3825e142c3e46d11a4ce8fef13ce0d41b97fc1e4aab7c9576a812d5c30fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-11T14:01:56.921 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 14:01:56 smithi063 bash[117140]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j 2024-05-11T14:01:57.420 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 11 14:01:57 smithi063 podman[117154]: 2024-05-11 14:01:57.046462314 +0000 UTC m=+0.827904197 container remove bba3825e142c3e46d11a4ce8fef13ce0d41b97fc1e4aab7c9576a812d5c30fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-11T14:01:57.562 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.j.service' 2024-05-11T14:01:57.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:01:57.665 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-11T14:01:57.665 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-11T14:01:57.665 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.b 2024-05-11T14:01:58.190 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 14:01:57 smithi071 systemd[1]: Stopping Ceph mds.b for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:01:58.574 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 14:01:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b[63166]: 2024-05-11T14:01:58.363+0000 7fed2f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:01:58.574 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 14:01:58 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b[63166]: 2024-05-11T14:01:58.363+0000 7fed2f800700 -1 mds.b *** got signal Terminated *** 2024-05-11T14:02:02.690 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 14:02:02 smithi071 podman[71610]: 2024-05-11 14:02:02.384528064 +0000 UTC m=+4.258890378 container died 29a345b07e8dcea555e59b26d4efc3d9d8875da459df6600ba4b9890424e0df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-11T14:02:03.259 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 14:02:03 smithi071 podman[71610]: 2024-05-11 14:02:03.080010261 +0000 UTC m=+4.954372580 container cleanup 29a345b07e8dcea555e59b26d4efc3d9d8875da459df6600ba4b9890424e0df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T14:02:03.259 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 14:02:03 smithi071 bash[71610]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b 2024-05-11T14:02:03.690 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 11 14:02:03 smithi071 podman[71636]: 2024-05-11 14:02:03.272494298 +0000 UTC m=+0.876677621 container remove 29a345b07e8dcea555e59b26d4efc3d9d8875da459df6600ba4b9890424e0df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-11T14:02:03.748 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.b.service' 2024-05-11T14:02:03.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:03.861 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-11T14:02:03.861 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-11T14:02:03.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.e 2024-05-11T14:02:04.440 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 14:02:04 smithi071 systemd[1]: Stopping Ceph mds.e for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:02:04.940 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 14:02:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e[62544]: 2024-05-11T14:02:04.473+0000 7fc216400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:02:04.940 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 14:02:04 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e[62544]: 2024-05-11T14:02:04.473+0000 7fc216400700 -1 mds.e *** got signal Terminated *** 2024-05-11T14:02:06.690 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 14:02:06 smithi071 podman[71740]: 2024-05-11 14:02:06.453164082 +0000 UTC m=+2.098498113 container died b7299ece7f763dbd9c263b5a5ce4bd3c7d54547afe1e7e94cd1411c6127e434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-11T14:02:07.130 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 14:02:06 smithi071 podman[71740]: 2024-05-11 14:02:06.899892492 +0000 UTC m=+2.545226476 container cleanup b7299ece7f763dbd9c263b5a5ce4bd3c7d54547afe1e7e94cd1411c6127e434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-11T14:02:07.130 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 14:02:06 smithi071 bash[71740]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e 2024-05-11T14:02:07.440 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 11 14:02:07 smithi071 podman[71754]: 2024-05-11 14:02:07.142242008 +0000 UTC m=+0.650701088 container remove b7299ece7f763dbd9c263b5a5ce4bd3c7d54547afe1e7e94cd1411c6127e434d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-e, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:02:07.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.e.service' 2024-05-11T14:02:07.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:07.781 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-11T14:02:07.781 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-11T14:02:07.781 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.h 2024-05-11T14:02:08.189 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:07 smithi071 systemd[1]: Stopping Ceph mds.h for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:02:08.690 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h[62856]: 2024-05-11T14:02:08.367+0000 7fefe6e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:02:08.690 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:08 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h[62856]: 2024-05-11T14:02:08.367+0000 7fefe6e00700 -1 mds.h *** got signal Terminated *** 2024-05-11T14:02:10.690 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:10 smithi071 podman[71861]: 2024-05-11 14:02:10.426233824 +0000 UTC m=+2.164871220 container stop 45d2b6c5d1c8db39ad164a0a4064f9101851bd55c8e7c5db7731dc7933024f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-11T14:02:10.690 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:10 smithi071 podman[71861]: 2024-05-11 14:02:10.452834012 +0000 UTC m=+2.191471374 container died 45d2b6c5d1c8db39ad164a0a4064f9101851bd55c8e7c5db7731dc7933024f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-11T14:02:11.073 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:10 smithi071 podman[71861]: 2024-05-11 14:02:10.83595989 +0000 UTC m=+2.574597324 container cleanup 45d2b6c5d1c8db39ad164a0a4064f9101851bd55c8e7c5db7731dc7933024f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-11T14:02:11.073 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:10 smithi071 bash[71861]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h 2024-05-11T14:02:11.323 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.h.service' 2024-05-11T14:02:11.392 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:11 smithi071 podman[71875]: 2024-05-11 14:02:11.086929913 +0000 UTC m=+0.639514467 container remove 45d2b6c5d1c8db39ad164a0a4064f9101851bd55c8e7c5db7731dc7933024f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-11T14:02:11.392 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:11 smithi071 systemd[1]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-11T14:02:11.393 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:11 smithi071 systemd[1]: Stopped Ceph mds.h for a0d7c9ac-0f98-11ef-bc98-c7b262605968. 2024-05-11T14:02:11.393 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 11 14:02:11 smithi071 systemd[1]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.h.service: Consumed 8min 40.260s CPU time. 2024-05-11T14:02:11.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:11.423 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-11T14:02:11.423 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-11T14:02:11.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.k 2024-05-11T14:02:11.689 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 14:02:11 smithi071 systemd[1]: Stopping Ceph mds.k for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:02:12.440 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 14:02:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k[62232]: 2024-05-11T14:02:12.028+0000 7f6808600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:02:12.440 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 14:02:12 smithi071 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k[62232]: 2024-05-11T14:02:12.029+0000 7f6808600700 -1 mds.k *** got signal Terminated *** 2024-05-11T14:02:14.758 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 14:02:14 smithi071 podman[71983]: 2024-05-11 14:02:14.458026447 +0000 UTC m=+2.555690439 container died e4eb76c487caca452cf11df893a80fc4e30e8fa1dfd48de0e836e5df8e87c1b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-11T14:02:15.037 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 14:02:14 smithi071 podman[71983]: 2024-05-11 14:02:14.891340046 +0000 UTC m=+2.989004028 container cleanup e4eb76c487caca452cf11df893a80fc4e30e8fa1dfd48de0e836e5df8e87c1b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-11T14:02:15.037 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 14:02:14 smithi071 bash[71983]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k 2024-05-11T14:02:15.440 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 11 14:02:15 smithi071 podman[71996]: 2024-05-11 14:02:15.050391205 +0000 UTC m=+0.570796316 container remove e4eb76c487caca452cf11df893a80fc4e30e8fa1dfd48de0e836e5df8e87c1b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-11T14:02:15.517 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.k.service' 2024-05-11T14:02:15.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:15.623 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-11T14:02:15.624 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-11T14:02:15.624 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.c 2024-05-11T14:02:16.052 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 14:02:15 smithi165 systemd[1]: Stopping Ceph mds.c for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:02:16.552 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 14:02:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c[63181]: 2024-05-11T14:02:16.198+0000 7feda5600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:02:16.552 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 14:02:16 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c[63181]: 2024-05-11T14:02:16.198+0000 7feda5600700 -1 mds.c *** got signal Terminated *** 2024-05-11T14:02:18.738 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 14:02:18 smithi165 podman[76762]: 2024-05-11 14:02:18.422951268 +0000 UTC m=+2.342114390 container died 84d4b42d326ef755efadf6358a9221e3bc8cc195237a3a78d02d0faeea7f8fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-11T14:02:19.022 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 14:02:18 smithi165 podman[76762]: 2024-05-11 14:02:18.85941852 +0000 UTC m=+2.778581663 container cleanup 84d4b42d326ef755efadf6358a9221e3bc8cc195237a3a78d02d0faeea7f8fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T14:02:19.022 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 14:02:18 smithi165 bash[76762]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c 2024-05-11T14:02:19.302 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 11 14:02:19 smithi165 podman[76788]: 2024-05-11 14:02:19.035508678 +0000 UTC m=+0.605582739 container remove 84d4b42d326ef755efadf6358a9221e3bc8cc195237a3a78d02d0faeea7f8fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-11T14:02:19.509 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.c.service' 2024-05-11T14:02:19.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:19.619 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-11T14:02:19.619 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-11T14:02:19.619 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.f 2024-05-11T14:02:20.051 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:19 smithi165 systemd[1]: Stopping Ceph mds.f for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:02:20.552 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f[62874]: 2024-05-11T14:02:20.202+0000 7f4a55800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:02:20.552 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:20 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f[62874]: 2024-05-11T14:02:20.202+0000 7f4a55800700 -1 mds.f *** got signal Terminated *** 2024-05-11T14:02:22.698 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:22 smithi165 podman[76892]: 2024-05-11 14:02:22.43085128 +0000 UTC m=+2.350754130 container stop dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-11T14:02:22.699 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:22 smithi165 podman[76892]: 2024-05-11 14:02:22.457679605 +0000 UTC m=+2.377582421 container died dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True) 2024-05-11T14:02:23.034 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:22 smithi165 podman[76892]: 2024-05-11 14:02:22.804994244 +0000 UTC m=+2.724897087 container cleanup dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-11T14:02:23.034 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:22 smithi165 bash[76892]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f 2024-05-11T14:02:23.302 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 11 14:02:23 smithi165 podman[76906]: 2024-05-11 14:02:23.047692939 +0000 UTC m=+0.592271789 container remove dfa8e6f556159efb548893e7d0ae9b3a67abaf7c77757a8037940e51948e41c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T14:02:23.602 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.f.service' 2024-05-11T14:02:23.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:23.709 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-11T14:02:23.709 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-11T14:02:23.709 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.i 2024-05-11T14:02:24.276 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 14:02:23 smithi165 systemd[1]: Stopping Ceph mds.i for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:02:24.552 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 14:02:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i[63485]: 2024-05-11T14:02:24.289+0000 7ff51ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:02:24.552 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 14:02:24 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i[63485]: 2024-05-11T14:02:24.289+0000 7ff51ea00700 -1 mds.i *** got signal Terminated *** 2024-05-11T14:02:26.683 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 14:02:26 smithi165 podman[77012]: 2024-05-11 14:02:26.425111815 +0000 UTC m=+2.265418265 container died 7dfe9bca856ce347bc92c6c1091a5e2c33b4464c1f6b097fc05fc8103a7caa22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2) 2024-05-11T14:02:26.979 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 14:02:26 smithi165 podman[77012]: 2024-05-11 14:02:26.808708993 +0000 UTC m=+2.649015472 container cleanup 7dfe9bca856ce347bc92c6c1091a5e2c33b4464c1f6b097fc05fc8103a7caa22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-11T14:02:26.979 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 14:02:26 smithi165 bash[77012]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i 2024-05-11T14:02:27.302 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 11 14:02:26 smithi165 podman[77026]: 2024-05-11 14:02:26.992887858 +0000 UTC m=+0.556950674 container remove 7dfe9bca856ce347bc92c6c1091a5e2c33b4464c1f6b097fc05fc8103a7caa22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T14:02:27.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.i.service' 2024-05-11T14:02:27.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:27.566 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-11T14:02:27.566 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-11T14:02:27.566 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.l 2024-05-11T14:02:27.801 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 14:02:27 smithi165 systemd[1]: Stopping Ceph mds.l for a0d7c9ac-0f98-11ef-bc98-c7b262605968... 2024-05-11T14:02:28.528 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 14:02:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l[62564]: 2024-05-11T14:02:28.186+0000 7f0206800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T14:02:28.529 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 14:02:28 smithi165 ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l[62564]: 2024-05-11T14:02:28.186+0000 7f0206800700 -1 mds.l *** got signal Terminated *** 2024-05-11T14:02:30.747 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 14:02:30 smithi165 podman[77132]: 2024-05-11 14:02:30.435424506 +0000 UTC m=+2.470243732 container died 3a85cb4d9fd15a3cea6a28b8bc03c92e27d813551619eb3d6bdd26094f16a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-11T14:02:31.052 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 14:02:30 smithi165 podman[77132]: 2024-05-11 14:02:30.895796737 +0000 UTC m=+2.930615977 container cleanup 3a85cb4d9fd15a3cea6a28b8bc03c92e27d813551619eb3d6bdd26094f16a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-11T14:02:31.052 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 14:02:30 smithi165 bash[77132]: ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l 2024-05-11T14:02:31.552 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 11 14:02:31 smithi165 podman[77146]: 2024-05-11 14:02:31.113589864 +0000 UTC m=+0.648435078 container remove 3a85cb4d9fd15a3cea6a28b8bc03c92e27d813551619eb3d6bdd26094f16a9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-11T14:02:31.646 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0d7c9ac-0f98-11ef-bc98-c7b262605968@mds.l.service' 2024-05-11T14:02:31.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T14:02:31.750 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-11T14:02:31.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-11T14:02:32.083 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T14:02:37.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-11T14:02:37.622 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T14:02:42.408 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-11T14:02:42.741 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T14:02:47.837 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T14:02:47.934 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T14:02:48.027 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T14:02:48.107 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-11T14:02:48.109 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi063/crash 2024-05-11T14:02:48.110 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash -- . 2024-05-11T14:02:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:tar: /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-11T14:02:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:tar: Error is not recoverable: exiting now 2024-05-11T14:02:48.217 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi071/crash 2024-05-11T14:02:48.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash -- . 2024-05-11T14:02:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-11T14:02:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-05-11T14:02:48.316 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi165/crash 2024-05-11T14:02:48.317 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash -- . 2024-05-11T14:02:48.426 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-11T14:02:48.426 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-05-11T14:02:48.429 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-11T14:02:48.430 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-11T14:02:48.598 INFO:tasks.cephadm:Compressing logs... 2024-05-11T14:02:48.598 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T14:02:48.604 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T14:02:48.612 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T14:02:48.685 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-11T14:02:48.685 INFO:teuthology.orchestra.run.smithi063.stderr:: No such file or directory 2024-05-11T14:02:48.686 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log 2024-05-11T14:02:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-11T14:02:48.688 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-11T14:02:48.689 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-11T14:02:48.690 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-11T14:02:48.691 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-11T14:02:48.692 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.3.log: 87.6% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-11T14:02:48.692 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-11T14:02:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log 2024-05-11T14:02:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:: No such file or directory 2024-05-11T14:02:48.694 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-11T14:02:48.694 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-05-11T14:02:48.695 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log 2024-05-11T14:02:48.695 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-11T14:02:48.697 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-11T14:02:48.697 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-11T14:02:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-11T14:02:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-11T14:02:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-11T14:02:48.698 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-11T14:02:48.698 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-11T14:02:48.698 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-11T14:02:48.698 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-11T14:02:48.702 INFO:teuthology.orchestra.run.smithi063.stderr: 94.4% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-11T14:02:48.702 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-11T14:02:48.702 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-11T14:02:48.702 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-11T14:02:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-11T14:02:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-11T14:02:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-11T14:02:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-11T14:02:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-11T14:02:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-11T14:02:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-11T14:02:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-11T14:02:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-11T14:02:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.11.log: 87.6% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-11T14:02:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-11T14:02:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.5% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-11T14:02:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-11T14:02:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-11T14:02:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-11T14:02:48.712 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.7% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-11T14:02:48.714 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.8% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-11T14:02:48.718 INFO:teuthology.orchestra.run.smithi165.stderr: 84.0% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-11T14:02:48.720 INFO:teuthology.orchestra.run.smithi165.stderr: 90.4% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-11T14:02:48.751 INFO:teuthology.orchestra.run.smithi071.stderr: 84.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-11T14:02:48.757 INFO:teuthology.orchestra.run.smithi063.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-11T14:02:48.817 INFO:teuthology.orchestra.run.smithi063.stderr: 90.9% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-11T14:02:48.913 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-11T14:02:48.915 INFO:teuthology.orchestra.run.smithi071.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-11T14:02:48.930 INFO:teuthology.orchestra.run.smithi063.stderr: 90.2% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-11T14:02:48.945 INFO:teuthology.orchestra.run.smithi071.stderr: 90.4% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-11T14:02:48.949 INFO:teuthology.orchestra.run.smithi071.stderr: 90.9% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-11T14:02:48.975 INFO:teuthology.orchestra.run.smithi063.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-11T14:02:49.052 INFO:teuthology.orchestra.run.smithi063.stderr: 84.2% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-11T14:02:49.076 INFO:teuthology.orchestra.run.smithi165.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-11T14:02:49.093 INFO:teuthology.orchestra.run.smithi165.stderr: 94.4% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-11T14:02:49.183 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-11T14:02:49.419 INFO:teuthology.orchestra.run.smithi165.stderr: 91.7% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-11T14:02:49.515 INFO:teuthology.orchestra.run.smithi165.stderr: 94.6% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-11T14:02:49.573 INFO:teuthology.orchestra.run.smithi165.stderr: 94.8% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-11T14:02:50.811 INFO:teuthology.orchestra.run.smithi063.stderr: 94.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-11T14:02:50.909 INFO:teuthology.orchestra.run.smithi063.stderr: 90.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-11T14:02:51.108 INFO:teuthology.orchestra.run.smithi063.stderr: 93.9% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-11T14:02:51.360 INFO:teuthology.orchestra.run.smithi165.stderr: 95.3% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-11T14:02:52.391 INFO:teuthology.orchestra.run.smithi071.stderr: 92.0% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-11T14:02:56.766 INFO:teuthology.orchestra.run.smithi063.stderr: 90.2% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-11T14:03:28.275 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-11T14:03:38.658 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-11T14:03:43.192 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-11T14:04:08.376 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-11T14:04:08.380 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-05-11T14:04:08.380 INFO:teuthology.orchestra.run.smithi165.stderr:real 1m19.723s 2024-05-11T14:04:08.380 INFO:teuthology.orchestra.run.smithi165.stderr:user 3m27.815s 2024-05-11T14:04:08.380 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m14.868s 2024-05-11T14:06:38.909 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-11T14:06:41.479 INFO:teuthology.orchestra.run.smithi063.stderr: 91.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-11T14:06:45.682 INFO:teuthology.orchestra.run.smithi063.stderr: 91.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-11T14:07:30.957 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-11T14:08:07.855 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-11T14:08:15.373 INFO:teuthology.orchestra.run.smithi063.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-11T14:08:15.395 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-11T14:08:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:real 5m26.747s 2024-05-11T14:08:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:user 9m7.821s 2024-05-11T14:08:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:sys 1m1.549s 2024-05-11T14:09:04.301 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-11T14:09:57.152 INFO:teuthology.orchestra.run.smithi071.stderr: 96.6% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-11T14:09:57.256 INFO:teuthology.orchestra.run.smithi071.stderr: 94.3% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-11T14:11:46.131 INFO:teuthology.orchestra.run.smithi071.stderr: 94.4% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-11T14:12:07.413 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-11T14:12:19.656 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-11T14:12:52.013 INFO:teuthology.orchestra.run.smithi071.stderr: 94.4% -- replaced with /var/log/ceph/a0d7c9ac-0f98-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-11T14:12:52.017 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-05-11T14:12:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:real 10m3.360s 2024-05-11T14:12:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:user 12m52.997s 2024-05-11T14:12:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:sys 1m36.369s 2024-05-11T14:12:52.019 INFO:tasks.cephadm:Archiving logs... 2024-05-11T14:12:52.019 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi063/log 2024-05-11T14:12:52.020 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T14:13:42.840 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi071/log 2024-05-11T14:13:42.842 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T14:14:40.511 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi165/log 2024-05-11T14:14:40.513 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T14:15:15.910 INFO:tasks.cephadm:Removing cluster... 2024-05-11T14:15:15.910 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 --force 2024-05-11T14:15:16.327 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T14:15:17.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 --force 2024-05-11T14:15:18.252 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T14:15:20.213 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0d7c9ac-0f98-11ef-bc98-c7b262605968 --force 2024-05-11T14:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: a0d7c9ac-0f98-11ef-bc98-c7b262605968 2024-05-11T14:15:21.882 INFO:tasks.cephadm:Removing cephadm ... 2024-05-11T14:15:21.882 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-11T14:15:21.944 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-11T14:15:22.005 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-11T14:15:22.062 INFO:tasks.cephadm:Teardown complete 2024-05-11T14:15:22.062 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-11T14:15:22.074 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-11T14:15:22.074 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T14:15:22.082 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T14:15:22.089 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T14:15:22.327 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-11T14:15:22.337 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-11T14:15:22.338 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T14:15:22.346 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T14:15:22.354 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T14:15:22.397 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-11T14:15:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-05-11T14:15:22.407 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-05-11T14:15:22.446 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T14:15:22.446 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-05-11T14:15:22.446 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +902us[ +905us] +/- 54ms 2024-05-11T14:15:22.446 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -1063us[-1061us] +/- 78ms 2024-05-11T14:15:22.447 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -229us[ -227us] +/- 78ms 2024-05-11T14:15:22.447 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T14:15:22.504 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T14:15:22.504 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-11T14:15:22.504 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +902us[ +913us] +/- 54ms 2024-05-11T14:15:22.504 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1066us[-1066us] +/- 78ms 2024-05-11T14:15:22.504 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -215us[ -204us] +/- 78ms 2024-05-11T14:15:22.504 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T14:15:22.521 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T14:15:22.521 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-05-11T14:15:22.521 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +896us[ +889us] +/- 54ms 2024-05-11T14:15:22.521 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -1069us[-1069us] +/- 78ms 2024-05-11T14:15:22.521 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -230us[ -237us] +/- 78ms 2024-05-11T14:15:22.521 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T14:15:22.524 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-11T14:15:22.535 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-11T14:15:22.536 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-11T14:15:22.546 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T14:15:22.825 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T14:15:23.029 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T14:15:23.247 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T14:15:23.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T14:15:23.350 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T14:15:23.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T14:15:23.442 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T14:15:23.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T14:15:23.537 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-11T14:15:23.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-11T14:15:23.562 INFO:teuthology.task.internal:Duration was 4049.024870 seconds 2024-05-11T14:15:23.563 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-11T14:15:23.573 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-11T14:15:23.573 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T14:15:23.581 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T14:15:23.589 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T14:15:23.735 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T14:15:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T14:15:23.741 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T14:15:24.293 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-11T14:15:24.293 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-05-11T14:15:24.293 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T14:15:24.373 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-05-11T14:15:24.373 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T14:15:24.445 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-05-11T14:15:24.445 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T14:15:24.521 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-11T14:15:24.521 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T14:15:24.530 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T14:15:24.537 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T14:15:24.619 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-11T14:15:24.620 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T14:15:24.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T14:15:24.630 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T14:15:25.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-11T14:15:25.793 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-11T14:15:25.794 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T14:15:25.889 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T14:15:25.980 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T14:15:26.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-11T14:15:26.102 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T14:15:26.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T14:15:26.118 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T14:15:26.220 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-05-11T14:15:26.225 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-05-11T14:15:26.233 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-05-11T14:15:26.391 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T14:15:26.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T14:15:26.450 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T14:15:26.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T14:15:26.506 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T14:15:26.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T14:15:26.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-11T14:15:26.575 INFO:teuthology.task.internal:Transferring archived files... 2024-05-11T14:15:26.575 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi063 2024-05-11T14:15:26.576 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T14:15:26.707 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi071 2024-05-11T14:15:26.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T14:15:26.808 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702405/remote/smithi165 2024-05-11T14:15:26.808 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T14:15:26.910 INFO:teuthology.task.internal:Removing archive directory... 2024-05-11T14:15:26.910 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T14:15:26.919 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T14:15:26.927 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T14:15:26.980 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-11T14:15:26.993 INFO:teuthology.task.internal:Not uploading archives. 2024-05-11T14:15:26.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-11T14:15:27.048 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-11T14:15:27.048 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T14:15:27.056 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T14:15:27.064 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T14:15:27.107 INFO:teuthology.orchestra.run.smithi063.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 14:15 /home/ubuntu/cephtest 2024-05-11T14:15:27.114 INFO:teuthology.orchestra.run.smithi071.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 14:15 /home/ubuntu/cephtest 2024-05-11T14:15:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 14:15 /home/ubuntu/cephtest 2024-05-11T14:15:27.124 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-11T14:15:27.136 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-11T14:15:27.165 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 4049.0248696804047 flavor: default mon.a-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e mon.b-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e mon.c-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e owner: scheduled_rishabh@teuthology success: true 2024-05-11T14:15:27.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T14:15:27.261 INFO:teuthology.run:pass